Draft Submission to the 2003 ASEE Conference

Teaching Reflective Skills in an Engineering Course

David Socha Valentin Razmov Elizabeth Davis
UrbanSim Project Dept. of Computer Science & Engr. Davis Consulting
Dept. of Computer Science & Engr. University of Washington
University of Washington

Abstract
One of the most effective tools for lifelong learning is the ability to reflect and learn from past experience.
Reflection helps to clarify our understanding of the world and to create new distinctions and possibilities for the
future. It is a way of creating intention. By putting our attention on the perception of what has happened and
what we want to achieve, solutions to problems emerge more easily. We also believe reflective skills are among
the main characteristics that distinguish excellent engineers from merely good ones. This makes these skills
important to teach.
This paper describes a set of reflective practices that we implemented in a 9-week course in software
engineering at the junior undergraduate level. These techniques, many of them borrowed from professional
leadership training programs, include individual, team, and project practices such as retrospectives (e.g., “What
went well and what didn’t?”), informal chats with guest experts (e.g., “Do they really do it that way in
industry?”), workshop simulations (e.g., “How do we decide when to ship a product?”), journaling, and some
unusual activities (e.g., “Draw a picture of your team”). To gauge student progress we also used weekly
reflective assignments as well as reflective questions on the take-home final exam. All of these techniques were
well received by the students, as evidenced by anonymous, detailed end-of-course evaluations, and by voluntary
feedback students provided four months after the end of the course. The experience applying reflective practices
appears to have influenced a number of the students into viewing their project, careers, social interactions, and
life choices in a different, more positive light. Many have continued using several of these techniques after the
course.
We believe the practices worked particularly well because we set up the course with ample opportunities for
students to make mistakes (a fodder for reflection) and learn from them in a non-threatening (academic)
environment. While we recommend the approach to engineering educators interested in teaching “soft skills,” we
caution that to successfully apply it one needs to be comfortable identifying and handling conflict that may
emerge.

1. Introduction
This paper describes a set of reflective practices that formed the backbone of a 9-week software engineering
course at the junior undergraduate level. We report on our, and our students’, assessments of the effectiveness of
these practices, collected during the course, at the end of the course, and four months after the course.
One of the main goals we set for the course was to teach a set of team and project practices necessary for
doing projects effectively. Our industrial experience1 and assessments like ABET 2000 show that being skilled at
doing these practices is highly valuable for enabling software engineers to be successful. We believe experiential
learning [Weinberg 1985] is one effective way to teach these skills. It requires that students continually go
through a learning cycle whereby they practice, reflect on their difficulties, discover new models or have them
introduced by facilitators or other students, and then practice again.
In order to create an effective experiential learning situation within an academic context, we designed the
course around several strategies:

1
David has 11 years of industrial experience developing software and managing teams of software developers. Valentin has
3 years industrial experience developing software. Elizabeth has over 20 years experience as a family therapist, and thus has
a good understanding of how people “function” and learn.
 1. Project–based, a single large team. To force the students to deal with team and project coordination issues
that could be missed in smaller teams, the course was designed to be project-based with all 22 students
working in a single team for the duration of the quarter.
2. Teaching with our mouths shut. To maximize the chance for student learning, instructors focused on being
facilitators within an experiential learning environment, and on teaching by example. Instructors provided
some project requirements2, choices, observations, facilitation, and minor guidance. After an outside
marketing person presented the marketing requirements for the product, the students had the freedom and
responsibility to run and manage the entire project, including deciding what to do when they do not know
what to do.
3. Reflective practices. To maximize student learning, we embedded this project within a system of many
individual and group reflective practices.
4. Resembling real life. To maximize the practicality of what students would learn, the project was done in
an environment as close to that in industry as possible within the constraints of an academic 5-credit
course. The main exception was that, in the spirit of academia, student learning was emphasized as more
important than project success and there was no real customer3.
5. Expert practitioners. To ground our assessments of what an engineering job demands and to help students
appreciate the diversity of needs of different constituents, we had eight class sessions where outside expert
practitioners led experiential simulations or told reflective stories of their work.
6. Need-for over how-to. To fit within the constraints of a 9-week course, we were more concerned with
teaching need-for than how-to knowledge. The need-for is about appreciating that there is a need for some
type of action, while the how-to is about how specifically to perform that action effectively.

The results were encouraging. We were initially surprised by how many students readily provided us with
feedback four months after the course was over. University-wide statistics show that typically only one or two
students respond to queries after the end of a course. We had a 45% response rate and nearly all students were
very positive about their experience. To us this indicates that they were passionate about how the course was
taught and how significant it was for them personally.
Of the wide range of reflective practices we experimented with, several seem to have made a lasting positive
impression on students. Nearly all of our respondents said that they still continued to write in private journals
regularly and found it rewarding, and more than half of them also practiced team conversations, team
retrospectives, and the awareness of personality types in their interactions with colleagues.
Since the feedback we received was in response to questions about the techniques we applied in the course
and the reflective practices we introduced but not the actual course material, and since nothing about these
practices seems domain-specific, we believe they can be successfully applied in courses in all engineering
disciplines.

Next, in Section 2 we provide some background and lay out our basic approach to designing this course.
Section 3 describes the reflective practices around which the course skeleton was built, while the details of each
appear in Sections 4 through 6. Finally, Section 7 reports the results from student feedback and concludes.

2. Our Approach
Many courses have taught reflective practices [Jolly 2000, Sobek 2002]. There are many different types of
reflective practices. In this course we experimented with a few that stand out as being particularly valuable for a
wide range of people and circumstances. These include journaling, targeted reflective essays [Turns 1997],
portfolios, retrospectives, and story telling.
Journaling helps an individual to clarify what has happened and learn about herself. Targeted reflective
essays help the writer to learn from the exploration of a specific question. Portfolios help to build a sense of

2
We restricted the set of development tools so that (a) students would not spend too much time on that choice, and (b) we
could provide them with an industrial level set of development tools.
3
We would think carefully before having a real customer for such a course, because this would increase the cost of failure,
which in turn may prevent students from taking risks and trying something new. Academia is not the real world, and it may
be best to use that to our advantage by doing things that are not easily done in the real world, such as encouraging students
to focus on learning even if it may lead to project failure.

2
accomplishment as items accumulate [Upchurch 2001]. Retrospectives help teams to clarify what has happened
and learn how to operate better. And story telling helps people to learn from the reflections of others.
The teaching style and structure of the course reported on in this paper were strongly influenced by our
industry experience. Software engineering is about people working in teams on projects to create value for
customers. While tools and the “hard skills” are important for software engineering, projects almost never fail
solely because of technical limitations. Yet, the “soft skills” are largely missing from the curricula most
undergraduates go through, even though their absence does lead to a great many project failures. Our goal
therefore was to plant the seeds that would allow the students in our course to start and stay on the path toward
becoming effective in real-world situations.
In particular, we consider the following five skill domains to be invaluable for effective engineers:
• Reflective skills (for lifelong learning) [Schön 1983]
• Team skills (for working with people) [Dunham 2001]
• Project skills (for navigating projects to success)
• Value skills (for identifying and creating value) [Denning 2001]
• Design skills (for discovering and building good designs)
These “soft skills” are necessary to effectively apply the “hard skills” traditionally taught in engineering
courses [ABET 2000]. While an individual does not need to be proficient in all of these – that is the value of
working in a team – the most effective professionals we know are very good at all of them. At the least, for
someone to be effective in a team environment, they need to appreciate and respect the need for soft skills in
their team.
In some way, reflective skills are more “fundamental” than team, project, value, or design skills. Reflective
skills help one improve in all skill domains, whereas the same is not true for the other types of skills. For this
reason, we put much attention on reflective skills in our course. However, we caution that while they are
necessary, they are not sufficient for lifelong learning in any skill domain.
Unfortunately, a quarter is a very short time to teach such skills in completeness. They require a lot of
practice to master. Furthermore, the “right” way to implement them depends on the context, so even if we had
taught a right way, that way might not have been suitable in the next situation a student faced. Thus, we were
more concerned with teaching students by example to develop an appreciation for these skills than with teaching
the specifics of each skill set. We aimed to create an environment in which students had ample opportunities to
practice and learn from the mistakes they make while practicing. Reflection is an essential component of this
learning process.
We included a variety of individual and group reflective practices and exercises. Individual reflective
practices provide a set of tools that a student can apply in private in order to understand what has happened and
create intention in her life, but that view is often limited by what is possible within that student’s set of stories
about reality. Group reflective practices leverage the diversity of the group stories of what happened and what is
possible in order to create a much richer design experience and understanding; the cost of this comes in the form
of higher overhead to the team.
Providing a range of techniques allowed each student to assess which practices worked best for that student’s
personality and background, as well as for different situations she might encounter. Different people may learn
better through different types of reflective practices. Typical introverts [Keirsey 1998] may prefer the privacy of
a journal to participating in team retrospectives, while extroverts may be more effective the opposite way.
Writing uses a different part of our brain than talking does. Many people think better when they are physically
active [Thompson 2002].
While we experimented with many reflective practices aiming to cover a wide variety of individual learning
styles (our specific choice of techniques being based upon our experience and the limited time in the course),
several of these techniques proved to be valuable to many students.
The next four sections discuss the set of choices we made and the practices we implemented in the course.
They include the background for each of these, our assessments of how well they worked, and the results from
the student feedback.

3. Designing the Course Around Reflection

For students to learn and for their learning to last, they must be engaged in the material. We expected some
resistance to learning “soft skills” because most engineering students had gone through their academic lives
largely by working on their own and so had become accustomed to this style. Thus, we employed several

3
principles to try to awaken the students to the importance of these skills, though in some cases that realization
did not come until late in the quarter (or perhaps not at all). Many of these principles were motivated by trying to
create as corporate an environment as possible within the constraints of a university class in order to maximize
the learning.
The principles behind our course design were:
• Reflective practices require time and practicing.
• Large teams force students to deal with team and project coordination, and thus create material for
reflection.
• Experiential learning provides material for reflection.
• Reflection supports the development of key skills in all domains.
• For students to benefit from reflective writing, it helps to put them into a situation where they have to
struggle enough to collect material for reflection, but not too much [Socha 2003].

From these principles we derived the following major course elements that promote reflective learning:
1. Pre-course setup
a. Increased length of class sessions
b. Availability of both a lecture room and a computer lab
2. During the course
a. Setting the tone in the first class session
b. Journaling
c. Starting and staying on the path to Mastery
d. A single large project team
e. Team conversations
f. Weekly reflective essays
g. Experiential simulations & Expert practitioners telling stories
h. Retrospectives
3. End-of-course closure
a. Personality types
b. Portfolios
c. Peer reviews
d. Final Exam

The following sections describe these elements. Appendix 1 summarizes them and other reflective practices
we experimented with in the course.

4. Pre-course Setup
A couple of elements helped structure the course so that it could better accommodate our mix of project and
experiential simulations.

Increased length of class sessions. The first adjustment came before the course started. David changed the
course from its normal weekly schedule of three one-hour lectures and one one-hour lab to three two-hour
sessions so that we could fit in simulations. A simulation is an experiential learning situation carefully crafted for
a specific learning outcome and controlled by a facilitator [Tener 2001]. We believe a simulation ideally has at
least the following sequence of stages: an introduction, a period of acting in the simulation, a reflective debrief,
another period of acting in the simulation, and a final debrief. One hour does not give sufficient time for this.
Three hours seems to be the ideal length [Smith 2002b] but for practical reasons we settled on 2-hour periods.
Now, given the benefit of hindsight, we suspect that having yet another hour would have further improved the
learning process since the level of interaction and engagement tended to still be rising at end of each 2-hour
simulation session. This observation was also supported by feedback from one of our students.

Availability of both a lecture room and a computer lab. We also arranged for the course to have both a
standard lecture room and a computer lab available for all class sessions. This allowed us the flexibility to
choose the most appropriate environment as needed. Simulations typically benefit from a flexible seating and

4
desk arrangement, while project work benefits from in-class time spent working on the project where instructors
can observe and facilitate.

5. During the Course

This section lists the techniques emphasizing the importance of reflection as an essential component of the
learning process. Most of these were introduced in the first week of the course.

Setting the tone in the first class session. The beginning of a course is an important time. Students are fresh
from the recent break. How the class starts sets the tone that lasts for the duration of the course. The first session
was devoted to the reasons behind the course structure. We began with a discussion using some reflective
questions from Teaching With Your Mouth Shut [Finkel 2000] to uncover the learning cycle model [Weinberg
1985] of experiential learning upon which we based the course. This learning model, very similar to Kolb’s
[Tener 2001], explicitly includes reflection as a necessary component of learning. As we told the students, we
would be doing this over and over again, both in sessions with outside experts facilitating simulations and in the
project itself. The purpose was to put reflection foremost in the students’ attention.
From the beginning and throughout the course we made it clear that the students were the ones expected to
make most of the decisions in this class. The project was theirs to organize and run. We, instructors, largely
taught with our mouths shut, primarily acting as facilitators and providing feedback on homework and on the
state of the project. We would not tell them how to do the project. As expected of people not used to experiential
learning (in an academic setting), this disconcerted many of the students, but they gradually got used to this
freedom and responsibility.
We also outlined the five skill domains listed above and grounded the importance of these with stories from
our own experience in industry as well as using quotes from other industry practitioners.
Of course, these were merely words, and some students did not believe us until the project was well
underway and they had experienced the problems of not using the soft skills. In week six, one student apologized
to David for not having believed him earlier in the quarter.

Journaling. The first session also introduced journaling – a lightweight and particularly effective mechanism for
individual reflection. The act of choosing words to express our thoughts helps to clarify them. Writing in a
private journal is a safe and often effective way to get an insight into a situation, since it tends to reduce the
emotional load and thus makes it easier to understand the issues. It reduces the need to prove something or
defend one’s actions. Instead, it allows one to focus on creating an honest assessment of a situation. In particular,
writing about uncomfortable experiences often reveals that they were not as bad as they felt or uncovers ways to
deal with them. This is less likely to be the outcome of talking to someone else unless that person is both a close
and wise friend.
We required each student to use a physical journal to record her4 reflections during the class. We find many
good reasons for using a physical journal. Writing by hand uses different muscles than typing and thus promotes
different ways of thinking that in turn helps to construct new knowledge. Furthermore, even with the latest
advances in technology, the physical page still remains a much richer5 and more accessible medium. A physical
journal promotes writing down over continual revision, so students work more on the concepts and less on
reformatting the expression of a concept. Finally, physical journals form a more stable historical record.
We wanted the students to write often enough to start seeing the value of journaling, but we could not force
them to write in a journal outside of class. Instead, we dedicated the first and last five minutes of each class
session to this activity. David also emphasized the importance of journaling by purchasing an expensive leather-
bound journal and writing in it while students wrote in theirs.
We instructed students to “write about whatever is important to write about at that moment.” We do not claim
that this is preferable to the more structured approaches other educators have taken but it did not restrict the
students’ own choices and so allowed them to develop their own styles. More importantly, it implied that a
journal could be used for reflections on anything in their lives, not only on course-related issues. We believe this
is essential because school is just one venue for lifelong learning, and a temporary one at that. To us, of higher
importance was to instill the practices that would help these people become and stay lifelong learners.

4
We use feminine pronouns for all students, to preserve their anonymity.
5
For a thought-provoking description of the social aspects of paper see [Gladwell 2002].

5
 One of the difficulties of using experiential learning in a classroom is judging when to leave students alone
and when to intervene. We tended toward allowing them more freedom – something that many appreciated later,
even if it caused them some discomfort at the time. By letting them run the entire project, we allowed each
student to challenge herself as she felt appropriate.
With journals, for instance, we considered whether we should ask to read the students’ journals in order to
provide feedback, or have them be private so that the students could freely express themselves even if their
writings contained criticisms of the instructors.
In the end, we decided in favor of the journals remaining private to the students. We felt this freedom to write
anything without having to think about who would read it was more valuable than the feedback we could have
provided on their journal entries, especially since we were providing feedback on their weekly reflective essays
that they submitted to a public forum. We believed it would increase the chance that they write about what
matters to them in as open a manner as possible, even if these issues had nothing to do with the class. For
instance, one of the students became a “converted” fan of journaling about half way through the course when
writing about her sibling’s problems greatly helped both her and her sibling. She has kept it up ever since and
now, four months later, claims it has changed her life6.
We did not provide example structures to use in reflective writing, such as in the Reflective Learner [Turns
1997] or the Critical Incident Reports used by [Jolly 2000]. We felt that a completely unrestrained forum would
be the best tool to complement other reflective techniques in maximizing student learning.

Starting and staying on the path to Mastery. The first assigned reading was Mastery [Leonard 1991]. We
chose this book because of the value it had provided to us and others we knew, and the emphasis it placed on the
practice necessary for lifelong learning. The book had an important impact on the tone of the course. (Simply
seeing it on the required reading list changed Valentin’s perspective of this course from being “just another
software engineering course” into being something novel and interesting.) The book itself is a profoundly
reflective work, so reading it implicitly reinforced the message about the importance of reflection. Writing
reflective essays on parts of Mastery during the first two weeks forced students to practice reflection themselves,
and our written commentary on each of their writings gave them feedback and served as individual guidance.
Mastery also created a common vocabulary that survived for the duration of the course – something we had not
anticipated but which was a welcome success. Students continually reflected upon concepts from the book, such
as how different situations related to their own “path to mastery,” their tendencies toward being a “dabbler,”
“obsessive,” or “hacker,” and seeing “homeostasis” in action. Many students remarked on the value they got
from this book. As one student told us three months after the class, Mastery helped focus student attention on
reflective techniques.
Not all students appreciated Mastery. Many were eager to get to the “meat” of the course – the project –
which is precisely the reason why we delayed telling them about the project definition, domain, or tool set until
half way through the second week. This gave us the time necessary to first lay the foundation for lifelong
learning and development of team skills.

A single large project team. During the second session we told the students that they all would work together
on a single instance of a project. The motivation was to force them to deal with the coordination and leadership
issues of teams, which would support our goal of teaching them the soft skills that are so crucial for effective
(software) engineers. Small teams, however, can perform well with a much smaller use of formal team and
project skills. In some cases, even if the rest of the team is not performing, a single super-star developer can
“rescue” a small team’s project. With 22 people in a single team, however, that is hardly possible.
The larger team also provided ample material for reflection. It increased the chance for inter-personal
problems as some students assessed that others were not doing their fair share or keeping their promises. Even if
a sub-group of the students worked very well together, they would still have to deal with the rest of the class and
there is a very high chance that someone will get upset at someone else. While this type of conflict is often
viewed as negative, learning to deal with it effectively in a safe academic environment can provide invaluable
lessons before the pressures of industry set in [Socha 2003].
The decision to use a large team disconcerted many of the students. We believe this was partly due to the
common practice of limiting team sizes to at most 5 individuals in virtually all courses taught at our department

6
This is grounded by her changes in career and life direction.

6
and at most other universities, and partly due to students lacking any prior team or project skills, much less
exposure to complex environments with larger teams. The decision, however, was critical for creating a more
authentic environment where the students could explore the coordination skills necessary for many of the team
and project experiences they would encounter in industry. Most students came to realize the value of the large
team. Of the ten people who gave us feedback (on our broad question what we should keep and what we should
change in future versions of this course) four months after the course had ended, five of the six who mentioned
the large project team said they found it valuable. One student said she “especially found working in a large
group environment to be eye-opening and educational.” Another “particularly liked working in a large group. I
found this really taught me the most about group dynamics, and how many people can come together to produce
a large product in a relatively small amount of time.”
After the instructors led a class discussion on potential strategies for effectively organizing many people,
students were given one minute to choose their teams. (An alternative would have been to let them take as long
as they needed to, but we expected them to learn a lot regardless of which team they were in, and we felt that
there were more valuable things to concentrate on especially in light of the very short quarter.) The students
made a decision to organize into five teams of 4-6 students with each team having a lead. Additionally, all team
leads formed a “Lead Team”.

Team conversations. After forming teams, we introduced the ten conversations that Robert Dunham [Dunham
2001] believes constitute a team. He defines a team as a set of ongoing conversations taking place in a
community of people. The claim is that teams must have these conversations in order to be effective, even if the
conversations are done quite informally. This is a view of teams that provides clear guidance on how to establish
teams and keep them healthy, even if it still requires a lot of practice.
One of these conversations is about safety. In order to maximize learning, it is important to provide an
environment where students feel safe enough to take risks and learn from mistakes. The first conversation that
we suggested to the sub-teams was aimed at answering the following two questions: “What must happen for you
to feel safe?” and “What must not happen for you to feel safe?”
Each team spent 30 minutes writing their collective answers onto a large sheet of paper. The entire class then
discussed what different teams had found in order to see the similar yet different needs of the different teams.
Each team’s answers became their established “rules” which the members of that team had agreed to abide by in
order to foster an atmosphere of safety, and by extension creativity, within the team.
However, no inter-team rules were established, which led to misunderstandings as early as the second week
of the project.
Another conversation was about leadership. After it was agreed that teams should have leaders, each team
spent 30 minutes electing its representative. As it quickly became apparent in the weeks to follow, the elected
team leads had little effective authority vested in them by members of their teams, which indicates a failure in at
least one of the conversations. Still, even though their leadership was mostly ineffective – nearly all students felt
that way at the end of the course – the reflective essays team leads wrote indicate that these students were able to
nevertheless learn some important lessons from their experience.

Weekly reflective essays. Writing reflective essays to which instructors provide feedback enhances student
reflective skills and learning [Turns 1997]. In our course, students had to submit two essays per week, each
answering two reflective questions (see below). Instructors returned these with written comments within a week.
These essays also provided feedback to the instructors and were a starting ground for conversations between
instructors and students.
Students submitted their essays to a web site where they were publicly visible. Our hope was that in allowing
public access to this site, students would learn from reading and reflecting on the writings of their peers.
Although we know that some of the students read other students’ writings, we do not know how prevalent or
useful this was.
The questions on the reflective essays were:
1. Describe what you learned in this class during the past week and why it was significant to you.
2. Give an example of your best work from last week and explain why it was a good accomplishment.
We also assigned reading-specific writings. The ones for the Pragmatic Programmer [Hunt 1999] were:
3. What ONE thing will you do during the next week that you would not have done if you had not read this
part of "The Pragmatic Programmer"? Why?

7
 4. Reflect upon ONE specific thing that you did last week. How well did it work? Were you satisfied? Why?
Did it achieve the result you had expected? If not, was the unexpected better than the expected? Why or
why not? What would you do differently to make it more successful next time? Why?
Answers to question #1 provided the most indication of reflection. The examples of best work from the
previous week were often things mentioned in the answer to question #1. Question #2 did not work so well,
since the students almost never kept to what they said they would do in their answers to question #1. We did not
address this problem during the course.

Experiential simulations & Expert practitioners telling stories. During the quarter, industry experts
facilitated eight experiential simulations or interactive sessions where they told reflective stories from their
careers. The simulations were group exercises crafted for a specific type of outcome and facilitated by the
guests. This gave students a chance to practice doing, learning, and reflecting under expert tutelage, and
provided them with some key lessons.
The first simulation, led by David Schmaltz and Amy Schwab [Schwab 2002], addressed the question of
“What do you do when you don’t know what to do?” The facilitators introduced the Satir Change Model
[Weinberg 1997] and then had the students play the “Small Change Game” whose explicit goal was to maximize
a certain metric. After a while, they paused the simulation and asked students if they knew what they were doing.
Most students thought that they did, but as the conversation evolved it became clear that none of them did. What
looked like a simple “success” criterion turned out to be extremely vague. The students could now look back and
see what they were doing when they did not know what they were doing. Many of them reflected upon this in
their later writings. As one student wrote four months later, this question is something that “I’ll always keep with
me.”
In week five Steven Smith led a simulation about “When is a product ready to ship?” [Smith 2002a]. Again,
this was a seemingly simple game that required students to make the tradeoffs as would be expected in industry.
The difference was that it was done in a safe environment where they could learn from their “mistakes.” On one
of the final exam essays one student wrote that “Through my inability to effectively debug ... in Steve Smith’s
marble QA game, I realized and then began making sure I had a reason for doing something before investing
time in doing it.”
When James Bullock came in week seven to do an experiential session, he quickly changed gears when it
became clear that the students were already overloaded with their project work and had more pressing questions
they wished to ask him. He spent the full two hours answering student questions about how to handle certain
problems they were having and whether such problems really happen in industry. This provided a valuable
perspective to help students gauge whether the lessons they were learning were valuable. By that time, students
were fully into reflecting about their project.
Later that week Bjorn Freeman-Benson talked about how engineers create value. Then he randomly chose
students and asked them what their value was to the customer. This caused students some discomfort, because
most of them were not prepared to answer this question. One student wrote about her discomfort when she
“miserably failed to answer the question.” It is precisely this sort of discomfort that prompted several students to
reflect upon this event in that week’s reflective writing.

Retrospectives. Retrospectives [Kerth 2001] are a mechanism to leverage the design capacity of groups to learn
from what happened, strengthen teams, and create intention for what to do in the future. They also help students
to realize how much was accomplished, which brings satisfaction and confidence, and how much was not
accomplished, which brings realism and identifies areas for innovation. We scheduled one 30-minute
retrospective every other week, with full 2-hour retrospectives half way through the course and at the end of it
(see Appendix 2 for our complete course schedule). These retrospectives used a variety of exercises done by
groups or by the entire class. One of the most successful exercises was the Emotions Seismograph [Kerth 2001,
p. 127] where teams drew a chart showing how each team member’s mood (or emotions) changed over a given
period of time. This exercise introduced emotions and moods as legitimate aspects of work, recognizing their
strong influence on individuals and others around them.
Using a variety of group reflective exercises exposed students to different ways of reflecting and creating
intent for the future. Doing group exercises allowed them to see the difference between their individual
reflections and those generated by the entire class. Many of these exercises derived from exercises in the

8
Weinberg & Weinberg leadership courses, Norm Kerth’s book on Project Retrospectives [Kerth 2001], and the
Amplifying Your Effectiveness conferences [AYE].
Students appreciated the variety of the retrospectives, though one student recommended (in the feedback we
solicited four months after the class) that the retrospectives be more focused on how well the prior week’s plan
had been accomplished.

6. End-of-course Closure
We ended the course with several elements designed to bring closure and help students to recognize their
individual and group achievements.

Personality types. Recognizing that other people really do think differently and have different perceptions of
what is “best” makes it easier to respect people. We had not planned to cover personality types until a student
recommended it and by that time it had become clear that many students were struggling with respecting some of
their colleagues. We adjusted the original schedule to cover personality types in the second-last session.
In that session, the students first took an online version of the Myers-Briggs [Keirsey 1998] Personality Type
Indicator test and then proceeded to the classroom to discuss the results. Because of the danger that people might
put too much emphasis on being of a particular type, we were careful to state that the measures reflected current
preferences, not definitions of who the test taker is and always will be. They measure a person’s “strengths” for
that day and in that setting, but can change over time as well as across different contexts (e.g., at home versus at
work).

Portfolios. Portfolios [Olds 2000] are a mechanism for students to gain an appreciation of how much they have
accomplished and can be valuable when students prepare for a job interview. On the last day of class, each
student handed in a hard-copy portfolio of examples of their best work related to the class. Each example had to
have a short description of it. Although we were surprised by the creativity of some of these, it was not clear
how much value students got from this exercise. Perhaps one reason for this is that portfolios were never a
central aspect of the course design.

Peer reviews. Peer reviews are a valuable industry technique for learning how your peers see you. In order to
provide this benefit to the students, and to emulate yet another aspect of industry, we had each student fill out
(on the last day of class) an anonymous peer review of all the students in the class on the last day. (To alleviate
concerns about grading, we had stated that we would not use the results of the peer review for grading purposes
even though we would have those results at that time.) The instructors compiled the results and sent each student
her results after the end of the quarter. Only one student wrote back acknowledging the receipt of her results, so
it is hard to gauge the effectiveness of the practice. We suspect that if the practice had been more tightly
integrated into the course – perhaps performed several times during the quarter – it might have affected student
behavior and caused learning.

Final Exam. The take-home Final Exam consisted of four questions intended to get the students to consolidate
their learning and create intention for the future:
1. Of the things that you did in [this course] which would you do again and why?
2. Of the things that you did in [this course] which would you not do again and why?
3. Describe your current assessment of what a “software engineer” does and how.
4. From the customer’s point of view what value did you add to this project?
In order to help them focus on building their reflective skills and on learning during the quarter, we had told
them the first two questions in the first week of class and had repeated them a few weeks later.
The final exams demonstrated a variety of lessons. Most students claimed they would continue journaling.
Many had thought through their answers to the “value”-question (question #4) on which they had stumbled upon
earlier. Here are some example quotes:
• “These habits made me unproductive ... seeing them when writing the weekly reflections was
demoralizing”
• “That period of prolonged chaos we went through was useful.... That’s where we started asking ourselves
some tough questions, and reevaluating our previous decisions.”
• “The environment [of the class] itself made learning from my classmates easy.”

9
 • “There were a lot of things all of us in the class did that didn’t work so well, but we learned from it, and I
wouldn’t want to change that.”

7. Results and Conclusions

Our goal was to get the students to appreciate the need for “soft skills” so that they would spend the time and
effort honing their people skills. While the reflective essays and final exams gave us some indication of what the
students had learned, these could be biased by their desire for a good grade, especially since they knew that the
practice of reflective techniques was a main factor in determining their grades. Similarly, how they rated various
parts of the course does not necessarily indicate what they learned. To assess what students have actually learned
we asked them the following questions four months after the course had ended:
1. “If we teach this course again, which aspects of it would you recommend that we keep and which ones
may need to be changed? Some things did not go as well as we all wanted. Others went better than we had
expected. What is your take on this?”
2. “Have you found yourself applying some of the lessons you learned in [the class] and if so, which ones (of
the following): journaling, team conversations, team retrospectives, thinking of the path to mastery,
portfolio, personality types, learning cycle, Pragmatic Programmer lessons, Peer Review feedback.”
Eight of the 22 students replied to the second question, while we received 10 responses to the first. The key
points from the feedback are:
• The number of those eight students who have continued using that lesson are: journaling (5), thinking of
the path to mastery (4), team conversations (3), team retrospectives (3), personality types (3), learning
cycle (2), Pragmatic Programmer lessons (2), portfolio (1), Peer Review feedback (1).
• Five of the six people who mentioned the large team believed it was valuable. One student wrote, “I
especially found working in a large group environment to be eye-opening and educational.” Another wrote
“I particularly liked working in a large group.”
• Several students mentioned the value of the experiential simulations. One remarked on how she liked the
“variety of the exercises,” though she would have liked more of them to be closely linked to the project
material.
• Five students said they would have liked more direction from the instructors, though one of these also
mentioned that they had not believed us when we spoke of the need for a detailed project plan and that “If
we hadn’t faltered as we did, I don’t think we would have fully realized the importance and need for these
tools.” We believe it would have helped students to get more examples of good instances of tools for
managing projects, such as different types of project plans. Even so, it is not clear whether they would
have adopted any of those until they were forced to by having failed to achieve the desired results
otherwise.
• As shown above, 5 of the 8 students have continued journaling. One student wrote, “I think that
[journaling] is the most valuable thing I picked up from this class.” Another one with many years of
executive experience said, “I was not in the practice of journaling everyday. I use this now and find it
useful.”
Even when we consider the full-class response on journaling in the end-of-course questionnaire, it
appeared to be the practice that they saw as bringing them the most value: 75% of the students rated
journaling as giving them ‘value’, while 32% qualified it as one of ‘high value’. In the final exams,
journaling was the most commonly mentioned practice students said they would continue.
• Half of the students continued thinking of the path to mastery. One said “I think of the following when I
ran into some difficulties: thinking of the path to mastery, learning cycle.”
• Three of the students continued to use team conversations.
• Several of the students noted the benefit of the challenges they faced. “I absolutely enjoyed the class better
because of the difficulties we had. It made the project much more challenging, and it made it feel like a
much bigger accomplishment when we were finished.”

In addition, we have received unsolicited emails from students. In one of them a student had the following to
say about the impact of this course on her career: “The team conversations and retrospective methods have been
highly effective in building strong and creative communications within the team... Nevertheless, the tool I have

10
found to be the most effective has been journaling, it has helped me to manage and track my learning
experiences, converting goals into reality and ideas into actions.”
From their reflective essays and the post-course feedback it appears we were successful in teaching reflection
and in raising the importance of team and project practices. As one student put it: “This simulated corporate
scenario in a controlled environment was the best learning experience in all my 4 years at the UW. More
important than learning out of a book, I learned about myself.”

Acknowledgments
We thank the many colleagues who helped us design and deliver this course: Jonathan Bach, Bjorn Freeman-
Benson, James Bullock, George Engelbeck, Vibha Sazawal, David Schmaltz, Amy Schwab, Steve Smith, Stani
Vlasseva, Skip Walter, and Sharon Whiting. This research was supported in part by NSF Grant numbers EIA-
0090832 and EIA-0121326.

References
[ABET 2000] http://www.abet.org/
[AYE] http://www.ayeconference.com
[Denning 2001] Peter J. Denning, Robert Dunham. The Core of the Third-Wave Professional. CACM 44(11):
21-25, 2001.
[Dunham 2001] Robert Dunham. Constitutive Conversations of Teams, Handout in the Action In Management
course, Enterprise Design, Inc., 2001.
[Finkel 2000] Donald L. Finkel. Teaching With Your Mouth Shut. Boynton/Cook Publishers, Inc., Portsmouth,
NJ, 2000.
[Gladwell 2002] Malcolm Gladwell. The Social Life of Paper: Looking for Method in the Mess. The New
Yorker, http://www.newyorker.com/critics/books/?020325crbo_books, 2002.
[Hunt 1999] Andy Hunt and Dave Thomas. The Pragmatic Programmer: From Journeyman to Master.
Addison-Wesley Pub. Co., 1999.
[Jolly 2000] Lesley Jolly and David Radcliffe. Strategies for Developing Reflexive Habits in Students. In Proc.
American Society for Engineering Education (ASEE) Annual Conference and Exposition, 2000.
[Keirsey 1998] David Keirsey. Please Understand Me II. Prometheus Nemesis Book Co., 1998.
[Leonard 1991] George Leonard. Mastery: The Keys to Success and Long-Term Fulfillment. Plume Publishing
House, 1991.
[Olds 2000] Barbara M. Olds. Reflection as an Assessment Measure. In Proc. American Society for Engineering
Education (ASEE) Annual Conference and Exposition, 2000.
[Schmaltz 2002] David A. Schmaltz and Amy Schwab. The Small Change Game. True North pgs, Inc.,
http://www.projectcommunity.com, 2002.
[Schön 1983] Donald A. Schön. The Reflective Practitioner: How Professionals Think in Action. Basic Books,
NY, 1983.
[Schwab 2002] Amy Schwab. The Small Change Game. True North pgs, Inc.,
http://www.projectcommunity.com, 2002.
[Smith 2002a] Steven M. Smith and Gerald M. Weinberg. Exploring Tradeoffs: Quality versus Speed.
Simulation at the 3rd Annual Amplifying Your Effectiveness (AYE) Conference,
http://www.ayeconference.com (wiki), 2002.
[Smith 2002b] Steven M. Smith. Personal communication, 2002.
[Sobek 2002] Durward K. Sobek II. Use of Journals to Evaluate Student Design Processes. In Proc. American
Society for Engineering Education (ASEE) Annual Conference and Exposition, 2002.
[Socha 2003] David Socha, Valentin Razmov, and Elizabeth Davis. When Conflict Helps Learning. In
submission.

11
[Tener 2001] Robert K. Tener, Michael T. Winstead, Edward J. Smaglik. Experiential Learning from Internships
in Construction Engineering. In Proc. American Society for Engineering Education (ASEE) Annual
Conference and Exposition, 2001.
[Thompson 2002] Kenneth L. Thompson. Learning as a Biological Process. Pegasus Communications, Inc.,
http://www.pegasuscom.com/levpoints/learnbio.html, 2002.
[Turns 1997] Jennifer Turns, Wendy Newstetter, Janet K. Allen, and Farrokh Mistree. Learning Essays and the
Reflective Learner: Supporting Reflection in Engineering Design Education. In Proc. American Society
for Engineering Education (ASEE) Annual Conference and Exposition, 1997.
[Upchurch 2001] Richard L. Upchurch, Judith E. Sims-Knight. The Learning Portal. In Proc. American Society
for Engineering Education (ASEE) Annual Conference and Exposition, 2001.
[Weinberg 1985] Daniela Weinberg and Gerald M. Weinberg. Learning by Design: Constructing Experiential
Learning Programs. Readings for Problem Solving Leadership, Weinberg and Weinberg, 1985.
[Weinberg 1997] Gerald M. Weinberg. Quality Software Management: Anticipating Change (Vol. 4). New York,
Dorset House Publishing, 1997.

12
Appendix 1. List of Reflective Practices
Table 1: List of reflective practices and techniques we used in the course
Technique Brief Description Reflective Purpose
TWYMS test7 • What were your most significant learning • Gets students to reflect in first class
experiences in life? session
• Was an instructor directly involved?
Learning cycle • Do; reflect; introduce new models; do • Identifies reflection as an integral part
again of learning
• Discourages looking for quick solutions
Journaling • Students wrote in journals for the first and • Builds the habit and discipline for
last five minutes of each class session practicing
• Promotes individual reflection
Team • 10 types of conversations that effective • Emphasizes the importance of teams
conversations teams regularly have • Provides tools for running teams
Safety exercise • What must happen for you to feel safe? • Gets students to do a group reflective
• What must not happen for you to feel safe? exercise
Teaching with • Instructors created an environment for • Provokes students to not fear the
our mouths shut experiential learning, used facilitation, and difficulties and practice
avoided lecturing • Forces students to make their own
choices and construct their knowledge
Emphasis on • Instructors made it clear that they were • Fosters a culture of lifelong learning
learning most interested in student learning
• Reflective exercises were a major grading
component
Staying on the • Students read the Mastery book and wrote • Gets students to reflect upon
path to mastery about it o their learning style
o how mastery relates to the subject
matter (software engineering)
• Connects reflection to something bigger
(the student’s path to mastery)
Reflective • Weekly 1-page essays directed by • Forces practicing reflection
writings questions • Helps instructors gauge learning and
• A take-home Final Exam guide students individually
Simulations • Experiential workshops each aimed at a • Forces practicing reflection under the
specific learning outcome guidance of an expert facilitator
• Demonstrates how the learning cycle
works
• Creates material for reflection
Operating as • All 22 students worked together on a • Provides reflective material by forcing
one large team single project students to deal with team and project
with sub-teams coordination
Project • Group reflective exercises • Gives students practice in a variety of
retrospectives group reflective exercises
Stories from • Sessions where outside industry experts • Exposes students to many reflective
expert told work-related stories stories in an interactive setting
practitioners

7
A set of questions drawn from Teaching With Your Mouth Shut [Finkel 2000]

13
 Technique Brief Description Reflective Purpose
Personality type • Took and discussed the Myers-Briggs • Promotes reflection on how a
test personality type test personality type changes one’s
perceptions and actions
Individual • Each student prepared an end-of-term • Forces a reflection on the value of the
student portfolio of her best work individual’s contribution to the team
portfolios and on the learning that the experience
led to
Challenging • Instructors provided assessments of • Provokes students to reflect
students student and project performance, including
challenging students’ own assessments
Individual • Instructors provided ample individual • Provides material for reflection
feedback on feedback on reflective essays • Helps teach techniques for reflective
student writings writing
Openness of • Weekly student essays were stored • Enables learning from peer reflections
student writings digitally in a public-domain site
Public • Presentation #1: Each team presented the • Forces students to objectively evaluate
presentations architecture of their respective component their product and put those assessments
• Presentation #2: Final official customer in the perspective of what brings value
presentation to instructors and an outside to the customer
marketing person

14
Appendix 2. Our Class Calendar

Week Monday Wednesday Friday

Week Welcome Join company Action Meeting
1 Why are we here? Form teams Team conversations
Retrospective (last 30 min)
Week Workshop: Managing Change Action Meeting Workshop: Use cases
2 Meet a Marketing person Retrospective (last 30 min)
Introduce project & development Due: project plan v.1 at 5pm
system
Week Workshop: Supporting Work in Action Meeting Finish architecture spec
3 Distributed Organizations Work on project Retrospective (last 30 min)
Due: project plan v.2 at 5pm Due: project plan v.3 at 5pm Due: project plan v.4 at 5pm (arch
+ schedule)
Week Workshop: Testing is Science Workshop: About Business Action Meeting
4 Work on project
Retrospective (last 30 min)
Week Workshop: When to ship Action Meeting Midterm Retrospective
5 Work on project

Week Work on project Action Meeting In-class Team Presentations

6 Work on project Retrospective (last 30 min)

Week Workshop: Development Action Meeting Guest Retrospective (first 30 min)

7 Systems Work on project Work on projects

Week Work on project Action Meeting Practice customer presentation

8 Work on project Retrospective (last 30 min)
Due: project CD-ROM at 5pm
Week Customer presentation and Personality Types Peer appreciation
9 project demo Fill in peer evaluations
Customer feedback Due: Portfolios, take-home final
exam, questionnaire

15