Team Collaboration for Command and Control:

A Critical Thinking Model

Jared T. Freeman, Ph.D. Daniel Serfaty

Aptima, Inc. Aptima, Inc.
1030 15th St., Suite 400 12 Gill St., Suite 1400
Washington, DC 20005 Woburn, MA 01801
[email protected] [email protected]

Abstract
We define an initial framework for understanding, measuring, and managing one form of
collaboration: team critical thinking. The framework will be used to understand how team
members critique and refine team performance, develop measures of performance, and
eventually to create training and decision aids that support this form of collaboration. The
framework leverages recent research and theory concerning individual critical thinking,
teamwork, and information-age warfare. A sample of measures is presented.

with computer-mediated collaboration revealed

Introduction that computer support for collaboration
Great attention has been paid to the decreased group effectiveness, increased time to
advanced weapons, communications networks, task completion, and decreased team member
and information systems used in modern battles satisfaction (Baltes, Dickson, Sherman, Bauer,
and experiments. To make effective use of these and LaGanke, 2002). A meta-analysis by Dennis
resources, however, warfighters require and Wixom (2001) showed that decision quality
effective collaboration tools and practices. In was lower in virtual teams using collaboration
this sense, collaboration is a crucial element of technologies than for co-present teams, largely
information age warfare. Analyses of a recent as a function of lower process facilitation in the
Future Joint Forces (FJF) experiment suggest virtual condition. There is clearly room for
that warfighters recognize the importance of significant improvement in the development and
collaboration technologies, and clearly use of collaboration techniques and the tools
distinguish good technologies from bad that support them.
(Freeman, et al., 2002). These data have led FJF To detect these faults and support
leadership to argue that improving collaboration collaboration better, we need to measure
tools and their use may provide a better return collaboration, refine the technologies and
on investment for DOD than more common techniques, then measure and refine again and
hardware acquisition programs (Brooks, again. To develop valid, reliable, and useful
Institute for Defense Analysis, personal measures, we must understand collaboration.
communication, March 2002). However, collaboration has proved to be a very
However, systems in military experiments subtle phenomenon, even when tools are
and the commercial sector illustrate that provided to constrain and shape it. Shirany,
collaboration tools and practices are often Tafti, and Affisco (1999) found that the success
awkwardly implemented and their use is poorly of collaboration techniques varies with task
understood. A meta-analysis of laboratory demands. They found, for example, that
studies comparing face-to-face collaboration experimental teams using group support systems
generated more ideas, while teams using basic validation of a closely related model of
email performed deeper analyses of the knowledge management, based on field
problems (as indicated by a greater proportion observations at a Marine intelligence operation
of inferences drawn). Maznevski and Chudoba and re-analyzed field data from an analysis of
(2000) found that effective teams addressed Virtual Information Center (VIC) at The Pacific
such issues by implementing strategies for Command (PACOM).
selecting face-to-face or remote communication Research concerning critical thinking has
as a function of the task at hand, and by validated a theory of how individual warfighters
developing a rhythm of communications that make decisions under uncertainty. According to
interspersed face-to-face collaboration the recognition-metacognition framework
interspersed with remote communication. (Cohen, Freeman, and Thompson, 1998; Cohen
The inherent complexity of collaboration and Freeman, 1997; Freeman, Cohen, and
has produced communities of research, under Thompson, 1998), expert warfighters (1)
flags such as Situated Cognition and Computer monitor for opportunities to critique their
Supported Cooperative Work. However, there is assessments and plans, (2) identify sources of
not yet a coherent body of theory concerning uncertainty (specifically gaps, untested
collaboration. There is, for example, no assumptions, and conflicting interpretations),
generally accepted mapping of team attributes to and (3) reduce or shift that uncertainty by
technology and tasks (Christensen and gathering information, testing assumptions,
Fjermestad, 1997), and no solution to the forming contingency plans, etc., before taking
smaller problem of fitting collaboration action.
technology to collaborative tasks (Zigurs and Research conducted under ONR’s A2C2
Buckland, 1998). Accordingly, there is little to program and other Air Force Research
guide analysis of collaboration and its Laboratories human engineering projects has
measurement generally, and little to inform its produced a rich body of measures concerning
application within the domain command and the processes by which teams coordinate their
control in particular. Yet, collaboration theory activities explicitly, that is, through explicit
and measures are crucial if we are to achieve the communication, and implicitly, through reliance
full promise of information age warfare. on shared information, shared interpretations of
We are initiating a program of research to information patterns, and standardized
focus on one, crucial type of collaboration, responses to those patterns (MacMillan, Entin,
which we call team collaboration in critical & Serfaty, in press; Macmillan, et al., 2001;
thinking (TC2T). In this program, we are Miller, Price, Entin, & Rubineau, 2001; Moon,
drawing on three relevant research threads: et al., 2000). In addition, this research has
Recent theory and fieldwork concerning refined the methods of model-based
individual and collaborative cognition in measurement and model-based experimentation,
command and control and other information- in which experimental hypotheses are developed
intensive organizations has provided insights from models, measures are built to
into the processes by which team members may operationalize key variables, hypotheses are
interpret data to develop information, build tested in models, then tested empirically, and
understanding that informs decisions, and the results are used to refine models for the next
collaborate to ensure that information and round of hypothesis generation, measure
knowledge are shared in support of construction, and testing.
synchronized action to shape events. Alberts, Our approach is to weave together these
Garstka, Hayes, and Signori (2001) have three research threads – collaborative cognition,
developed a framework that clearly defines individual critical thinking, and model-based
these (italicized) constructs as primitives of measurement of team performance – to develop
performance in information age warfare. a model and validated measures that help us to
Freeman, et al. (2000) conducted initial understand team critical thinking, and
eventually to predict and manage this form of To define the processes of collaborative
collaboration. critical thinking, we turn to the recognition-
metacognition framework of critical thinking
The TC2T Model (above) proposed and validated by Cohen, et al.
We leverage the work on collaborative (1997, 1998) in studies of critical thinking by
cognition (above) by extending the framework individual warfighters. Team members monitor
proposed by Alberts, et al. (2001) to draft a the state of the mission and the team to
TC2T model. That framework specifies that discriminate those problems that are
information age warfare involves events in three satisfactorily addressed with rapid, recognitional
domains (represented in the diagram by three decision-making (c.f., Klein, 1993), from those
strata): the physical domain of actions and that require more effortful (and time-
events; the information domain in which events consuming) critiques. Monitoring triggers a
are observed and communicated; and the critique of understanding when (1) uncertainty
cognitive domain in which the warfighter is high, (2) stakes warrant high confidence or
develops understanding in support of decisions accuracy, and (3) time is available for a critique.
that determine further actions. The flow is That critique should uncover gaps in
illustrated in the left portion of Figure 1. knowledge, conflicting interpretations of the
Collaboration between warfighters is evidence at hand, and/or untested assumptions.
represented by overlap or intersections between It may foster decisions and subsequent actions
two of these diagrams. Several points of to passively wait out the development of the
intersection or linkage are possible (but not situation, or actively shape the battle to acquire
illustrated here). Shared (or linked) information or advantage.
understanding represents common situation A collaborative extension of this model
understanding and expectations. Coordinated addresses a number of new issues. For example,
decisions constitute plans. Coordinated actions it:
denote synchronization. In Figure 1, we have • Acknowledges and specifies the role of
overlaid a node and links to represent environmental enables to collaboration,
collaborative critical thinking, a special case of such as technologies, team processes, and
collaboration. Collaborative critical thinking team architecture. These enablers make
resides partly in the information domain because topics of TC2T more (or less) salient and
it consists of practices, technologies, team actionable;
architectures, and other enablers that support the • Addresses the ways in which team
sharing of information. It resides partly in the members raise and resolve issues
cognitive domain because the content that is deserving of critical thought across the
shared and critiqued includes individual team;
understanding and decisions as well as • Defines how team members coordinate
observations of effects1. their actions to resolve or shift uncertainty,
by filling gaps in knowledge, resolving
conflicting assessments, and testing or
1
replacing assumptions and hypotheses;
The notion that cognition exists outside the mind of
the individual stretches traditional definitions of the
• Define the ways in which teams
term. The case for interpreting cognition in this disseminate the results of their
manner has been well argued by Hutchins (1995a), collaboration and monitor their impact.
who demonstrated that cognitive acts are afforded
and constrained by external entities and objects. • Defines planning processes by which
Thus, shipboard navigation is seen as a cognitive members identify the issues that should be
process that results from coordination between a team monitored and the range of acceptable
of people and by their use of navigation instruments.
Hollan et al., (2000) has elaborated this view, arguing uncertainty or deviation from a norm;
that cognition unfolds not just by the manipulation of Analytical and empirical studies under
symbols within the mind of an individual, but through
interpersonal exchanges and manipulation of objects. ONR’s A2C2 research program and related Air
Force Research Laboratories projects have Acknowledgements
produced a rich understanding of the Aspects of this work were funded by the
relationship between team process, structures, Office of Naval Research. We are obliged to the
and outcomes, and methods of measuring all of men and women of the Navy and Marine Corps
them. These measures address a host of who participated in interviews bearing on
processes relevant to collaboration including collaboration in information-intensive
information sharing, information transfer via environments. The opinions expressed here are
push and pull, specific measures of coordination the authors' and do not necessarily reflect the
communications, negotiation over shared assets, views of the U.S. Navy or the Department of
task delays due to resource contention, etc. The Defense.
program has also produced useful measures of
the effects of collaboration, such as level of References
synchronization in time and coordination of
asset allocation relative to the threat (e.g., the Alberts, David S., Garstka, J.J., Hayes, R.E., and
Signori, D.A. (2001). Understanding Information
value of munitions relative to the value of the
Age Warfare. www.dodccrp.org.
threat). These and other measures from the Baltes, B. B., Dickson, M. W., Sherman, M. P.
literature form the foundation of a basic Bauer, C. C., & LaGanke, J. S. (2002).
measurement system that addresses four issues: Computer-mediated communication and group
• Collaboration Technology Measures of decision making: A meta-analysis.
Performance (MOPs) address the impact Organizational Behavior and Human Decision
of selected technologies on the occurrence Processes, 87, pp. 156-179.
and quality of TC2T. Christensen, E. W., & Fjermestad, J. (1997).
• TC2T MOPs measure the occurrence, Challenging group support systems research:
latency, and quality of collaborative The case for strategic decision-making. Group
Decision and Negotiation, 6, 351-372.
critical thinking.
Cohen, M.S. & Freeman, J.T. (1997). Improving
• C2 MOPs assess the impact of TC2T on Critical Thinking. In Flin, R., et al. (eds.),
team information management, shared Decision Making Under Stress: Emerging
understanding, coordination, and decision- Themes and Applications. Brookfield, VT:
making concerning the C2 environment. Ashgate Publishing Co.
• Measures of Effectiveness (MOEs) tap the Cohen, M.S., Freeman, J.T. and Thompson, B.T.
impact of TC2T on battlefield events, team (1998). Critical Thinking Skills in Tactical
structure, and team process. Decision Making: A Model and A Training
The relationship of the measures to the Method. (Canon-Bowers, J. and E. Salas, eds.),
Decision-Making Under Stress: Implications for
constructs in the draft model is illustrated in
Training & Simulation. Washington, DC:
Figure 2. Representative measures are American Psychological Association
documented in Table 1. Publications.
Dennis, A.R., & Wixom, B. H. (2001).
Summary Investigating the moderators of group support
In sum, we are developing a framework for systems use with meta-analysis. Journal of
understanding team collaboration in critical Management Information Systems, 18, 235-257.
thinking. This work draws on several extant Freeman, J. Thompson, B., Littleton, E.B., Craig, P.,
threads of research and is intended to produce Rubineau, B., Bailin, S., Serfaty, D., and Cohen,
useful measures of this phenomenon and, in M.S. (2000). Metrics for Evaluation of Cognitive
Architecture-Based Collaboration Tools. Aptima
time, tools for training and supporting the
Technical Report AP-R-1119. Woburn, MA.
application of this skill in C2 environments. Freeman, J., Cohen, M.S., and Thompson, B.T.
(1998). Effects of Decision Support Technology
and Training on Tactical Decision Making.
Proceedings of the 1998 Command and Control
 Research & Technology Symposium, Monterey, Moon, H., Hollenbeck, J., Ilgen, D., West, B.,
CA Ellis, A., Humphrey, S., Porter, A. (2000).
Freeman, J., Entin, E., Serfaty, D., Gray, J., Asymmetry in structure movement:
Linegang, M., and Morley, R. (2002). Analyses Challenges on the road to adaptive
of Organizational Issues, Decision Making and
organization structures. In Proceedings of the
Human Factors at the Future Joint Forces
Experiment 1. Woburn, MA: Aptima, AP-R-
CCRT Symposium 2000, Monterey, CA.
Shirani, A. I., Tafti, M. H., & Affisco, J. F. (1999).
1169.
Task and technology fit: a comparison of two
Hollan, J.D., Hutchins, E.L., and Kirsh, 2000.
technologies for synchronous group
Distributed cognition: A new theoretical
communication. Information & Management,
foundation for human-computer interaction
36, 139-150.
research. ACM Transactions on Human-
Zigurs, I., & Buckland, B. K. (1998). A theory of
Computer Interaction.
task/technology fit and group support systems
Hutchins, E. (1995b). How a cockpit remembers its
effectiveness. MIS Quarterly, 22, 313-334.
speed. Cognitive Science, 19:265--288.
Klein, G. A. (1993). A recognition primed decision,
RPD, model of rapid decision making, in Klein
G. A., Orasanu, O., Calderwood, R. and
Zsambok E. (Eds.) Decision making in action:
Models and methods, Ablex Publishing Corp.,
139-47.
MacMillan, J., Entin, E.E., & Serfaty, D. (in
press). A Framework for understanding the
relationship between team structure and the
communication necessary for effective team
cognition. In E. Salas, S.M. Fiore, J. Cannon-
Bowers,(Eds.) Team Cognition: Process and
Performance at the Inter- and Intra-
Individual Level Washington, DC: American
Psychological Association.
MacMillan, J., Paley, M.J., Levchuk, Y.N.,
Entin, E.E., Freeman, J. & Serfaty, D.
(2001), Designing the Best Team for the
Task: Optimal Organizational Structures for
Military Missions. In McNeese, M., Salas,
E. & Endsley, M. (Eds.) New Trends in
Cooperative Activities. Santa Monica, CA:
Human Factors and Ergonomics Society
Press.
Maznevski, M. L., & Chudoba, K. M. (2000).
Bridging space over time: Global virtual team
dynamics and effectiveness. Organization
Science, 11, pp. 473-492.
Miller, D., Price, J., Entin, E., Rubineau, B.
(2001). Does Planning Using Groupware
Foster Coordinated Team Performance? In
Proceedings of the Human Factors and
Ergonomics Society 45th Annual Meeting,
October 2001, Minneapolis, Minnesota:
Human Factors Society.
 Team Member #1 Team Member #2
Cognitive domain Cognitive domain
Content:
Situation
assessments &
Decision Decision
plans
Team process &
structure
Understanding Understanding
Collaborative
Critical Thinking
Information domain Information domain
Enablers:
Practices
Observation Technologies Observation
Team structure &
process

Physical domain Physical domain

Effects Effects

Actions Actions

Figure 1: A preliminary model of Team Collaboration for Critical Thinking.

Cognitive domain
C2 MOPs

Decision Improved command and control

(understanding and decision-making)

Understanding
Supports
TC2T MOPs

Team collaboration in critical thinking

Information domain
Drives
Technology
Observation Enables
MOPs
Collaboration technology

Physical domain

Effects
Effects & outcomes

MOEs
Actions Synchronized actions

Figure 2: The relationship of four types of measures to the model.

Table 1: Representative measures
Type of measure Measure Definition Source
Reach: Team Interconnectivity of team members to each other
connectivity (“Communities of interest”) in Phase I report
Alberts, et al. (2001)
Reach: Information Interconnectivity of team members to information
connectivity sources (“Information access”) in Phase I report
Collaboration Reach: Translation Degree to which tool supports translation between
technology MOPs representations, domains, and languages
Richness: Structured Degree to which the tool requires structured
problem representation representations of the problem at hand Freeman, et al. (2000)
Richness: Deconfliction Degree to which the tool supports coordination of (Phase I report)
activities via a shared workspace
Planning for TC2T Collaboration communications (e.g., paraphrasing
MacMillan, Entin, &
others, explicit statement of monitoring plans &
Serfaty (in press)
critieria) during planning stage.
Monitoring Number of critiques initiated concerning high priority
2 issues
TC T MOPs
Diagnosis Number of gaps, conflicts, and untested assumptions
Cohen, Freeman, and
identified
Thompson (1998)
Action Instances of probing own resources for data, testing
enemy or environment for data, intentionally waiting
out problem
Shared situation Degree to which team members share memory for
awareness current location of objects in the tactical picture
Endsley, 1988
Shared predictions of Degree to which team members share predictions of
future situation the location of objects in the tactical picture
Shared situation Degree to which team members share assessments of Cohen, Freeman, and
assessment the intent of entities of operational interest Thompson (1998)
Mutual awareness of Degree to which team members express shared goals
C2 MOPs n.a.
goals
Mutual awareness of Degree to which team members anticipate the
information needs information needs of teammates
Freeman, Entin, et al.
Mutual awareness of next Degree to which team members anticipate the actions
(2002); Entin & Entin
action(s) of teammates
(2000)
Mutual awareness of Accuracy with which team members estimate the
workload subjective workload of teammates
Synchronization: Proportion of tasks executed without conflicts in
Diedrich, et al., (2002)
Allocation resource allocation
Synchronization: Proportion of tasks requiring coordination that are MacMillan, Entin, &
Execution successfully executed Serfaty (in press)
Synchronization: Lag in readiness for execution of synchronized events
Diedrich, et al., (2002)
Precision between first and last operator ready to act.
MOEs
Effects: Enemy losses Proportion of targets or threats destroyed
Traditional
Effects: Friendly losses Proportion of friendly forces destroyed or lost
Effects: Delay in enemy Latency in enemy maneuvers or actions (e.g.,
Traditional
ops bombing targets) due to friendly actions
Effects: Team process Average delay in task execution relative to optimal MacMillan, Paley, et
task execution schedule al. (2001)