To an ever greater extent, technology is explicitly designed
to enhance human cognitive powers or to compensate for human
cognitive weaknesses. To do this succesffully, designers must
have a good understanding of what those strengths and weaknesses
are. CTI staff have contributed significantly to the research
literature on cognitive models of thinking and decision making,
and on the practical use of such models for cognitive task
analysis, training, and decision aiding.
CTI has conducted empirical studies of real-world
decision making in domains ranging from commercial airline
pilots to Army battlefield commanders and Navy ship captains.
These studies have involved controlled experiments, simulator
studies, real-time operational observations, and intensive
critical incident interviews.
We have developed a number of techniques for
analyzing and understanding the knowledge and decision making
skills exemplified in these performances. CTI has developed
and applied innovative methods for for eliciting and representing
mental models,
or domain knowledge. CTI staff developed a knowledge elicitation
method called Cognitive Structure Analysis which systematically
and efficiently probes for different classes of knowledge
representation. More recently, CTI has done extensive work
on techniques for automated
knowledge acquisition. THis work has utilized Latent
Semantic Analysis and Discriminant Analysis to identify
knowlegde structures underlying expertise.
CTI's training and decision aiding applications draw on general
principles that we have learned from our experience
applying these techniques in many domains. In additon, new
applications always draw on intensive investigation of the
cognitive skills demanded for success in the new problem domain.
Comparison of the performance of more and less proficient
decision makers has led us to formulate a model of critical
thinking in real world domains. The
Recognition / Metacognition Model emphasizes both fast
recognition of familiar situations and more reflective strategies
for detecting and resolving potential problems in a current
situation picture or plan. The R/M model is consistent with
a wide research literature including CTI's own empirical research.
On the normative side, a theoretical basis is provided for
it by a broader
theory of critical thinking recently developed by CTI,
as a process of constructing mental models about possible
situations by asking and answering questions, in order to
achieve a purpose. The R/M model has been used as a guide
for both training
and decision aiding, and a
computational
architecture has been implemented for purposes of simulation
and adaptive training.
Decision Making Under Uncertainty and Time Stress
(Naval Air Warfare Center / Training Systems Division, Orlando,
FL)
CTI has conducted a series of projects within the TADMUS
program. These included:
Studies of individual decision making and development
of individual training for critical thinking,
Studies of team decision making and development of team
training for critical thinking, and
Design of decision support system modules for critical
thinking along with design of training to facilitate use
of those modules.
In its research on individual and team decision making, CTI
has developed models of knowledge structures and decision
making strategies of Naval officers in anti-air and anti-surface
warfare. Interview data have been analyzed and modeled to
understand how decision making succeeds and fails.
Situation Assessment, Mental Models, & Critical Thinking
(Army Research Institute, Ft. Leavenworth Field Unit, KA)
CTI has also conducted a series of projects for the Army
Resesarch Institute. These have included:
Modeling Army command staff situaiton assessment sKills
and development of training to improve those skills,
Capturing the knowledge structures (or mental models)
utiliized to understand the battlefield and make decisions,
and developing training to improve critical thinking in
the use of knowledge, and
Developing a comprehensive framework for Army research
on critical thinking, including evaluation of empirical,
normative, and practical issues
Research on situation assessment, mental models, and critical
thinking skills was based on in-depth interviews or think-aloud
problem-solving simulations with approximately 100 Army command
staff officers at a variety of levels of experience and rank.
As part of this work, CTI developed a prototype computer-based
system that reperesents an individual decision maker's situation
understanding and decisin making as it evolves in real time.
Cognitive Bases of Automated Target Recognition Design
(Army Research Laboratory / Human Research & Development)
CTI conducted research on the perceptual and cognitive underpinnings
of target recognition performance by Army combat helicopter
pilots, and derived recommendations for the design of automated
target recognition (ATR) devices. These recommendations are
designed to be compatible with users' perceptual, linguistic,
and cognitive processing constraionts.
This work resulted in experiments, models, and design recommendations
on a variety of topics. For example:
A model of perceptual processing stagesimages was developed
based on error and reaction time data. Enhancement of specific
image features was recommended to facilitate pilots' rapid
verification of ATR classifications.
Methods for predicting the optimal level of classification
labeling by the ATR were developed and tested.
Methods for alerting pilots to ATR uncertianty and for
guiding pilot's attention to potential problems were developed
and tested
Automated Assessment of Team Expertise
(Navy Personnel Research & Development Center)
The readiness of teams to perform complex tasks
hinges critically on cognitive abilities. But the relvant abilities
are not well measured in current team evaluations. CTI developed
a highly automated approach to assessing the use of mental models
associated with proficient performance. Specifically, the approach
enables us to
assess the global similarity of individuals and teams
to domain experts and journeymen
specify mental models used by experts and journeymen
diagnose specific strengths and deficiencies in the use
of those models by individuals and teams.
Commercial Airline Pilot Decision Making
(National Aeronautics and Space Administration, Ames Research
Center)
CTI has conducted a series of experimental studies with active
duty commercial airline pilots to examine their decision making
strategies. The studies have focused on the effects of time
constraints, uncertainty, and competing goals. They have also
examined interactions among the distributed team composed
of pilots, co-pilots, company dispatch, and air traffic control.
Three different studies have found, respectively:
Differences between more and less experienced pilots in
the use of dispatch advice and in critical thinking strategies
for evaluating that advice
Differences between more and lesse experienced pilots
in their ability to adapt the amount of time they take for
a decisin to the amount of time available.
Differences between more and lesse experienced pilots
in their ability to adapt information collection strategies
to different conditions of uncertainty.
Distributed Team Training
(Naval Air Warfare Center / Training Systems Division, Orlando,
FL)
CTI developed a framework for modeling the decision making
performance of geographically separated subteams (Naval air
squadrons) who train separately and then must perform together
(as an airwing). A cognitive performance analysis, based on
interviews, surveys, and observation of exercises, includes
cognitive modeling of knowledge structures and processes relevant
to distributed team performance, and an analysis of current
successes and shortfalls in current performance.
