Training Evaluation

CTI has tested its Critical Thinking Training at three Navy training facilities and at a large number of Army posts around the country. The table summarizes training evaluation results.

A version Army Critical Thinking Training (with limited interactive capability) is available on CD or over the web.

The Navy Critical Thinking Training has been used in both individual and team contexts and is supported by an automated AEGIS Combat Center simulation. We have also tested the effect of critical thinking training on officers' interaction with a decision aid.


Measures

Measures of training include effects on cognitive processes and effects on the quality of decisions. We also look at subjective measures, such as the effects of training on confidence in decisions and rating of the training by trainees.

Critical thinking processes include the extent to which officer consider critical issues in the scenario, identification of conflicting evidence and conflicting goals, handling cases of conflict (e.g., by re-evaluating assumptions), exploring alternative options, and developing contingency plans in case problems occur. These critical thinking processes are described in the Recognition / Metacognition model.

Measures of decision quality are objective if possible, e.g., with reference to a ground truth vector that is used to generate an experimental scenario. In other cases, measures of decision quality are based on agreement with subject matter experts, as well as agreement among trainees themselves.



Results

Statistically significant effects of critical thinking training have been found with measures of both process and decision quality (singificant effects are shown in red in the table). All changes in both process and accuracy -- whether statisically significant or not -- were always in a positive direction.

Training to think critically did not reduce confidence, and in fact tended to slightly increase it.

Officers' subjective evaluations of the training were also consistently positive. There were many positive comments about specific aspects of the training, e.g., the devil's advocate method, as well as about the general sharpening of thinking skills. By far the most common negative comment was "I already do this."



 





 

 

Copyright © 2000-2011 Cognitive Technologies, Inc.
Questions? Comments? Contact webmaster@cog-tech.com