Cognitive reflection: Diagnostics and individual differences

Abstract

This article summarizes the results of the original psychometric and correlational studies on cognitive reflection in a sample of adolescents and young adults. The first series of articles discusses the methodological foundations and psychometric qualities of a cognitive reflection test. Particular attention is paid to the potential of task familiarity due to the circulation of items with intuitive traps in the Russian-language Internet. It was established that items from the classic, verbal, and children’s versions of the test may be susceptible to this familiarity effect. The results of a confirmatory factor analysis support the acceptability and consistency of a single-factor model of the cognitive reflection test for both reflective (CRTr) and intuitive (CRT-i) responses. The reliability for the reflective response was higher than for the intuitive response (McDonald’s ω = 0.778 and 0.631, respectively). In addition to scale analysis, an examination of marginal scribbles on the assessment forms was conducted. This spontaneous, unplanned visualization of reflection — in the form of crossed-out answers and mathematical calculations — provides supplementary information on the mechanisms of interaction between primary and secondary cognitive systems. The second series of articles discusses the intellectual and personal correlates of cognitive reflection. In a sample of school students, significant correlations were found between cognitive reflection and both general and verbal intelligence. Associations were also identified between performance on the cognitive reflection test and critical thinking (specifically, the skill of generating arguments) and, to a lesser extent, emotional intelligence (empathy). Reliable personality correlates were fewer in number than intellectual ones: a reflective response was associated with lower anxiety and higher curiosity, while an intuitive response was linked to lower organization and to specific prosocial values such as conformity, tradition, and kindness. Finally, evidence was found linking cognitive reflection deficits with destructiveness. Interestingly, elevated narcissism combined with low psychopathy significantly predicted the level of cognitive reflection in students. The findings demonstrate that the cognitive reflection test is a relevant and promising tool for investigating individual differences in rationality and decision-making competence.