In the long history of intelligence research, there have been numerous attempts to examine new factors of intelligence. A prominent example is the ability to solve complex problems (CPS), which has recently attracted attention... [ view full abstract ]
In the long history of intelligence research, there have been numerous attempts to examine new factors of intelligence. A prominent example is the ability to solve complex problems (CPS), which has recently attracted attention due to its integration in educational large-scale assessments (e.g., Programme for International Student Assessment, PISA). Although there is a strong conceptual overlap between CPS and intelligence, CPS is considered as partly independent of established factors of intelligence (e.g., fluid reasoning). Indeed, several empirical studies provided some evidence of convergent, discriminant and incremental validity of CPS. Therefore, two studies were conducted to further examine whether CPS should be considered as an independent factor within the nomological network of intelligence. Based on structural equation modelling (SEM), a substantial impact of the operationalization of intelligence was found in both studies. Study 1 (n = 227): If a narrow operationalization of intelligence (i.e., figural reasoning such as Raven's Matrices Test) was used as in most of the previous studies, we found 49% (95% CI [39.4, 57.8]) shared variance between CPS and intelligence, and an incremental explanation of variance in school grades up to 4% above and beyond intelligence. However, if a construct-representative operationalization (i.e., Berlin Intelligence Structure Model test) was used, we found 60% (95% CI [52.4, 68.0]) of shared variance between CPS and intelligence, and no incremental validity of CPS in explaining school grades above and beyond intelligence (0.7%). Study 2 (n = 253): Following the results of the first study, we systematically varied the operationalization of intelligence (i.e., fluid reasoning) with regard to the task content (i.e., figural, numerical, verbal) and examined the relation to CPS. Dependent on the specific operationalization of intelligence, we found latent correlations of .38 (95% CI [.19, .54]) to .78 (95% CI [.61, .94]). In summary, we were able to replicate the findings of previous studies regarding the relation between CPS and intelligence but only if we used a narrow operationalization of intelligence as in previous studies. If we used a broad operationalization of intelligence, there was little to no evidence for an independent CPS construct within the established nomological network of intelligence. Therefore, previous findings of CPS as an independent intelligence factor should be scrutinized because of too narrow operationalizations of intelligence. In general, our findings emphasize the importance of construct-representative operationalizations when the nomological network of intelligence is investigated.
