Background: Sex differences in spatial task performance are among the most well-established findings within the empirical Psychological literature. Therefore, findings of an exploratory study in the late 1990s indicating that sex differences vanish when tasks are presented by means of three-dimensional models instead of two-dimensional representations of objects were surprising. Based on these results, it has been argued, that sex differences in spatial task performance are due to the necessity to abstract two-dimensional figures to three-dimensional mental representations (i.e., cross-dimensionality). However, past empirical evidence has shown that results of exploratory studies may be considerably biased (resulting, typically, in effect inflation). However, effect inflation in exploratory studies can often be seen as a consequence of large error variances due to low statistical power and an inclination of authors and publishers alike to prefer publication of novel and surprising (and arguably implausible) results. Considering the well-established nature of sex differences in spatial ability tasks, dissemination biases may be expected to work in an opposite fashion (i.e., leading to effect underestimation) when novel hypotheses are explored. In the present meta-analysis, we investigate summary effects, dissemination bias, and non-genuine (i.e., artifact-related) time-trends based on the available evidence in the scientific literature about sex differences in stereoscopically presented 3D spatial ability tasks.
Methods: To retrieve potentially eligible studies, we searched five electronic databases for published literature (ISI Web of Knowledge, Pubmed, PsycInfo, Psyndex, Scopus) using cited reference and keyword searches as well as the Open Access Theses and Dissertations database to identify grey literature. In all, 13 independent studies as well as three unpublished data sets from the senior author of this contribution [GG] fulfilled our inclusion criteria, yielding stereoscopically administered three-dimensional spatial task performance data from 16 samples of healthy men and women (N = 1,908). We used random-effects estimations, weighted mixed-effects meta-regressions, as well as several methods for dissemination bias assessment to investigate summary effects, time-trends, and bias.
Results: The observed summary effect yielded sex differences that are (i) consistent with expected signs (i.e., favoring men) and (ii) broadly comparable in strength with effects that are typically obtained in standard (i.e., two-dimensional) assessments (Cohen d = 0.483). Publication year was significantly and positively associated with observed effect sizes in published, but not in unpublished studies, thus indicating increasing effect strengths over time. The effect size of the initial study exploring this novel hypothesis was about five times smaller than the meta-analytical summary effect (Cohen d = .094, p > .05), indicating evidence for a so-called winner’s curse (i.e., non-replicability of initial study findings). Moreover, Sterne and Egger’s regression approach, trim-and-fill analyses, as well as Ioannidis and Trikalinos excess significance estimation yielded consistent evidence of missing studies with strong and significant effects.
Discussion: The present meta-analysis shows evidence for robust sex differences in stereoscopically administered three-dimensional spatial ability tasks. This raises doubts about cross-dimensionality as a plausible cause for sex differences in spatial task performance. Moreover, the observed systematic increases in the strength of observed published study effects are consistent with expected trajectories originating in dissemination bias-related mechanisms and resembling an inverse decline effect. The causes for implausible findings may be attributed to large error variances of exploratory research designs which are rooted in low power. Our findings show that concerns about effect replicability in the Psychological literature seem to be justified.
