IDA alumna Iris Willigers presented her Master thesis at the graduation ceremony of the IDA cohort 2022-2024.
The Raven’s Standard Progressive Matrices exhibits Flynn Effects over time, potentially leading to ceiling effects and unreliable test outcomes. Our simulation studies explored how increasing mean levels of true (fluid) intelligence affect various outcomes, including mean sum scores, (corrected) standard deviations, true reliability, KR-20, KR-21 and (corrected) observed Cohen’s ds. We assessed the effect of different conditions of latent mean (differences) to assess item difficulties under the Rasch model with input of item responses of an existing dataset. Rasch Analysis revealed no major issues in measuring (fluid) intelligence. However, the sum score distributions did suffer from floor and ceiling effects. True reliability and reliability estimates were low for low and high latent means. In addition, observed Cohen’s d was biased compared to latent Cohen’s d, and corrections with the KR-20 or KR-21 did not decrease bias for all conditions. In addition, shorter test versions of the test yielded even lower reliability estimates. We also examined meta-analytical implications for intelligence interventions on between-groups Cohen’s d measured by the Raven’s Standard Progressive Matrices. There was no significant mean meta-analytic effect found, with also no moderating effects of KR-21 and number of items. There was a positive relationship between the number of items and KR-21 for these studies. In conclusion, our study shows that the measurement precision of the Raven’s Standard Progressive Matrices is inadequate for making inferences on the sum score distribution and Cohen’s d of samples in the lower or higher latent means.
Keywords: Raven’s Standard Progressive Matrices, SPM, simulation study, meta-analysis, reliability, measurement precision, ceiling effects
Iris is now continuing her career in academia as a PhD candidate at Tilburg University. Congratulations Iris and best of luck with your new position!
Be First to Comment