We inform authors and readers that, following an agreement with the Karolinum publishing house, from 2024 (Volume 18), the journal Orbis scholae will be published only in electronic form.

Orbis scholae is an academic journal published by Charles University, Prague. It features articles on school education in the wider socio-cultural context. It aims to contribute to our understanding and the development of school education, and to the reflection of teaching practice and educational policy.

ORBIS SCHOLAE, Vol 12 No 2 (2018), 125–140

Demonstration of Simpson’s Paradox in PISA 2015 Data: Confusing Differences between Boys and Girls

Gašper Cankar

DOI: https://doi.org/10.14712/23363177.2018.296
published online: 25. 01. 2019


This paper explores the occurrence of a Simpson’s paradox in PISA 2015 science literacy data. Simpson’s paradox, a case of contradicting interpretations when results are analysed by groups or aggregated as a whole, has both a practical and an academic significance. It is an interesting phenomenon that is far from theoretical and when it happens, it has profound effects on the interpretation and if left unidentified can cause confusion and misunderstanding. This paper demonstrates best ways to detect Simpson’s paradox through appropriate tables and graphs. Actual occurrences of a Simpson’s paradox and conditions leading to them are explored using PISA 2015 gender differences in science literacy data in five central European countries − Austria, Croatia, Czech Republic, Slovakia and Slovenia. In countries where the occurrence of a Simpson’s paradox was detected, we provide correct interpretation of the results. Beside creating problems with interpretation an occurrence of a Simpson’s paradox also provides new insight − it signifies that there is very different gender composition in different educational tracks which has important implications for the educational governance. We will discuss implications of these findings in context of Slovenian educational system.

keywords: PISA; Simpson’s paradox; gender differences; educational tracks; governance

references (16)

1. Blyth, C. R. (1972). On Simpson's paradox and the sure-thing principle. Journal of the American Statistical Association, 67(338), 364–366. CrossRef

2. Baker, S. G., & Kramer, B. S. (2001). Good for women, good for men, bad for people: Simpson's paradox and the importance of sex-specific analysis in observational studies. Journal of Women's Health and Gender-Based Medicine, 10(9), 867–872. CrossRef

3. Cohen, B. S., & Moch, P. L. (2017). Guarding against Simpson's paradox when combining data sets. Curriculum and Teaching Dialogue, 19(1 & 2), 153.

4. Demers, S., & Rossmo, D. K. (2015). Simpson's paradox in Canadian police clearance rates. Canadian Journal of Criminology and Criminal Justice, 57(3), 424–434. CrossRef

5. European Commission. (2016). She figures 2015. Luxembourg: Publications Office of the European Union. Retrieved from http://ec.europa.eu/research/swafs/pdf/pub_gender_equality/she_figures_2015-final.pdf.

6. Gardner, M. (1982). Aha! Gotcha: paradoxes to puzzle and delight. San Francisco: Freeman.

7. Lesser, L.M. (2001), Representations of reversal: An exploration of Simpson's paradox. In A. A. Cuoco, & F. R. Curcio (Eds.), The roles of representation in school mathematics (pp. 129-145). Reston, Virginia: National Council of Teachers of Mathematics.

8. OECD. (2017). PISA 2015 technical report. Paris: OECD Publishing.

9. R Core Team (2017). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. On-line: https://www.R-project.org.

10. Rücker, G., & Schumacher, M. (2008). Simpson's paradox visualized: The example of the Rosiglitazone meta-analysis. BMC Medical Research Methodology, 8(1). CrossRef

11. Simpson, E. H. (1951). The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society, 13(2), 238–241.

12. Smith, E. (2008). Using secondary data in educational and social research. New York, NY : McGraw Hill/Open University Press.

13. Tan, A. (1986). A geometric interpretation of Simpson's paradox. College Mathematics Journal, 17, 340-341. CrossRef

14. UNESCO (2014). UNESCO's promise: Gender equality, a global priority. Paris: UNESCO. Retrieved from http://unesdoc.unesco.org/images/0022/002269/226923m.pdf.

15. Wright, B. (2012). Best of N contests: Implications of Simpson's paradox in tennis [Masters Thesis]. The Florida State University.

16. Yule, G. U. (1903). Notes on the theory of association of attributes in statistics. Biometrika, 2(2), 121–134. CrossRef

Creative Commons License
Demonstration of Simpson’s Paradox in PISA 2015 Data: Confusing Differences between Boys and Girls is licensed under a Creative Commons Attribution 4.0 International License.

230 x 157 mm
periodicity: 3 x per year
print price: 150 czk
ISSN: 1802-4637
E-ISSN: 2336-3177