Linking Mathematics TIMSS Achievement with National Examination Scores and School Marks: Unexpected Gender Differences in Slovenia

In the article, we present the results of the Slovene national study of three different assessments of mathematics for students in Grade 8 and Grade 13: the independent TIMSS or TIMSS Advanced outcomes, the national external examination scores and internal teacher’s marks. Grade 8 students who participated in TIMSS also took the national assessment (NA) one year after TIMSS assessment; TIMSS Advanced math students took the Matura examination from mathematics two months after the TIMSS Advanced assessments. The data on school marks from mathematics were collected with the nationally added questions to the international TIMSS and TIMSS Advanced questionnaires for students, together with the series of questions about the effort put into solving the test. One year later, the outcomes from TIMSS assessment, national examinations (Grade 9 and Grade 13) and school grades for each student were linked and the differences between boys and girls, attitudes toward mathematics and plans for future education were analysed. It was found that gender differences at scores from national exams as well as in school marks differ from gender differences in TIMSS and TIMSS Advanced achievement (i.e. Grade 13 students’ Matura results are slightly in favour of girls while TIMSS Advanced show better achievement for boys). Comparison of three outcomes reveal some characteristics of both national examinations and teachers’ marking not evident otherwise. Matura gives to the most able students proportionally less opportunity to demonstrate the highest cognitive level of knowledge. Boys who demonstrated the same knowledge in TIMSS as girls get lower national marks as girls, in exams and by teachers. Girls put less effort than boys in solving the TIMSS test which could help to explain the changing gender gap from TIMSS to the national examinations. In Grade 8, the marks and TIMSS scores also show inconsistencies on student level. They are differently associated with attitudes toward mathematics which can provide some ideas for improvement of low motivation for learning mathematics in Slovenia.

1. Baye, A. & Monseur, C. (2016). Gender differences in variability and extreme scores in an international context. Large-Scale Assessment in Education 4(1). CrossRef

2. Bergold, S., Wendt, H., Kasper, D., & Steinmayr, R. (2017). Academic competencies: Their interrelatedness and gender differences at their high end. Journal of Educational Psychology, 109(3), 439–449. CrossRef

3. Burusic, J., Babarovic, T., & Seric, M. (2012). Differences in elementary school achievement between girls and boys: Does the teacher gender play a role? European Journal of Psychology of Education, 27(4), 523–538. CrossRef

4. Cankar, G. (2010). PISA 2006 main findings in Slovenia and Simpson's paradox. V: XIV. IOSTE Symposium, International Organization for Science and Technology Education, June 13.–18. 2010, Bled, Slovenia. In S. Dolinšek (Ed.), Socio-cultural and human values in science and technology education: proceedings. Institute for Innovation and Development of University. Ljubljana.

5. Cankar, G. (2016). Governing the transition to higher levels of education and differences between achievement and school grades by gender. The Journal of Educational Research, 8(2), 60–86.

6. Ceci, S. J., Williams, W. M. & Barnett, S. M. (2009). Women's underrepresentation in science: Sociocultural and biological considerations. Psychological Bulletin, 135(2), 218–261. CrossRef

7. Eklöf, H. (2006). Development and validation of scores from an instrument measuring student test-taking motivation. Educational and Psychological Measurement, 66 (4), 643-656. CrossRef

8. Else-Quest, N. M, Shibley Hyde J., & Linn, M.C. (2010). Cross-national patterns of gender differences in mathematics: A meta-analysis. Psychological Bulletin, 136(1), 103–127. CrossRef

9. Holman L., Stuart-Fox D., & Hauser C. E. (2018). The gender gap in science: How long until women are equally represented? PLOS Biology 16(4), e2004956. CrossRef

10. Houtte, M. V. (2004). Why boys achieve less at school than girls: The difference between boys' and girls' academic culture. Educational Studies, 30(2), 159–173. CrossRef

11. Hyde, J. S. (2005). The gender similarities hypothesis. American Psychologist, 60(6), 581–592. CrossRef

12. IEA (2017a). TIMSS Advanced SPSS Data. TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College. Retrieved from https://timssandpirls.bc.edu/timss2015/advanced-international-database.

13. IEA (2017b). TIMSS SPSS Data. TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College. Retrieved from https://timssandpirls.bc.edu/timss2015/international-database.

14. Klapp Lekholm A., & Cliffordson C. (2008). Discrepancies between school grades and test scores at individual and school level: effects of gender and family background, Educational Research and Evaluation, 14(2), 181–199. CrossRef

15. Lindberg, S. M., Hyde, J. S., Petersen, J. L. & Linn, M. C. (2010). New trends in gender and mathematics performance: A meta-analysis. Psychological Bulletin, 136(6), 1123–1135. CrossRef

16. Mullis, I. V. S. & Martin, M. O. (Eds.). (2013). TIMSS 2015 assessment frameworks. TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College. Retrieved from http://timssandpirls.bc.edu/timss2015/frameworks.html.

17. Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2016a). TIMSS Advanced 2015 International Results in Advanced Mathematics and Physics. TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College. Retrieved from http://timssandpirls.bc.edu/timss2015/international-results/advanced.

18. Mullis, I. V. S, Martin, M. O., Foy, P. & Hooper, M. (2016b). TIMSS 2015 International Results in Mathematics. Boston College, TIMSS & PIRLS International Study Center. Retrieved from http://timssandpirls.bc.edu/timss2015/international-results.

19. Nosek, A. B., Smyth, F. L., Sriram, N., Lindner, N. M., Devos, T., Ayala, A., … , Greenwald, A. G. (2009). National differences in gender-science stereotypes predict national sex differences in science and math achievement. Proceedings of the National Academy of Sciences, 106, 10593-10597. CrossRef

20. OECD (2015). The ABC of gender equality in education: Aptitude, behaviour, confidence. Pisa, OECD Publishing.

21. Rutkowski, D., Rutkowski L., & Plucker, J. A. (2012). Trends in education excellence gaps: a 12-year international perspective via the multilevel model for change. High Ability Studies, 23(2), 143–166. CrossRef

22. Spinath, B., Eckert, C., & Steinmayr, R. (2014). Gender differences in school success: what are the roles of students' intelligence, personality and motivation? Educational Research, 56(2), 230–243. CrossRef

23. Steinmayr, R. & Spinath, B. (2008). Sex differences in school achievement: What are the roles of personality and achievement motivation? European Journal of Personality, 22(3), 185–209. CrossRef

24. Voyer, D., & Voyer, S. D. (2014). Gender differences in scholastic achievement: A meta-analysis. Psychological Bulletin, 140(4), 1174–1204. CrossRef