The numbers do not lie: the teacher or the centralized final test
Preparatory secondary vocational education, senior secondary general education or pre-university education? Pupils in the final year of primary school are issued a recommendation on the most suitable type of secondary school. The question is: who is better at predicting the appropriate level? The teacher or the centralised final test? Rens van de Schoot, alumnus and professor of Statistics for Small Data Sets, is currently investigating this question with fellow statistician and alumnus Kimberley Lek.
The type of secondary school you attend will shape the rest of your life. In addition to your future studies, it will affect the people you meet and the type of job you end up doing. Unsurprisingly, the way in which secondary school recommendations are formulated has been a cause for concern. This has prompted years of debate: should more weight be given to the final test, or is the teacher's opinion more important? Proponents of teacher recommendations argue that the centralised final test is merely a snapshot, and takes no account of the child's home situation or learning style. Opponents argue teachers are not ‘measuring instruments’ and cannot be objective. They also suggest teachers are less capable of weighing information in order to arrive at the right decision.
In an effort to scientifically determine who knows best, Van de Schoot and Lek explored the Statistics Netherlands archives. As it turned out, the results were surprising. Van de Schoot: ‘We assessed the teacher recommendations and centralised final tests for 119,751 pupils that were in the final year of primary school during the 2014/2015 school year.
We compared them to the level at which these pupils had actually ended up three years later.’ As it turned out: ‘The centralised final test proved an accurate predictor for pupils at pre-university level, whereas the teacher recommendations were more accurate in the case of preparatory secondary vocational education pupils. Neither predictor is necessarily more accurate, in other words. We recommend combining the two recommendations.’
The Netherlands classifies children at specific educational levels at an extremely early age
Apples and oranges
This presented another challenge for the two researchers. ‘You can't simply combine the recommendations: it's apples and oranges. Practical experiences with test scores. Gut feelings about statistics. So how do you align the two predictors?’ The primary education system ultimately provided a solution. ‘Primary school teachers teach at different levels. For example, one child might be better at languages, while the other can easily calculate 9 x 125. Whether consciously or unconsciously, this leads to the formation of groups in each class. That gave us the idea for an app which prompts teachers to classify their pupils in levels as they would in the classroom (from low to high). Kimberly's app translates the teacher's insights into a statistical distribution. Test results can also be expressed in the same format, which allows you to consolidate the two distributions in a single recommendation.’
Still, an app can never solve every problem, as Van de Schoot explains. ‘What if there's a huge gap between the teacher's recommendation and the test result? We're currently doing a follow-up study to address that problem.’ ‘For now, we've settled on a practical solution and are only offering recommendations for transitional classes’, explains Van de Schoot. ‘If you allow the first – and even second years – of secondary school to accommodate preparatory secondary vocational education/senior secondary general education and senior secondary general education/pre-university education pupils, late bloomers have a chance of ending up at the right level. Primary school pupils that did relatively well in their safe former environment and start falling behind once they transition to secondary school will also be able to cope more effectively.’ This recommendation reflects the joint plan presented by educational organisations including the Secondary Education Council, Netherlands Association of Senior Secondary Vocational Schools and Christian Teachers' Union on 21 January 2020. The plan stressed that children should be issued a definitive recommendation in the third year of secondary school rather than the final year of primary education. Van de Schoot: ‘The Netherlands classifies children at specific educational levels at an extremely early age. It's better to spread out the selection process over several years and make sure transferring is easy and doesn't involve any financial consequences imposed by the Education Inspectorate.’
In addition to studying human intelligence – e.g. in the context of the centralised final test – Professor Van de Schoot is also examining the role of artificial intelligence (AI). For example, he is currently investigating how AI can be applied in support of the research process. ‘Researchers have to go through stacks of articles before starting on a new study. In some cases, thousands of articles are relevant to their particular project. Going through them all takes a great deal of time and energy. AI can do the bulk of that job, and select the 50 most relevant articles. Obviously, that will save us a lot of time. I think this type of AI application can also be relevant to teaching. In future, lecturers may even be able to leave tasks like checking assignments to AI while they do the actual teaching work.’
Rens van de Schoot is Professor of Statistics for Small Data Sets Van de Schoot is a member of the Royal Netherlands Academy of Arts and Sciences Young Academy, and Association for Multivariate Experimental Psychology (SMEP). Having completed a higher professional education-level programme in Medical Imaging and Radiotherapeutic Techniques, he studied Psychology and Development and Socialisation in Childhood and Adolescence at Utrecht University, obtaining both his Research Master's and PhD with a cum laude distinction.
Kimberley Lek (1990) obtained both a Research Master's in Educational Science and a Research Master's in Methodology and Statistics at Utrecht University, both with a cum laude distinction. She will obtain her doctorate on the basis of this study on 22 April 2020, and has since found employment at Cito.
This article appeared in alumni magazine Illuster.