Comparing comparatively with Comproved

• Name teacher: Stephanie Kruiper
• Faculty: FSW
• Tool: Comproved

The course Introduction to Educational Science - in addition to studying substantive educational science topics - focuses on learning academic skills, such as being able to find and critically process scientific literature. During this course, students work on long-term assignment: writing an introduction to a scientific paper. This includes an explicit focus on learning skills around giving, receiving and asking for feedback. The course is taught by 7 instructors to approximately 100 students.

Goals

Instructor Stephanie Kruiper used comparative assessment as a tool in this course. To do so, she used the tool Comproved, which she heard about through a fellow instructor. Her aim was twofold: to increase the reliability and quality of the final assessment of the long-term assignment and to promote feedback literacy among students. The basic principle of comparative assessment is that assessment is based on the comparison of products. Often teachers find grading assignments complicated, especially when the outcomes can be very different. Comparative grading assumes that it is easier to compare performances than to have to grade them independently and is seen as a valid and reliable method, especially for multi-teacher grading. In addition, it has advantages for students because the method can be used formatively, with students getting to see many more examples of work than they would otherwise receive. This can help them form a better idea of what a good text should meet, which can lower their feelings of uncertainty.

"As with any tool, at the beginning it takes some time to figure out how it works. The program itself works pretty intuitively and there is an explanation for every part where you have to set something up yourself. If things don't work out, support is never far away" Stephanie says. "We really used the tool as a thread through the education, the workgroup assignments were also adapted to what we could do with the program. Beforehand, we thought carefully about w]hat we wanted to achieve and actively (re)designed our education on this".

Results

The tool was deployed for the second time this year after a successful test last year. Among other things, it was mentioned then that seeing all the other (anonymous) papers and the feedback given on them had great added value. In addition, students appreciated the fact that they received feedback from more than one person. Not only did comparative grading give them an idea of the differences in assignments, but they were able to use the examples and feedback from higher-ranked assignments to improve their own work: 'I liked the fact that you could get a lot of feedback this way, and that allowed you to improve your long-running assignment in a focused way. It also gave you a little idea of how others had approached the writing. This in turn allowed you to get ideas or points for improvement. On the other hand, not every student took giving feedback seriously, so not everyone was provided with quality feedback.

Students and teachers experience the evaluation as fairer, because the evaluation no longer depends on one teacher. Based on the evaluation, steps appear to have been made in feedback literacy. This is especially evident in the fact that students used Comproved a lot to review other students' work and feedback. Students themselves reported liking this and seeing the added value in improving their own work.

In conclusion, Stephanie wants to give fellow teachers a tip about the method: "Striving for objectivity is nice, but secretly we all know that we never look at a work completely objectively. Instead of seeing this as a problem, Comproved allows you to embrace it. In fact, use that diversity for reliable assessment!"