Some questions on measuring impact

Which public engagement activities work, which don't, and how can you measure that? These are some of the questions occupying Stephanie Helfferich at the Centre for Science and Culture. Below, she answers some questions on how to measure the effects of your public engagement activities.

Why do you want to measure impact?

First, to be able to determine whether you are doing the right things. As a researcher, you would like the energy and time you spend on a public engagement activity to be worth it. Also, you wish to show financers, be they the university or external funders, that money spent on public engagement is worth it. And we want to be able to give better advice to researchers who wish to engage a general audience, and in general to raise the quality of public engagement at Utrecht University.

Isn't “impact” too vague a concept to measure?

It's very difficult to prove for individual public engagement activities how they contribute to societal impact over a longer term, because so many other factors play a role as well. What you can show, however, is how a public engagement activity has an impact on its participants. Madelijn Strick, researcher at Social and Behavioural Sciences, very clearly explains this in this video, reporting on our first attempt at an impact measurement at the science festival Betweter Festival (in Dutch):

How can you measure impact?

At the Betweter Festival, we took a classical survey, but you can also be much more creative and playful in how you ask your questions, which makes it more fun for respondents and lowers the threshold to respond. Biologist Corette Wierenga participated in the Weekend van de Wetenschap with an activity to show children how the brain works. She added a quiz, which was fun for the children and gave an indication as to how much they had understood. Microbiologist Mirjam Duijvestijn, taking part at the same event with an activity about pathogens, had the children make a puzzle to see what they had learned:

And after the activity, they could indicate how much they had enjoyed it with a very clever survey set-up:

Nikae te Moller researches the muscoskeletal system and made a dance performance about her research. She asked the audience to indicate their understanding and enjoyment of the performance, using simply a blackboard and magnets.

This way, using relatively simple means, you can still find out something about what you achieve with your audience. Most important is to determine beforehand what you wish to achieve and with whom. Subsequently, you devise an activity which you think can achieve this aim. And then you think of ways to measure whether you did or not.

Do you have good examples, tips and experiences concerning measuring impact? Please let us know, so we can share this knowledge with other researchers. And of course you can always contact us should you have any questions about this: contact us.