A complete list of my publications can be found on Google Scholar.
Publications
2026
Scholarly publications
Birillo, A., Rostovskii, A., Golubev, Y., & Keuning, H. (2026). Understanding Student Interaction with AI-Powered Next-Step Hints: Strategies and Challenges. In SIGCSE TS 2026 - Proceedings of the 57th ACM Technical Symposium on Computer Science Education V.1 (pp. 134-140). Association for Computing Machinery. [DOI] [Repository]
2025
Scholarly publications
Alpizar-Chacon, I., Keuning, H., De Jong, I., Lykourentzou, I., & Rings, S. (2025). Excited, Skeptical, or Worried? A Multi-Institutional Study of Student Views on Generative AI in Computing Education. In J. Leinonen, & R. Duran (Eds.), Proceedings of 25th International Conference on Computing Education Research, Koli Calling 2025 Article 13 (Proceedings of 25th International Conference on Computing Education Research, Koli Calling 2025). Association for Computing Machinery. [DOI] [Repository]
Carneiro Oliveira, E., Keuning, H., & Jeuring, J. (2025). Uncovering Behavioral Patterns in Student-LLM Conversations during Code Refactoring Tasks. In J. Leinonen, & R. Duran (Eds.), Proceedings of 25th International Conference on Computing Education Research, Koli Calling 2025 (pp. 1-11). Article 39 (Proceedings of 25th International Conference on Computing Education Research, Koli Calling 2025). Association for Computing Machinery. [DOI] [Repository]
Karol, D., Artser, E., Vlasov, I., Golubev, Y., Keuning, H., & Birillo, A. (2025). KOALA: A Configurable Tool for Collecting IDE Data When Solving Programming Tasks. In CompEd 2025 - Proceedings of the ACM Global Computing Education Conference 2025 (pp. 183-189). (CompEd 2025 - Proceedings of the ACM Global Computing Education Conference 2025; Vol. 1). Association for Computing Machinery. [DOI]
Lohr, D., Kiesler, N., Keuning, H., & Jeuring, J. (2025). Ignore These Errors for Now' - How Experts Provide Feedback on Steps Novices Take Towards Solving Programming Problems. In CompEd 2025 - Proceedings of the ACM Global Computing Education Conference 2025 (pp. 149-155). (CompEd 2025 - Proceedings of the ACM Global Computing Education Conference 2025; Vol. 1). Association for Computing Machinery. [DOI] [Repository]
Carneiro Oliveira, E., Keuning, H., & Jeuring, J. (2025). 'Can You Refactor This for Me?': Investigating How Students Use ChatGPT in Code Refactoring Exercises. In ITiCSE 2025 - Proceedings of the 30th ACM Conference on Innovation and Technology in Computer Science Education (pp. 788-788). (Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE; Vol. 2). Association for Computing Machinery. [DOI] [Repository]
Birillo, A., Vlasov, I., Dzialets, K., Keuning, H., & Bryksin, T. (2025). In-IDE Programming Courses: Learning Software Development in a Real-World Setting. In Proceedings - 2025 IEEE/ACM 2nd IDE Workshop, IDE 2025 (pp. 1-6). (Proceedings - 2025 IEEE/ACM 2nd IDE Workshop, IDE 2025). IEEE. [DOI] [Repository]
Alpizar-Chacon, I., & Keuning, H. (2025). Student's Use of Generative AI as a Support Tool in an Advanced Web Development Course. In ITiCSE 2025 - Proceedings of the 30th ACM Conference on Innovation and Technology in Computer Science Education (pp. 312-318). (Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE; Vol. 1). Association for Computing Machinery. [DOI] [Repository]
Nurollahian, S., Keuning, H., & Wiese, E. (2025). Teaching Well-Structured Code: A Literature Review of Instructional Approaches. In Proceedings - 2025 IEEE/ACM 37th International Conference on Software Engineering Education and Training, CSEE and T 2025 (pp. 205-216). (Software Engineering Education Conference, Proceedings). IEEE. [DOI] [Repository]
Lohr, D., Keuning, H., & Kiesler, N. (2025). You're (Not) My Type- Can LLMs Generate Feedback of Specific Types for Introductory Programming Tasks? Journal of Computer Assisted Learning, 41(1), Article e13107. [DOI] [Repository]
2024
Scholarly publications
Mitchell, A., Scott, M. J., Alshaigy, B., Geraldes, W. B., Garcia, R., Gutica, M., Keuning, H., Lovellette, E., Parthasarathy, P. D., Russell, S., Schulz, S., & Wu, X. (2024). Exploring Approaches to Assessing Student Teamwork in Undergraduate Computing Projects. 285-286. Abstract from 1st ACM Virtual Global Computing Education Conference V. 2, SIGCSE Virtual 2024, Virtual, Online, United States. [DOI]
Keuning, H. (2024). The interplay between rich and big data in programming education research. 19-21. Abstract from Die 22. Fachtagung Bildungstechnologien der Gesellschaft fur Informatik e.V., DELFI 2024 - 22nd Conference on Educational Technologies of the German Informatics Society, DELFI 2024, Fulda, Germany. [DOI]
Carneiro de Oliveira, E., Keuning, H., & Jeuring, J. (2024). Investigating Student Reasoning in Method-Level Code Refactoring: A Think-Aloud Study. In Koli Calling '24: Proceedings of the 24th Koli Calling International Conference on Computing Education Research Article 8 Association for Computing Machinery. [DOI] [Repository]
Artser, E., Birillo, A., Golubev, Y., Tigina, M., Keuning, H., Vyahhi, N., & Bryksin, T. (2024). Clustering MOOC Programming Solutions to Diversify Their Presentation to Students. In Koli Calling '24: Proceedings of the 24th Koli Calling International Conference on Computing Education Research Article 19 Association for Computing Machinery. [DOI] [Repository]
Birillo, A., Artser, E., Potriasaeva, A., Vlasov, I., Dzialets, K., Golubev, Y., Gerasimov, I., Keuning, H., & Bryksin, T. (2024). One Step at a Time: Combining LLMs and Static Analysis to Generate Next-Step Hints for Programming Tasks. In Koli Calling '24: Proceedings of the 24th Koli Calling International Conference on Computing Education Research Article 9 Association for Computing Machinery. [DOI] [Repository]
Keuning, H., Luxton-Reilly, A., Ott, C., Petersen, A., & Kiesler, N. (2024). Goodbye Hello World - Research Questions for a Future CS1 Curriculum. In Koli Calling '24: Proceedings of the 24th Koli Calling International Conference on Computing Education Research Article 27 Association for Computing Machinery. [DOI] [Repository]
Keuning, H., Alpizar-Chacon, I., Lykourentzou, I., Beehler, L., Köppe, C., de Jong, I., & Sosnovsky, S. (2024). Students' Perceptions and Use of Generative AI Tools for Programming Across Different Computing Courses. In J. Leinonen, & A. Mühling (Eds.), Koli Calling '24: Proceedings of the 24th Koli Calling International Conference on Computing Education Research (pp. 1-12). Association for Computing Machinery. [DOI] [Repository]
Lohr, D., Kiesler, N., Keuning, H., & Jeuring, J. (2024). "Let Them Try to Figure It Out First" - Reasons Why Experts (Do Not) Provide Feedback to Novice Programmers. In ITiCSE 2024: Proceedings of the 2024 on Innovation and Technology in Computer Science Education (pp. 38-44). (Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE; Vol. 1). [DOI] [Repository]
Roest, L., Keuning, H., & Jeuring, J. (2024). Next-Step Hint Generation for Introductory Programming Using Large Language Models. In N. Herbert, & C. Seton (Eds.), ACE 2024 - Proceedings of the 26th Australasian Computing Education Conference, Held in conjunction with: Australasian Computer Science Week (pp. 144-153). (ACM International Conference Proceeding Series). Association for Computing Machinery. [DOI]
2023
Scholarly publications
Prather, J., Denny, P., Leinonen, J., Becker, B. A., Albluwi, I., Craig, M., Keuning, H., Kiesler, N., Kohn, T., Luxton-Reilly, A., MacNeil, S., Peterson, A., Pettit, R., Reeves, B. N., & Savelka, J. (2023). The Robots are Here: Navigating the Generative AI Revolution in Computing Education. (pp. 1-51). arXiv. [DOI] [Repository]
Jeuring, J., Groot, R., & Keuning, H. (2023). What Skills Do You Need When Developing Software Using ChatGPT? (Discussion Paper). (pp. 1-11). arXiv. [Repository]
Börstler, J., Bennin, K. E., Hooshangi, S., Jeuring, J., Keuning, H., Kleiner, C., MacKellar, B., Duran, R., Störrle, H., Toll, D., & van Assema, J. (2023). Developers talking about code quality. Empirical Software Engineering, 28(6), 1-31. Article 128 . [DOI] [Repository]
Kiesler, N., Lohr, D., & Keuning, H. (2023). Exploring the Potential of Large Language Models to Generate Formative Programming Feedback. (pp. 1-6). IEEE. [DOI] [Repository]
Tigina, M., Birillo, A., Golubev, Y., Keuning, H., Vyahhi, N., & Bryksin, T. (2023). Analyzing the Quality of Submissions in Online Programming Courses. In 2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET) (pp. 271-282). Article 10172564 (Proceedings - International Conference on Software Engineering). IEEE. [DOI] [Repository]
Keuning, H., Jeuring, J., & Heeren, B. (2023). A Systematic Mapping Study of Code Quality in Education -- with Complete Bibliography. arXiv. [DOI] [Repository]
Birillo, A., Artser, E., Golubev, Y., Tigina, M., Keuning, H., Vyahhi, N., & Bryksin, T. (2023). Detecting Code Quality Issues in Pre-written Templates of Programming Tasks in Online Courses. In ITiCSE 2023 - Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education (pp. 152-158). (Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE; Vol. 1). Association for Computing Machinery. [DOI] [Repository]
Prather, J., Denny, P., Leinonen, J., Becker, B. A., Albluwi, I., Caspersen, M. E., Craig, M., Keuning, H., Kiesler, N., Kohn, T., Luxton-Reilly, A., MacNeil, S., Petersen, A., Pettit, R., Reeves, B. N., & Savelka, J. (2023). Transformed by Transformers: Navigating the AI Coding Revolution for Computing Education: An ITiCSE Working Group Conducted by Humans. In ITiCSE 2023 - Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education (pp. 561-562). (Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE; Vol. 2). Association for Computing Machinery. [DOI] [Repository]
Keuning, H., Jeuring, J., & Heeren, B. (2023). A Systematic Mapping Study of Code Quality in Education. 5-11. Paper presented at 28th Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE 2023, Turku, Finland. [DOI]
Oliveira, E., Keuning, H., & Jeuring, J. (2023). Student Code Refactoring Misconceptions. In ITiCSE 2023 - Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education: Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (pp. 19–25). (Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE; Vol. 1). Association for Computing Machinery. [DOI] [Repository]
2022
Scholarly publications
Jeuring, J., Keuning, H., Marwan, S., Bouvier, D., Izu, C., Kiesler, N., Lehtinen, T., Lohr, D., Peterson, A., & Sarsa, S. (2022). Towards Giving Timely Formative Feedback and Hints to Novice Programmers. In ITiCSE-WGR '22: Proceedings of the 2022 Working Group Reports on Innovation and Technology in Computer Science Education (pp. 95–115). Association for Computing Machinery. [DOI] [Repository]
Jeuring, J., Keuning, H., Marwan, S., Bouvier, D., Izu, C., Kiesler, N., Lehtinen, T., Lohr, D., Petersen, A., & Sarsa, S. (2022). Steps Learners Take When Solving Programming Tasks, and How Learning Environments (Should) Respond to Them. In ITiCSE 2022 - Proceedings of the 27th ACM Conference on Innovation and Technology in Computer Science Education (pp. 570–571). (Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE; Vol. 2). Association for Computing Machinery. [DOI] [Repository]
2021
Scholarly publications
Keuning, H., Heeren, B., & Jeuring, J. (2021). A Tutoring System to Learn Code Refactoring. In SIGCSE '21: Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (pp. 562-568). Association for Computing Machinery. [DOI] [Repository]
2020
Scholarly publications
Keuning, H. W., Heeren, B. J., & Jeuring, J. T. (2020). Student Refactoring Behaviour in a Programming Tutor. In N. Falkner, & O. Seppala (Eds.), Koli Calling '20: Proceedings of the 20th Koli Calling International Conference on Computing Education Research (pp. 1-10). Article 4 Association for Computing Machinery. [DOI] [Repository]
Keuning, H. W. (2020). Automated Feedback for Learning Code Refactoring. http://www.hkeuning.nl/PhD%20Thesis%20Hieke%20Keuning.pdf
2019
Scholarly publications
Keuning, H. W., Heeren, B. J., & Jeuring, J. T. (2019). How teachers would help students to improve their code. In Proceedings of ITiCSE 2019: the 24th Annual Conference on Innovation and Technology in Computer Science Education (pp. 119-125). Association for Computing Machinery. [DOI] [Repository]
2018
Scholarly publications
Keuning, H., Jeuring, J. T., & Heeren, B. (2018). A systematic literature review of automated feedback generation for programming exercises. ACM Transactions on Computing Education, 19(1). [DOI] [Repository]
2017
Scholarly publications
Estey, A., Keuning, H. W., & Coady, Y. (2017). Automatically Classifying Students in Need of Support by Detecting Changes in Programming Behaviour. In SIGCSE '17: Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (pp. 189-194). Association for Computing Machinery. [DOI]
Börstler, J., Störrle, H., Toll, D., van Assema, J., Duran, R., Hooshangi, S., Jeuring, J., Keuning, H., Kleiner, C., & MacKellar, B. (2017). "I Know It when I See It" Perceptions of Code Quality: ITiCSE '17 Working Group Report. In Proceedings of the 2017 ITiCSE Conference on Working Group Reports (pp. 70-85). (ITiCSE-WGR '17). Association for Computing Machinery. [DOI] [Repository]
Keuning, H., Heeren, B., & Jeuring, J. (2017). Code quality Issues in Student Programs. (Technical Report Series; No. UU-CS-2017-006). UU BETA ICS Departement Informatica. [Repository]
Arends, H., Heeren, B., Keuning, H., & Jeuring, J. (2017). An intelligent tutor to learn the evaluation of microcontroller I/O programming expressions. (Technical Report Series; No. UU-CS-2017-016). UU BETA ICS Departement Informatica. [Repository]
Arends, H., Heeren, B. J., Keuning, H. W., & Jeuring, J. T. (2017). An intelligent tutor to learn the evaluation of microcontroller I/O programming expressions. In Proceedings of Koli Calling 2017: the 17th Koli Calling International Conference on Computing Education Research (pp. 2-9). Association for Computing Machinery. [DOI] [Repository]
Keuning, H. W., Heeren, B. J., & Jeuring, J. T. (2017). Code quality issues in student programs. In Proceedings of ITiCSE 2017: the 22nd Annual Conference on Innovation and Technology in Computer Science Education (pp. 110-115). Association for Computing Machinery. [DOI] [Repository]
2016
Scholarly publications
Keuning, H. W., Jeuring, J. T., & Heeren, B. J. (2016). Towards a systematic review of automated feedback generation for programming exercises. In Proceedings of ITiCSE 2016: the 21st Annual Conference on Innovation and Technology in Computer Science Education (pp. 41-46)
2014
Scholarly publications
Keuning, H. W., Heeren, B. J., & Jeuring, J. T. (2014). Strategy-based feedback in a programming tutor. In Proceedings of the Computer Science Education Research Conference [DOI]
Keuning, H., Heeren, B., & Jeuring, J. (2014). Strategy-based feedback in a programming tutor. (Technical Report Series; No. UU-CS-2014-026). UU BETA ICS Departement Informatica. [Repository]