Human-centered AI colloquium with Kees van Deemter and Eduardo Calò: How to Classify a Hallucination?

The Human-centered Artificial Intelligence Colloquium Committee invites you to attend the last colloquium of the 2023-2024 academic year. Kees van Deemter and Eduardo Calò will talk about how to classify an AI hallucination. After the colloquium, you are warmly invited to join us for drinks.

AI hallucinations

Text generation systems that are based on Deep Learning have made great strides, yet the texts they produce are known to suffer from various kinds of defects. Some of the most glaring defects arise when the texts that are generated express information that is false or misleading. Defects of this kind are loosely associated with the term 'hallucination'.

In this talk, Van Deemter and Calò will discuss a new annotation scheme which attempts to define some different kinds of hallucination in a way that, unlike its predecessor schemes, (1) is logically well-founded, (2) is applicable by non-expert human annotators, (3) can be checked automatically by a Natural Language Inference system.

For simplicity, the scheme focusses on texts that are generated from structured data. Having discussed this annotation scheme, we will report on a recent experiment in which annotators were asked to apply the scheme to a set of textual descriptions of hotels that were generated from structured data by trivago N.V. in Düsseldorf, and we will discuss the lessons that we learnt from this experiment. This talk is joint work with Albert Gatt and Saad Mahamood.

Van Deemter and Calò

Kees van Deemter is an emeritus professor at UU whose main area of expertise is Natural Language Generation (NLG). He specifically takes an interest in logical issues arising from this work. Eduardo Calò is a third-year PhD candidate at UU under Kees's supervision. He works in NLG from logical forms.