Virtual Interaction

Zerrin Yumak and Arjan Egges

Virtual Humans that are capable of advanced social and environmental interaction help create immersive virtual worlds for educational role-playing games.

Learning by doing and experimenting is found to be the most effective learning method. Role-playing games can aid in the development and assessment of job skills by immersing the trainees into virtual worlds that are similar to real-life contexts. Virtual Humans with advanced social interaction capabilities are required to realize this. We are developing a virtual interaction system that enables autonomous behavior for socially aware virtual characters during multi-party social interactions, in particular for negotiation and communication skills scenarios.

Interactions between virtual humans and users so far focused on one-to-one interactions, and multi-party interactions involving several participants still remains a challenge. Our goal is to develop autonomous behavior for virtual characters driven by the sensory input in the real environment. We are in particular interested in the estimation of appropriate timing of gaze behaviors to signal turn-taking decisions as well as the believable generation of non-verbal behaviors.

We are also interested in perception of animations and evaluation of fully immersed virtual reality experiences in collaboration with experts from social sciences. With the advent of technologies that allow video-game designers to develop games that have a high degree of immersion, combining neuroscientific knowledge in this process is a logical step.  For instance, while designing non-player characters (NPC) it is important to understand what aspects of the NPC’s appearance and behavior cause the player to “feel” the emotions he/she would normally feel if he/she were to have the same interaction in the real world.  A good way to test if the NPC is successful is to measure the subject’s reactions to it by using both subjective and objective behavioral measures.

Our research is currently supported by the Horizon 2020 RAGE EU project (No. 644187) and Game Research Focus Area Seed Funding. The software we have built can be used in other projects. Please contact Zerrin Yumak or Arjan Egges for further information.