PhD defence: Efficient Modelling Across Time of Human Actions and Interactions

PhD defence of A.G. Stergiou MSc


Videos can depict people. The actions and activities of people in videos can vary significantly in terms of their performance. Human cognitive perception can easily adapt to address these variations. This however, is not the case for machine perception as information from the world is encoded to a digital signal. The temporal aspect of videos when encoding action sequences is distinctly important, considering that only through time a fuller meaning of the action performed can be conveyed. However, there is not a standardised approach to effectively address the temporal extent of videos, and extract informative features that can be descriptive of the actions performed.

Our focus in this thesis is the refinement of temporal information representation in convolutional neural networks (CNNs). We aim to improve the recognition of variations in the performance of actions by first proposing an extension to the locality of convolutional kernels, by incorporating local feature relevance over the entire videos. We then study the extraction of features over different temporal durations, creating feature volumes that encapsulate action features over multiple durations. We further address the association between specific features and classes by creating connections over action classes and features of different complexities. Finally, within the scope of improving the representation capabilities of space-time CNNs, we explore the features that are learned and provide representation methods to aim their understanding as explainable qualitative measures.

Start date and time
End date and time
Academiegebouw, Domplein 29 & online (link)
PhD candidate
A.G. Stergiou MSc
Efficient Modelling Across Time of Human Actions and Interactions
PhD supervisor(s)
prof. dr. R.C. Veltkamp
dr. ir. R.W. Poppe