Multi-Agent Learning

Multi-Agent Learning

Gerard Vreeswijk

How to intelligently adapt an agent's behaviour to populated, dynamic, and often unpredictable environments is a complex but fundamental question in artificial intelligence.

The world of software systems increasingly consists of autonomous software agents that interact in open, dynamic and unpredictable environments. In such environments it is often impossible to anticipate on all interactions beforehand. Therefore, intelligent adaptation of behaviour among artificial agents is a key research issue in artificial intelligence and agent technology. 

Research in multi-agent learning (MAL) studies software agents that learn and adapt to the behaviour of other software agents. The presence of other learning agents complicates learning, which makes the environment non-stationary (a situation of learning a moving target) and non-Markovian (a situation where not only experiences from the immediate past but also earlier experiences are relevant).

MAL is a mix of game theory, probability theory, and multi-agent systems theory.  Important topics include statistical learning and single-agent reinforcement learning, (evolutionary) game theory, fictitious play, gradient dynamics, no-regret learning, multi-agent reinforcement learning (MinMax-Q, Nash-Q), leader (teacher) vs. follower (learner) adaptation, and the emergence of social conventions. Examples of domains that need robust MAL algorithms include manufacturing systems (managers of a factory coordinate to maximise their profit), distributed sensor networks (multiple sensors collaborate to perform a large-scale sensing task under strict power constraints), robo-soccer, disaster rescue, and recreational games of imperfect information such as poker.