Topics in Multiagent Learning

While machine learning techniques have had significant success in single-agent settings, an increasingly large body of literature has been studying settings involving several learning agents with different objectives. In these settings, standard training methods such as gradient descent are less successful and the simultaneous learning of the agents commonly leads to non-stationary and even chaotic system dynamics. Motivated by these challenges, this course presents the foundations of multi-agent systems from a combined game-theoretic, optimization, and learning-theoretic perspective. We start with basic matrix games, such as rock-paper-scissors, and advance to more complex forms including imperfect information games and structured games like combinatorial games, polymatrix games, and stochastic games. Topics will cover various equilibrium concepts, aspects of equilibrium computation and learning, as well as the computational complexity of finding equilibria. Additionally, we will examine how these models and methods have driven recent breakthroughs in AI, producing human- and superhuman-level agents for well known games such as Go, poker, Diplomacy, and Stratego.