Efficient Inverse RL


Interactive learning systems like self-driving cars, recommender systems, and large language model chatbots are becoming increasingly ubiquitous in everyday life. From a machine learning perspective, the key technical challenge underlying such systems is that rather than simple prediction on i.i.d. data, an interactive learner influences the distribution of inputs it sees via the choices it has made in the past, dramatically increasing the statistical and computational complexity of the problem. In this talk, we’ll focus on how to perform interactive imitation learning efficiently. We derive a unifying, game-theoretic framework for imitation learning and provide several efficient reductions to more tractable problems like supervised or online learning. We also consider sample efficiency, both in terms of expert demonstrations and learner-environment interactions and derive minimax-optimal and polynomial-time algorithms, respectively.


Gokul Swamy is a rising 4th Year PhD student in the Robotics Institute at Carnegie Mellon University where he works on algorithms for imitation learning, focusing on efficiency and robustness to unobserved confounders. Before that, he was a master’s student and undergraduate at UC Berkeley. Even before that, he had a lot of tacos and spent as much time on the beach as possible in San Diego. He has spent summers at SpaceX, NVIDIA, Aurora, Microsoft Research, and Google Research.


19 July 2023