The evaluation of empirical algorithm performances in RL appears a closed topic. However, some (sparse) recent research provides unattended criticisms of key elements of the evaluations which are central to the conclusions of many research papers. This talk discusses some insights from the literature on evaluations in RL and some problems that still prevail. A lightning introduction to the methodological framework of multi-criteria decision analysis is then given with a view to providing a clue to resolve the problem.
Roland Dubb is a MSc candidate in applied mathematics at Shocklab, at the University of Cape Town. For his MSc, he is researching the empirical performance evaluations of reinforcement learning algorithms, under the supervision of Assoc. Prof. Jonathan Shock.
3 May 2023