The evaluation of empirical algorithm performances in RL appears a closed topic. However, some (sparse) recent research provides unattended criticisms of key elements of the evaluations which are central to the conclusions of many research papers. This talk discusses some insights from the literature on evaluations in RL and some problems that still prevail. A lightning introduction to the methodological framework of multi-criteria decision analysis is then given with a view to providing a clue to resolve the problem.