RLiable: Towards Reliable Evaluation & Reporting in Reinforcement Learning

Linking pages

GitHub - google-research/rliable: [NeurIPS'21 Outstanding Paper] Library for reliable evaluation on RL and ML benchmarks, even with only a handful of seeds. https://github.com/google-research/rliable 8 comments

Linked pages

Scientists rise up against statistical significance https://www.nature.com/articles/d41586-019-00857-9 518 comments
Machine learning - Wikipedia http://en.wikipedia.org/wiki/Machine_learning 161 comments
Full article: Moving to a World Beyond “p < 0.05” https://www.tandfonline.com/doi/full/10.1080/00031305.2019.1583913 126 comments
Confidence interval - Wikipedia http://en.wikipedia.org/wiki/Confidence_interval 67 comments
GitHub - google-research/rliable: [NeurIPS'21 Outstanding Paper] Library for reliable evaluation on RL and ML benchmarks, even with only a handful of seeds. https://github.com/google-research/rliable 8 comments
[1207.4708] The Arcade Learning Environment: An Evaluation Platform for General Agents https://arxiv.org/abs/1207.4708 6 comments
[1803.07055] Simple random search provides a competitive approach to reinforcement learning https://arxiv.org/abs/1803.07055 4 comments
[1903.00374] Model-Based Reinforcement Learning for Atari https://arxiv.org/abs/1903.00374 3 comments
[1709.06560] Deep Reinforcement Learning that Matters https://arxiv.org/abs/1709.06560 3 comments
Google AI Blog: An Optimistic Perspective on Offline Reinforcement Learning https://ai.googleblog.com/2020/04/an-optimistic-perspective-on-offline.html 1 comment
Chip Design with Deep Reinforcement Learning – Google AI Blog https://ai.googleblog.com/2020/04/chip-design-with-deep-reinforcement.html 0 comments
Uncertainty quantification - Wikipedia https://en.wikipedia.org/wiki/Uncertainty_quantification#Aleatoric_and_epistemic_uncertainty 0 comments
Google AI Blog: Improving Generalization in Reinforcement Learning using Policy Similarity Embeddings http://ai.googleblog.com/2021/09/improving-generalization-in.html 0 comments
GitHub - deepmind/dm_control: DeepMind's software stack for physics-based simulation and Reinforcement Learning environments, using MuJoCo. https://github.com/deepmind/dm_control 0 comments
Procgen Benchmark https://openai.com/blog/procgen-benchmark/ 0 comments
Rliable: Better Evaluation for Reinforcement Learning - A Visual Explanation | Antonin Raffin | Homepage https://araffin.github.io/post/rliable/ 0 comments
Bootstrapping (statistics) - Wikipedia https://en.wikipedia.org/wiki/Bootstrapping_(statistics) 0 comments