Huang, Shengyi und Dossa, Rousslan Fernand Julien und Raffin, Antonin und Kanervisto, Anssi und Wang, Weixun (2022) The 37 Implementation Details of Proximal Policy Optimization. In: The ICLR Blog Track 2023. ICLR 2022, 2022-04-25 - 2022-04-29, Virtual.
Dieses Archiv kann nicht den Volltext zur Verfügung stellen.
Offizielle URL: https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/
Kurzfassung
Proximal policy optimization (PPO) has become one of the most popular deep reinforcement learning (DRL) algorithms. Yet, reproducing the PPO's results has been challenging in the community. While recent works conducted ablation studies to provide insight on PPO's implementation details, these works are not structured as tutorials and only focus on details concerning robotics tasks. As a result, reproducing PPO from scratch can become a daunting experience. Instead of introducing additional improvements, or doing further ablation studies, this blog post takes a step back and focuses on delivering a thorough reproduction of PPO in all accounts, as well as aggregating, documenting, and cataloging its most salient implementation details. This blog post also points out software engineering challenges in PPO and further efficiency improvement via the accelerated vectorized environments. With these, we believe this blog post will help people understand PPO faster and better, facilitating customization and research upon this versatile RL algorithm.
elib-URL des Eintrags: | https://elib.dlr.de/191986/ | ||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Dokumentart: | Konferenzbeitrag (Anderer) | ||||||||||||||||||||||||
Titel: | The 37 Implementation Details of Proximal Policy Optimization | ||||||||||||||||||||||||
Autoren: |
| ||||||||||||||||||||||||
Datum: | März 2022 | ||||||||||||||||||||||||
Erschienen in: | The ICLR Blog Track 2023 | ||||||||||||||||||||||||
Referierte Publikation: | Ja | ||||||||||||||||||||||||
Open Access: | Nein | ||||||||||||||||||||||||
Gold Open Access: | Nein | ||||||||||||||||||||||||
In SCOPUS: | Nein | ||||||||||||||||||||||||
In ISI Web of Science: | Nein | ||||||||||||||||||||||||
Status: | veröffentlicht | ||||||||||||||||||||||||
Stichwörter: | ppo, reinforcement learning, implementation, policy optimization | ||||||||||||||||||||||||
Veranstaltungstitel: | ICLR 2022 | ||||||||||||||||||||||||
Veranstaltungsort: | Virtual | ||||||||||||||||||||||||
Veranstaltungsart: | internationale Konferenz | ||||||||||||||||||||||||
Veranstaltungsbeginn: | 25 April 2022 | ||||||||||||||||||||||||
Veranstaltungsende: | 29 April 2022 | ||||||||||||||||||||||||
HGF - Forschungsbereich: | Luftfahrt, Raumfahrt und Verkehr | ||||||||||||||||||||||||
HGF - Programm: | Raumfahrt | ||||||||||||||||||||||||
HGF - Programmthema: | Robotik | ||||||||||||||||||||||||
DLR - Schwerpunkt: | Raumfahrt | ||||||||||||||||||||||||
DLR - Forschungsgebiet: | R RO - Robotik | ||||||||||||||||||||||||
DLR - Teilgebiet (Projekt, Vorhaben): | R - Autonome, lernende Roboter [RO] | ||||||||||||||||||||||||
Standort: | Oberpfaffenhofen | ||||||||||||||||||||||||
Institute & Einrichtungen: | Institut für Robotik und Mechatronik (ab 2013) Institut für Robotik und Mechatronik (ab 2013) > Kognitive Robotik | ||||||||||||||||||||||||
Hinterlegt von: | Raffin, Antonin | ||||||||||||||||||||||||
Hinterlegt am: | 08 Dez 2022 16:12 | ||||||||||||||||||||||||
Letzte Änderung: | 31 Mai 2024 09:20 |
Nur für Mitarbeiter des Archivs: Kontrollseite des Eintrags