Scalable Robust Multi-Agent Reinforcement Learning for Model Uncertainty
- Title
- Scalable Robust Multi-Agent Reinforcement Learning for Model Uncertainty
- Authors
- Jwa, Younkyung; Gwak, Minseon; Kwak, Jiin; Ahn, Chang Wook; PARK, POOGYEON
- Date Issued
- 2023-12-15
- Publisher
- Institute of Electrical and Electronics Engineers Inc.
- Abstract
- A robust multi-agent reinforcement learning (MARL) algorithm using a nature actor has been shown to be effective in finding a robust Nash equilibrium (NE) of a Markov game with model uncertainty. However, since a game-size scaling increases the search space and challenges reaching the NE, the robust property of the algorithm is reduced in environments with many agents. This paper proposes an evolutionary diversity-maintaining population curriculum (EDPC) framework with a robust attention-based multi-agent deep deterministic policy gradient (RA-MADDPG) algorithm, which enables an efficient robust NE search by a structured search space expansion. In the EDPC framework, the MARL divides into several stages, and when moving on to the next stage, a population consisting of larger games is made with two parent games from the previous stage. We introduce reward-proportionate parent selection and reward-guided mutation methods to continue reinforcing superior agents and maintain the diversity of the population. Furthermore, the RA-MADDPG is used to solve the robust Markov game at each stage with nature actors with attention-based architectures. The scalability and robustness of the proposed method are evaluated for different numbers of agents and levels of model uncertainty.
- URI
- https://oasis.postech.ac.kr/handle/2014.oak/121458
- Article Type
- Conference
- Citation
- 62nd IEEE Conference on Decision and Control, CDC 2023, page. 3402 - 3407, 2023-12-15
- Files in This Item:
- There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.