![]() ![]() As a result, the computational and sample costs scale poorly in the number of agents.Īnother approach to extend single-agent reinforcement learning methods to multi-agent systems is to apply them to each agent independently. Moreover, selecting a joint action that maximizes the Q-function usually requires that, as in deep Q-networks, the (now joint) actions are output nodes of the network. However, this approach heavily relies on the function approximation abilities of the neural network, since it must generalize across a discrete action space whose size is exponential in the number of agents. A straightforward way to extend such methods to the multi-agent setting is by simply replacing the action by the joint action ⟨ a 1, ⋯, a n ⟩ of all agents Q ( s, ⟨ a 1, ⋯, a n ⟩ ; θ ). Single-agent value-based reinforcement learning methods use (deep) neural networks to represent the discrete action-value function Q ( s, a ; θ ) to select actions directly or as a ‘critic’ in an actor-critic scheme. While these approaches have shown good results, there is a general lack of theoretical insight, and often it remains unclear what the neural networks used by these approaches are learning, or how we should enhance their learning power to address the problems on which they fail. In recent years, a variety of deep MARL approaches have been developed and successfully applied. Multi-agent reinforcement learning (MARL) uses reinforcement learning to train multiple agents for such systems, and can lead to flexible and robust solutions. (Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS’19.International Foundation for Autonomous Agents and Multiagent Systems, pp 1862–1864, 2019) and quantify how well various approaches can represent the requisite value functions, and help us identify the reasons that can impede good performance, like sparsity of the values or too tight coordination requirements. ![]() Our results extend those in Castellini et al. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. In this work, we empirically investigate the learning power of various network architectures on a series of one-shot games. However, given the lack of theoretical insight, it remains unclear what the employed neural networks are learning, or how we should enhance their learning power to address the problems on which they fail. Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |