Mega-reward: Achieving human-level play without extrinsic rewards

Conference item

Intrinsic rewards were introduced to simulate how human intelligence works; they are usually evaluated by intrinsically-motivated play, i.e., playing games without extrinsic rewards but evaluated with extrinsic rewards. However, none of the existing intrinsic reward approaches can achieve human-level performance under this very challenging setting of intrinsically-motivated play. In this work, we propose a novel megalomania-driven intrinsic reward (called mega-reward), which, to our knowledge, is the first approach that achieves human-level performance in intrinsically-motivated play. Intuitively, mega-reward comes from the observation that infants' intelligence develops when they try to gain more control on entities in an environment; therefore, mega-reward aims to maximize the control capabilities of agents on given entities in a given environment. To formalize mega-reward, a relational transition model is proposed to bridge the gaps between direct and latent control. Experimental studies show that mega-reward (i) can greatly outperform all state-of-the-art intrinsic reward approaches, (ii) generally achieves the same level of performance as Ex-PPO and professional human-level scores, and (iii) has also a superior performance when it is incorporated with extrinsic rewards.

Authors: Song, Y; Wang, J; Lukasiewicz, T; Xu, Z; Zhang, S

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Association for the Advancement of Artificial Intelligence
• Journal: 34th AAAI Conference on Artificial Intelligence (AAAI 2020)
• Volume: 34
• Issue: 4
• Pages: 5826-5833
• Publication date: 2020-06-16
• Acceptance date: 2019-11-11
• Event title: 34th AAAI Conference on Artificial Intelligence (AAAI 2020)
• Event location: New York, New York, USA
• Event website: https://aaai.org/Conferences/AAAI-20/
• Event start date: 2020-02-07
• Event end date: 2020-02-12
• DOI: 10.1609/aaai.v34i04.6040
• EISSN: 2374-3468
• ISSN: 2159-5399
• ISBN: 9781577358091
• Language: English
• Keywords: FFR