Reinforcement learning for bandits with continuous actions and large context spaces

Conference item

We consider the challenging scenario of contextual bandits with continuous actions and large context spaces. This is an increasingly important application area in personalised healthcare where an agent is requested to make dosing decisions based on a patient’s single image scan. In this paper, we first adapt a reinforcement learning (RL) algorithm for continuous control to outperform contextual bandit algorithms specifically hand-crafted for continuous action spaces. We empirically demonstrate this on a suite of standard benchmark datasets for vector contexts. Secondly, we demonstrate that our RL agent can generalise problems with continuous actions to large context spaces, providing results that outperform previous methods on image contexts. Thirdly, we introduce a new contextual bandits test domain with multi-dimensional continuous action space and image contexts which existing tree-based methods cannot handle. We provide initial results with our RL agent.

Authors: Duckworth, P; Vallis, KA; Lacerda, B; Hawes, N

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: IOS Press
• Host title: ECAI 2023
• Pages: 590-597
• Series: Frontiers in Artificial Intelligence and Applications
• Series number: 372
• Publication date: 2023-09-28
• Event title: 26th European Conference on Artificial Intelligence (ECAI 2023)
• Event location: Kraków, Poland
• Event website: https://ecai2023.eu/
• Event start date: 2023-09-30
• Event end date: 2023-10-04
• DOI: 10.3233/FAIA230320
• EISSN: 1879-8314
• ISSN: 0922-6389
• EISBN: 9781643684376
• ISBN: 9781643684369
• Language: English