Asymptotic randomised control with applications to bandits

Journal article

We consider a general multi-armed bandit problem with correlated (and simple contextual and restless) elements, as a relaxed control problem. By introducing an entropy premium, we obtain a smooth asymptotic approximation to the value function. This yields a novel semi-index approximation of the optimal decision process. This semi-index can be interpreted as explicitly balancing an exploration–exploitation trade-off as in the UCB (Upper Confidence Bound) principle where the learning premium explicitly describes asymmetry of information available in the environment and non-linearity in the reward function.

Performance of the resulting Asymptotic Randomised Control (ARC) algorithm compares favourably well with other approaches to correlated multi-armed bandits.

Authors: Cohen, SN; Treetanthiploet, T

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: American Institute of Mathematical Sciences
• Journal: Numerical Algebra, Control and Optimization
• Publication date: 2026-05-07
• Acceptance date: 2026-02-20
• DOI: 10.3934/naco.2026016
• EISSN: 2155-3297
• ISSN: 2155-3289
• Language: English
• Keywords: multi-armed bandit; stochastic control; asymptotic approximation