Asymptotic bias of stochastic gradient search

Journal article

The asymptotic behavior of the stochastic gradient algorithm with a biased gradient estimator is analyzed. Relying on arguments based on differential geometry (Yomdin theorem and Lojasiewicz inequality), relatively tight bounds on the asymptotic bias of the iterates generated by such an algorithm are derived. The obtained results hold under mild and verifiable conditions and cover a broad class of complex stochastic gradient algorithms. Using these results, the asymptotic properties of the actor-critic reinforcement learning are studied. © 2011 IEEE.

Authors: Tadic, V; Doucet, A

• Journal: Proceedings of the IEEE Conference on Decision and Control
• Pages: 722-727
• Publication date: 2011-01-01
• DOI: 10.1109/CDC.2011.6160812
• ISSN: 0191-2216
• Language: English
• Keywords: biased gradient estimation; reinforcement learning; Stochastic gradient search