Conference item
Stochastic expectation maximization with variance reduction
- Abstract:
- Expectation-Maximization (EM) is a popular tool for learning latent variable models, but the vanilla batch EM does not scale to large data sets because the whole data set is needed at every E-step. Stochastic Expectation Maximization (sEM) reduces the cost of E-step by stochastic approximation. However, sEM has a slower asymptotic convergence rate than batch EM, and requires a decreasing sequence of step sizes, which is difficult to tune. In this paper, we propose a variance reduced stochastic EM (sEM-vr) algorithm inspired by variance reduced stochastic gradient descent algorithms. We show that sEM-vr has the same exponential asymptotic convergence rate as batch EM. Moreover, sEM-vr only requires a constant step size to achieve this rate, which alleviates the burden of parameter tuning. We compare sEM-vr with batch EM, sEM and other algorithms on Gaussian mixture models and probabilistic latent semantic analysis, and sEM-vr converges significantly faster than these baselines.
- Publication status:
- Published
- Peer review status:
- Peer reviewed
Actions
Access Document
- Files:
-
-
(Preview, Version of record, pdf, 552.1KB, Terms of use)
-
Authors
- Publisher:
- Massachusetts Institute of Technology Press
- Host title:
- Neural Information Processing Systems
- Journal:
- Neural Information Processing Systems More from this journal
- Publication date:
- 2018-12-02
- Acceptance date:
- 2018-12-02
- Event location:
- Montréal, Canada
- ISSN:
-
1049-5258
- Pubs id:
-
pubs:969052
- UUID:
-
uuid:e1b7d53c-8ac5-42cc-a12d-47abdd363c8c
- Local pid:
-
pubs:969052
- Source identifiers:
-
969052
- Deposit date:
-
2019-02-06
- ARK identifier:
Terms of use
- Copyright date:
- 2018
- Notes:
- This paper was presented at the 32nd Conference on Neural Information Processing Systems (NIPS 2018). The article is also available here: https://papers.nips.cc/paper/8021-stochastic-expectation-maximization-with-variance-reduction
If you are the owner of this record, you can report an update to it here: Report update to this record