Journal article
The computational complexity of finding stationary points in non-convex optimization
- Abstract:
-
Finding approximate stationary points, i.e., points where the gradient is approximately zero, of non-convex but smooth objective functions f over unrestricted d-dimensional domains is one of the most fundamental problems in classical non-convex optimization. Nevertheless, the computational and query complexity of this problem are still not well understood when the dimension d of the problem is independent of the approximation error. In this paper, we show the following computational and query complexity results:
- The problem of finding approximate stationary points over unrestricted domains is PLScomplete.
- For d = 2, we provide a zero-order algorithm for finding ε-approximate stationary points that requires at most O(1/ε) value queries to the objective function.
- We show that any algorithm needs at least Ω(1/ε) queries to the objective function and/or its gradient to find ε-approximate stationary points when d = 2. Combined with the above, this characterizes the query complexity of this problem to be Θ(1/ε).
- For d = 2, we provide a zero-order algorithm for finding ε-KKT points in constrained optimization problems that requires at most O(1/ √ ε) value queries to the objective function. This closes the gap between the works of Bubeck and Mikulincer [2020] and Vavasis [1993] and characterizes the query complexity of this problem to be Θ(1/ √ ε).
- Combining our results with the recent result of Fearnley et al. [2022], we show that finding approximate KKT points in constrained optimization is reducible to finding approximate stationary points in unconstrained optimization but the converse is impossible.
- Publication status:
- Published
- Peer review status:
- Peer reviewed
Actions
Access Document
- Files:
-
-
(Preview, Version of record, pdf, 3.6MB, Terms of use)
-
- Publisher copy:
- 10.1007/s10107-024-02139-3
Authors
- Publisher:
- Springer
- Journal:
- Mathematical Programming More from this journal
- Volume:
- 213
- Issue:
- 1
- Pages:
- 281-341
- Publication date:
- 2024-09-27
- Acceptance date:
- 2024-08-12
- DOI:
- EISSN:
-
1436-4646
- ISSN:
-
0025-5610
- Language:
-
English
- Pubs id:
-
2026925
- Local pid:
-
pubs:2026925
- Deposit date:
-
2024-09-11
- ARK identifier:
Terms of use
- Copyright holder:
- Hollender and Zampetakis
- Copyright date:
- 2024
- Rights statement:
- Copyright © 2024, The Author(s). This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
- Licence:
- CC Attribution (CC BY)
If you are the owner of this record, you can report an update to it here: Report update to this record