Numerical schemes for stochastic hybrid control problems in finance

Thesis

In this thesis, we propose a class of numerical schemes for weakly coupled systems of Hamilton-Jacobi-Bellman quasi-variational inequalities (HJBQVIs) arising from stochastic hybrid control problems of regime-switching models with both continuous and impulse controls. By penalizing the difference between the solution and the obstacles, we reduce the HJBQVIs into a sequence of HJB equations, whose solutions converge monotonically to those of the HJBQVIs. We show that the penalty scheme is first-order accurate for HJBQVIs with discrete state spaces, half-order accurate for HJBQVIs with Lipschitz coefficients, and first-order accurate for equations with more regular coefficients. We also demonstrate the convergence of monotone discretizations of the penalized equations, and establish that policy iteration applied to the discrete equation is monotonically convergent with an arbitrary initial guess in an infinite dimensional setting. Numerical examples for infinite-horizon optimal switching problems are presented to illustrate that the penalty scheme along with a continuation procedure in the penalty parameter is significantly more efficient than the conventional direct control scheme for solving HJBQVIs.

We further propose an efficient neural network-based policy iteration algorithm for solving semilinear Hamilton-Jacobi-Bellman-Isaacs (HJBI) boundary value problems, which not only can be combined with the penalty approach to solve high-dimensional stochastic hybrid control problems, but is also applicable to high-dimensional stochastic games of diffusion processes with controlled drift. The algorithm exploits policy iteration to reduce the semilinear problem into a sequence of linear Dirichlet problems, which are subsequently approximated by a multilayer feedforward neural network ansatz. We establish that the numerical solutions converge globally and further demonstrate that this convergence is superlinear, by interpreting the algorithm as an inexact Newton iteration for the HJBI equation. Moreover, we construct the optimal feedback controls from the numerical value functions and deduce convergence. Numerical experiments on the stochastic Zermelo navigation problem are presented to illustrate the theoretical results and to demonstrate the effectiveness of the method.

Authors: Zhang, Y

• DOI: 10.5287/ora-e287owypk
• Type of award: DPhil
• Level of award: Doctoral
• Awarding institution: University of Oxford
• Language: English
• Keywords: error estimate; policy iteration; Hamilton-Jacobi-Bellman quasi-variational inequalities; neural network; impulse control; penalty method
• Subjects: Mathematics