How do I solve an optimal control problem where the dynamics depend on some function of the state?

Question

A typical optimal control problem with state $x(t)$ and control $y(t)$ can be expressed as $$\max_{x(t), y(t)} \int_0^{t_1} f(t,x(t), y(t)) dt$$ subject to $x'(t)= g(t, x(t), y(t))$ and boundary conditions for $x(t)$.

I want to solve a problem that looks very similar, but the dynamics are $$x'(t)= g(t, x(t), y(t), z(x(t)) ),$$ where, the sequence $z(\cdot)$ is to be chosen although it's argument is the state.

I'm not sure where one can look for solutions to this non-standard problem. How can I approach this problem?

Answer 1

Why would $z$ need to be external to $g$?

$$g'(t,x(t),y(t))=g(t,x(t),y(t),z(x(t)))$$

now use $g'$ as $g$

$g$ can be any arbitrary function, so any function $z$ could just be incorporated into $g$.

Regarding your restriction $h$ mentioned in the comment section. Any restrictions on the control input could be enforced via the cost function:

$$f_{new}(t,x(t),y(t))=f_{old}(t,x(t),y(t))-C\,h(x(t),y(t))^2$$

Where $C$ is sufficiently large to guarantee values of $h$ close enough to zero but not so large that numerical errors in $h$ would dominate the original $f$.

Answer 2

You can use discretization of the problem into $N$ points, such that you only have to determine a finite number of parameters (assuming $f$ and $g$ are somewhat continuous functions). For the derivative and integration you can use Euler method, higher order methods can be used, but make the problem harder to solve.

The reformulation gives: $$ h = \frac{t_1}{N-1}, \quad \vec{x} = [x_1, x_2, \dots, x_N], \quad \vec{y} = [y_1, y_2, \dots, y_N], $$

$$ \begin{align} \max_{\vec{x}, \vec{y}} & & \sum_{n=1}^{N-1} f(h (n-1), x_n, y_n) h\\ s.t. & & x_{n+1} = x_n + g(h (n-1), x_n, y_n) h, & & n = 1, 2, \dots, N-1 \end{align} $$

You also have to add the boundary constraints to the equality constraints of the optimization problem. You can use multiple different methods to solve this problem, for example if you have access to Matlab you could use fmincon, which minimizes the cost function which can be fixed by adding an minus sign in front of the sum. Often you also have to supply an initial guess, which might also affect the solution, since different guesses might converge to different local maxima. By increasing $N$ you should get a more and more accurate solution, but it will probably take longer to solve. It might converge faster if you use the solution of a problem with less points and interpolate them and then use that as an initial guess for the problem of the larger number of points.