# How do I solve an optimal control problem where the dynamics depend on some function of the state?

A typical optimal control problem with state $$x(t)$$ and control $$y(t)$$ can be expressed as $$\max_{x(t), y(t)} \int_0^{t_1} f(t,x(t), y(t)) dt$$ subject to $$x'(t)= g(t, x(t), y(t))$$ and boundary conditions for $$x(t)$$.

I want to solve a problem that looks very similar, but the dynamics are $$x'(t)= g(t, x(t), y(t), z(x(t)) ),$$ where, the sequence $$z(\cdot)$$ is to be chosen although it's argument is the state.

I'm not sure where one can look for solutions to this non-standard problem. How can I approach this problem?

• I guess the correct way to write it is $$x'(t) = g(t,x(t), y(t), z(x(t))$$. I will correct the original question. Apr 14 '15 at 23:41
• Welcome to engineering.SE, +1 for an excellent first question. Apr 15 '15 at 0:10
• Are you looking for a closed-form or formal solution, or are you asking about practical optimization? In the former case you should ask this on a site like math.stackexchange.com. In the latter case there is a range of disciplines devoted to practical optimization. In either case you need to provide more details to get a real answer. Apr 16 '15 at 1:35
• I am looking for practical optimization. More details: A subset of the control variable depend on $t$ (which I am calling $y$), and a subset of the control variables depend on $x$ (which I am calling $z$). Additionally I need to choose the function $x(t)$. The maximization is subject to a restriction $$h(z(x(t)), y(t)) =0$$ So an intuitive way to solve it is: - guess $x(t)$ - solve the (now standard) optimal control problem (given $x(t)$) - check if $h(z(x(t)), y(t)) =0$, if not, guess another $x(t)$ But you see there's no reason for the algorithm will converge. How do people solve this? Apr 17 '15 at 15:42

Why would $z$ need to be external to $g$?

$$g'(t,x(t),y(t))=g(t,x(t),y(t),z(x(t)))$$

now use $g'$ as $g$

$g$ can be any arbitrary function, so any function $z$ could just be incorporated into $g$.

Regarding your restriction $h$ mentioned in the comment section. Any restrictions on the control input could be enforced via the cost function:

$$f_{new}(t,x(t),y(t))=f_{old}(t,x(t),y(t))-C\,h(x(t),y(t))^2$$

Where $C$ is sufficiently large to guarantee values of $h$ close enough to zero but not so large that numerical errors in $h$ would dominate the original $f$.

You can use discretization of the problem into $N$ points, such that you only have to determine a finite number of parameters (assuming $f$ and $g$ are somewhat continuous functions). For the derivative and integration you can use Euler method, higher order methods can be used, but make the problem harder to solve.

The reformulation gives: $$h = \frac{t_1}{N-1}, \quad \vec{x} = [x_1, x_2, \dots, x_N], \quad \vec{y} = [y_1, y_2, \dots, y_N],$$

\begin{align} \max_{\vec{x}, \vec{y}} & & \sum_{n=1}^{N-1} f(h (n-1), x_n, y_n) h\\ s.t. & & x_{n+1} = x_n + g(h (n-1), x_n, y_n) h, & & n = 1, 2, \dots, N-1 \end{align}

You also have to add the boundary constraints to the equality constraints of the optimization problem. You can use multiple different methods to solve this problem, for example if you have access to Matlab you could use fmincon, which minimizes the cost function which can be fixed by adding an minus sign in front of the sum. Often you also have to supply an initial guess, which might also affect the solution, since different guesses might converge to different local maxima. By increasing $N$ you should get a more and more accurate solution, but it will probably take longer to solve. It might converge faster if you use the solution of a problem with less points and interpolate them and then use that as an initial guess for the problem of the larger number of points.