Stability of the optimal control law

Question

in linear optimal control,linear quadratic regulator,we have a system of the form: x=Ax+Bu,the optimal control law U is a state feedback,it's a function of the riccati equation solution and the state vector.I'm asking about stability,is the optimal control law U always stable?is the system is always stable? Can you give me some references about the demonstration?

Answer 1

So, just to formally repeat your question, we consider an infinite horizon continuous-time optimal control problem

$J^*(x_0) = min \int_0^{\infty} x(t)^TQx(t)+u(t)^TRu(t)dt$

which is subject to some constraints, basically just the system dynamics and the initial condition

$\dot x (t) = A x(t) + B u(t) \\ x(0) = x_0 $

We assume the cost matrices $Q = C^TC$ to be positive semidefinite and $R$ to be positive definite.

Then, if the pair $(A,B)$ is controllable and the pair $(A,C)$ is observable, we can find a unique, positive definite solution $P^*$ for the ARE (algebraic riccati equation)

$P^* A+A^TP^* -P^*BR^{-1}B^TP^* + Q = 0$

There are three nice statements we can make with this solution $P^*$:

1. The optimal cost is simply $J^*(x_0)=x_0^TP^*x_0$
2. The optimal input is $u^*(t)=-R^{-1}B^TP^*x^*(t)$
3. The closed loop $\dot x^* = (A-BR^{-1}B^TP^*)x^*$ is asymptotically stable.

The idea for the proof of statement 3 is, to take the optimal cost as a Lyapunov function.

$V(x) = x^TP^*x$, which is positive definite (because $P^*$ is positive definite)

So now we still need to show that $\dot V$ is negative semidefinite.

$\dot V \ = 2x^TP^*\dot x \\ \quad = 2x^TP^*(A-BR^{-1}B^TP^*)x \\ \quad = 2(x^TP^*Ax-x^TP^*BR^{-1}B^TP^*x) \\ \quad = x^T(P^*A+A^TP^*)x - 2x^T(P^*BR^{-1}B^TP^*)x \\ \quad = -x^TQx - x^TP^*BR^{-1}B^TP^*x$

Both $Q$ as well as $P^*BR^{-1}B^TP^*$ are positive semidefinite, so $\dot V$ is negative semidefinite.

This proves stability at the origin.

When we now consider the biggest invariant set $\{x \in \mathbb{R}^n|\dot V(x)=0\}$ and consider the output $y = Cx$. Since both $x^TQx$ as well as $x^TP^*BR^{-1}B^TP^*x$ are nonnegative, they must both independently be zero. With $x^TP^*BR^{-1}B^TP^*x=x^TP^*BR^{-1}RR^{-1}B^TP^*x = u^TRu \equiv 0$ we know that $u \equiv0$. We also know that $y^Ty \equiv 0$, which leads to

$y = Cx = 0 \\ \dot y = CAx = 0 \\ \ddot y = CA^2x = 0 \\ \vdots $

We made the initial assumption that (A,C) is observable, so the matrix $\begin{bmatrix}{\ C \\CA\\ \ \ \vdots}\end{bmatrix}$ has full rank and $x \equiv 0$ is the only invariant solution in the biggest invariant set $\{x \in \mathbb{R}^n|\dot V(x)=0\}$.

This (finally) yields that the origin is asymptotically stable.

I hope I didn't make a mistake somewhere, I'll look through it again later. For further reference, you might want to read for example this paper.