in linear optimal control,linear quadratic regulator,we have a system of the form: x=Ax+Bu,the optimal control law U is a state feedback,it's a function of the riccati equation solution and the state vector.I'm asking about stability,is the optimal control law U always stable?is the system is always stable? Can you give me some references about the demonstration?

  • 1
    $\begingroup$ The LQR controller is indeed stabilizing, to prove it consider 1/2 x‘ P x as Lyapunov function, P being the solution of the algebraic riccati equation, if there is no answer later today I can explain it in more detail $\endgroup$ – OpticalResonator Apr 12 '18 at 10:41

So, just to formally repeat your question, we consider an infinite horizon continuous-time optimal control problem

$J^*(x_0) = min \int_0^{\infty} x(t)^TQx(t)+u(t)^TRu(t)dt$

which is subject to some constraints, basically just the system dynamics and the initial condition

$\dot x (t) = A x(t) + B u(t) \\ x(0) = x_0 $

We assume the cost matrices $Q = C^TC$ to be positive semidefinite and $R$ to be positive definite.

Then, if the pair $(A,B)$ is controllable and the pair $(A,C)$ is observable, we can find a unique, positive definite solution $P^*$ for the ARE (algebraic riccati equation)

$P^* A+A^TP^* -P^*BR^{-1}B^TP^* + Q = 0$

There are three nice statements we can make with this solution $P^*$:

  1. The optimal cost is simply $J^*(x_0)=x_0^TP^*x_0$
  2. The optimal input is $u^*(t)=-R^{-1}B^TP^*x^*(t)$
  3. The closed loop $\dot x^* = (A-BR^{-1}B^TP^*)x^*$ is asymptotically stable.

The idea for the proof of statement 3 is, to take the optimal cost as a Lyapunov function.

$V(x) = x^TP^*x$, which is positive definite (because $P^*$ is positive definite)

So now we still need to show that $\dot V$ is negative semidefinite.

$\dot V \ = 2x^TP^*\dot x \\ \quad = 2x^TP^*(A-BR^{-1}B^TP^*)x \\ \quad = 2(x^TP^*Ax-x^TP^*BR^{-1}B^TP^*x) \\ \quad = x^T(P^*A+A^TP^*)x - 2x^T(P^*BR^{-1}B^TP^*)x \\ \quad = -x^TQx - x^TP^*BR^{-1}B^TP^*x$

Both $Q$ as well as $P^*BR^{-1}B^TP^*$ are positive semidefinite, so $\dot V$ is negative semidefinite.

This proves stability at the origin.

When we now consider the biggest invariant set $\{x \in \mathbb{R}^n|\dot V(x)=0\}$ and consider the output $y = Cx$. Since both $x^TQx$ as well as $x^TP^*BR^{-1}B^TP^*x$ are nonnegative, they must both independently be zero. With $x^TP^*BR^{-1}B^TP^*x=x^TP^*BR^{-1}RR^{-1}B^TP^*x = u^TRu \equiv 0$ we know that $u \equiv0$. We also know that $y^Ty \equiv 0$, which leads to

$y = Cx = 0 \\ \dot y = CAx = 0 \\ \ddot y = CA^2x = 0 \\ \vdots $

We made the initial assumption that (A,C) is observable, so the matrix $\begin{bmatrix}{\ C \\CA\\ \ \ \vdots}\end{bmatrix}$ has full rank and $x \equiv 0$ is the only invariant solution in the biggest invariant set $\{x \in \mathbb{R}^n|\dot V(x)=0\}$.

This (finally) yields that the origin is asymptotically stable.

I hope I didn't make a mistake somewhere, I'll look through it again later. For further reference, you might want to read for example this paper.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.