Optimal Control: LQR, Riccati, and the Kalman Filter

Optimal control is the branch of control theory that asks not merely "can we stabilize this system?" but "what is the best way to drive it, given a precise definition of best?" It is the point where control stops being a craft of loop-shaping and becomes a problem in variational calculus — and, for the linear-quadratic case, a problem with a clean closed-form answer that you can compute in a handful of matrix operations[fn::The word "optimal" does a lot of work in this name. It means optimal with respect to a specified quadratic cost. Change the cost and the optimal controller changes. The practical art is not in solving the Riccati equation — that is mechanical — but in choosing a cost that encodes what you actually care about.].

The structure of the subject is unusually clean. There is a state equation, a cost functional, and a minimization. When the dynamics are linear and the cost is quadratic, the solution is a linear feedback law whose gain comes from a matrix Riccati equation. When you add process noise and measurement noise — Gaussian, white — the state estimator that minimizes mean-square error is the Kalman filter, and the separation principle lets you compose the two independently. This composition is the LQG controller, and it is the workhorse of modern industrial control[fn::"Modern" here means post-1960. The LQR dates to Kalman's 1960 papers; the LQG separation principle to later work by Gunckel and Franklin, and Doyle and Stein. It is old enough to have grey hair, and yet it remains the thing you reach for first when the model is linear and the noise is roughly Gaussian.].

The setup

Consider the continuous-time LTI system

ẋ(t) = A x(t) + B u(t), x(0) = x₀,

with state x ∈ ℝⁿ and input u ∈ ℝᵐ. We want a feedback law u = −K x that drives the state to zero while trading off control effort against state deviation. The infinite-horizon quadratic cost is

J = ∫₀^∞ (xᵀ Q x + uᵀ R u) dt,

with Q ⪰ 0 (state penalty) and R ≻ 0 (input penalty). The matrices Q and R are the design knobs: Q says which states you mind letting wander, R says how much each actuator costs to fire.

Why quadratic? Two reasons, one principled and one pragmatic. The principled one: a quadratic cost penalizes large deviations disproportionately, which matches the intuition that a 2× overshoot is more than twice as bad as a 1× overshoot. The pragmatic one: a quadratic cost paired with linear dynamics yields a linear optimal control law, and linear laws are things you can actually implement with a matrix multiply[fn::A quartic cost would be "more nonlinear" but would yield a nonlinear feedback law u = −K(x)x, where the gain itself depends on state. That is implementable in software but far harder to verify, and the gain-scheduling headaches are severe. The quadratic choice is the one that keeps the whole pipeline linear.].

The algebraic Riccati equation

The optimal gain is

K = R⁻¹ Bᵀ P,

where P is the unique symmetric positive-semidefinite solution of the continuous algebraic Riccati equation (CARE):

AᵀP + PA − PBR⁻¹BᵀP + Q = 0.

This is a matrix quadratic equation — nonlinear in P because of the P(···)P term — but it has a unique stabilizing solution under mild assumptions (stabilizable (A,B), detectable (Q^{1/2}, A)). You solve it numerically (MATLAB's lqr, SciPy's solve_continuous_are); you almost never solve it by hand for n > 2.

The closed-loop matrix A − BK is guaranteed Hurwitz (all eigenvalues in the open left half-plane) when the problem is feasible. That is the stability guarantee: not a loop you tuned until it looked right, but a theorem[fn::The proof runs through the Lyapunov function V = xᵀPx. Along closed-loop trajectories, V̇ = −xᵀ(Q + KᵀRK)x ≤ 0, with equality only at x = 0. So V is a strict Lyapunov function and the origin is asymptotically stable. The Riccati equation is, in this sense, a designer Lyapunov equation — you pick Q and R, and the equation hands you a P whose quadratic form is the Lyapunov function.].

Worked example: the double integrator

The double integrator — ẍ = u, or in state form ẋ₁ = x₂, ẋ₂ = u — is the LQR equivalent of a "hello, world." It is a mass with a force on it, no damping, no spring. Take Q = I₂, R = 1. The CARE becomes a system of three scalar equations (P is symmetric 2×2); solving gives approximately

P ≈ [[√2, 1], [1, √2]], K ≈ [1, √2].

The closed-loop poles are the roots of s² + √2 s + 1, which are at −√2/2 ± j√2/2 — a damping ratio of ζ = cos(45°) ≈ 0.707. That is, the optimal LQR controller for Q = I, R = 1 places the poles at 45° on the second-order damping diagram. This is not a coincidence: it is a special case of the general result that LQR for a second-order system with equal state and input weights yields ζ ≈ 0.707, the "comfortable" damping that textbook loop-shaping also converges to[fn::This is why LQR is sometimes called "automatic loop-shaping": it reproduces, with less knobs, the damping ratios an experienced engineer would pick by hand for a second-order plant. For higher-order plants the correspondence is less clean, but the intuition that LQR tends to place poles in a "well-damped" region of the s-plane holds.].

The Kalman filter

The dual of the LQR is the Kalman-Bucy filter — the optimal state estimator for a linear system driven by Gaussian white noise. The plant is

ẋ = Ax + Bu + w, y = Cx + v,

with w ~ N(0, W) process noise and v ~ N(0, V) measurement noise. The estimator is

x̂̇ = Ax̂ + Bu + L(y − Cx̂),

where the gain L minimizes the steady-state error covariance. Duality says L = PCᵀV⁻¹ where P solves the filter Riccati equation,

AP + PAᵀ − PCᵀV⁻¹CP + W = 0,

which is the CARE with the substitutions A → Aᵀ, B → Cᵀ, Q → W, R → V. The controller and estimator are *the same equation, transposed*[fn::This duality, observed by Kalman in 1960, is one of the deeper structural facts in control. It says estimation and control are not two problems but one problem viewed from opposite ends. The practical consequence: any code that solves the CARE solves both the LQR and the Kalman filter, depending on which matrices you feed it.].

The separation principle and LQG

Compose the LQR controller and the Kalman estimator: use the Kalman filter to produce x̂ from noisy measurements y, then apply u = −K x̂ as if x̂ were the true state. The separation principle (or certainty-equivalence principle) says this composition is optimal for the LQG problem — minimize a quadratic cost on x and u subject to the noisy LTI plant — and that you can design K and L independently.

This is a remarkable result. It says the controller does not need to know how good the estimator is; it just operates on the estimate. And the estimator does not need to know what the controller will do with the estimate; it just estimates. The two halves are decoupled, designed separately, and composed by simple substitution[fn::The separation principle is fragile in ways that matter. It holds exactly for LQG — linear, quadratic, Gaussian. Drop any of the three adjectives and it generally fails: nonlinear dynamics, non-quadratic costs, or non-Gaussian noise each break the clean separation. Robust control (see Robust Control) was largely motivated by the fact that LQG, despite being "optimal" for its assumed model, has no guaranteed stability margin when the model is wrong — the optimality is conditional on assumptions that never quite hold.].

Why LQR is not the end of the story

The LQG controller is optimal for the model it was designed against and only for that model. It has no robustness guarantee. Doyle's 1978 result showed that an LQG controller can have arbitrarily small stability margin — a tiny plant perturbation can destabilize a controller that is, by construction, optimal for the nominal plant. This is not a bug; it is the inevitable consequence of optimizing for a single model. Robustness requires you to optimize against a set of plants, which is the province of H∞ control.

The practical takeaway: LQR/LQG is the right first controller to try. It is fast to compute, gives you a baseline, and for well-modelled plants with modest uncertainty it works beautifully. When the model is uncertain or the noise is non-Gaussian, you graduate to robust or nonlinear methods — but you graduate from LQG, not around it.

Discrete-time LQR

For a sampled system x_{k+1} = A_d x_k + B_d u_k, the discrete LQR minimizes

J = Σ_{k=0}^∞ (xᵀ_k Q x_k + uᵀ_k R u_k),

and the optimal gain is K = (R + Bᵀ_d P B_d)⁻¹ Bᵀ_d P A_d, where P solves the discrete algebraic Riccati equation (DARE):

P = Aᵀ_d P A_d − Aᵀ_d P B_d (R + Bᵀ_d P B_d)⁻¹ Bᵀ_d P A_d + Q.

The structure is the same; only the bookkeeping changes (a + where the continuous version had a −, because the cost-to-go recursion runs forward in discrete time). See Digital Control for the z-transform context.

Related

• State-Space Representation — the plant model LQR operates on
• Frequency Response — the loop-shaping alternative to LQR
• Robust Control — what you graduate to when the model is uncertain
• Nonlinear Control — when linearity itself fails
• Digital Control — the discrete-time / DARE variant
• PID Control — the classical predecessor; LQR reduces to PID for low-order plants with diagonal Q, R