Constrained Optimisation: Lagrange & KKT

Real designs have limits: stresses, budgets, capacities. Learn equality constraints and Lagrange multipliers, the geometric meaning (gradients parallel), inequality constraints and KKT conditions, active constraints, shadow prices (sensitivity), and penalty and SQP methods.

Minimise weight subject to stress limits; maximise profit subject to resources; minimise energy subject to physics. Lagrange multipliers handle these systematically and tell you how much the optimum improves if a limit is relaxed.

An analogy

Imagine walking on a hillside but confined to a path (the constraint): the lowest point on the path is where the path runs level with the contour lines — gradients of objective and constraint are parallel.

Substitution first

With one simple equality constraint, eliminate a variable and optimise normally — but this becomes messy for many constraints.

Lagrange multipliers

At the optimum, ∇f is a combination of constraint gradients. Solve the system of stationarity plus constraint equations. λ measures how much the optimum changes per unit change of the constraint.

KKT conditions

For inequalities: inactive constraints have μ = 0; active ones satisfy h = 0 with μ ≥ 0. Guess the active set, solve, then check signs.

Algorithms

Penalty and barrier methods, augmented Lagrangian, sequential quadratic programming (SQP — fmincon, SLSQP), interior-point methods. Convex problems (convex objective, convex feasible set) are reliably solvable.

Formulas

• Lagrangian: L = f(x) + λg(x) (+ Σμⱼhⱼ(x))
• Lagrange conditions: ∇f + λ∇g = 0, g = 0
• KKT: ∇f + Σλ∇g + Σμ∇h = 0; μ ≥ 0; μⱼhⱼ = 0 (complementary slackness)
• Shadow price: λ = −∂f*/∂(constraint value)

Worked example: Lagrange multiplier

Given: Minimise x² + y² subject to x + y = 10

Find: Optimum and λ.

• 2x + λ = 0, 2y + λ = 0 → x = y
• x + y = 10 → x = y = 5, λ = −10

Answer: (5, 5), f = 50; |λ| = 10 → f rises ≈ 10 per unit increase of the constant

Worked example: Box with fixed surface

Given: Open-top square box, surface 108 m²

Find: Maximum volume.

• V = x²h, x² + 4xh = 108
• Lagrange → x = 6, h = 3

Answer: V = 108 m³

Common mistakes

• Wrong: Ignoring the sign of KKT multipliers. — Right: μ ≥ 0 for active inequalities, else wrong active set.
• Wrong: Forgetting constraint equations in the system. — Right: Stationarity + feasibility together.

Applications

• Structural optimisation: Minimum weight under stress limits.
• Economics of production: Shadow prices of resources.
• Optimal control: Pontryagin's principle generalises Lagrange.

Key takeaways

• L = f + λg; solve stationarity + constraints.
• KKT for inequalities: μ ≥ 0, μh = 0.
• Multipliers = shadow prices.
• SQP / interior point in practice.

Going further

• Duality: Lagrange dual problems and bounds.
• Interior-point methods: Barrier functions for large problems.
• Topology optimisation: Material layout under constraints.