Linear Systems & Rank

Solving Ax = b is the most common computation in engineering. Learn augmented matrices, Gaussian and Gauss–Jordan elimination, row echelon forms, rank and the consistency test (Rouché–Capelli), unique/no/infinitely many solutions, homogeneous systems, and conditioning — with a NumPy example.

Mesh currents in a circuit, forces in a truss, temperatures on an FEM grid, least-squares fits — all reduce to linear systems, often with millions of unknowns. Understanding elimination and rank tells you when a solution exists and how reliable it is.

An analogy

Each equation is a constraint (a plane); the solution is where all planes meet. They can meet at a point, never meet, or share a whole line.

Gaussian elimination

Use row operations to reach upper-triangular (row echelon) form, then back-substitute. Gauss–Jordan continues to reduced row echelon form (identity on the left).

Rank and solution types

A zero row with non-zero right side (0 = c) means inconsistent. Fewer pivots than unknowns means free variables and infinitely many solutions — common in underconstrained mechanisms.

Homogeneous systems

Ax = 0 always has x = 0; non-trivial solutions exist iff det A = 0 — the basis of eigenvalue problems and buckling/vibration analyses.

Numerical practice

Partial pivoting avoids division by small numbers. Condition number κ measures sensitivity: κ ≈ 10ᵏ loses about k digits. Large sparse systems use iterative solvers (M9).

Formulas

Worked example: 3 × 3 elimination

Given: 2x + y − z = 8; −3x − y + 2z = −11; −2x + y + 2z = −3

Find: x, y, z.

Answer: x = 2, y = 3, z = −1

Worked example: Mesh currents

Given: 15i₁ − 5i₂ = 10, −5i₁ + 15i₂ = 0

Find: Currents.

Answer: i₁ = 0.75 A, i₂ = 0.25 A

Solving systems, eigenvalues and SVD with NumPy

import numpy as np

# Solve a 3x3 system and study eigenvalues / SVD
A = np.array([[2, 1, -1],
              [-3, -1, 2],
              [-2, 1, 2]], dtype=float)
b = np.array([8, -11, -3], dtype=float)

x = np.linalg.solve(A, b)                 # LU factorisation under the hood
print("x =", x)
print("det(A) =", round(np.linalg.det(A), 6))

M = np.array([[4, 1], [2, 3]], dtype=float)
vals, vecs = np.linalg.eig(M)
print("eigenvalues =", vals)
print("eigenvectors (columns) =\n", np.round(vecs, 4))

U, s, Vt = np.linalg.svd(np.array([[3, 0], [4, 5]], dtype=float))
print("singular values =", np.round(s, 4), " cond =", round(s[0] / s[-1], 4))
# x = [ 2.  3. -1.]
# det(A) = -1.0
# eigenvalues = [5. 2.]
# eigenvectors (columns) =
#  [[ 0.7071 -0.4472]
#  [ 0.7071  0.8944]]
# singular values = [6.7082 2.2361]  cond = 3.0

np.linalg.solve uses LU factorisation with partial pivoting — faster and more accurate than computing A⁻¹. The same library gives eigenvalues and the SVD used in later lessons.

Common mistakes

Applications

Key takeaways

Going further