Inner Product Spaces, Special Matrices, the Jordan Form and Quadratic Forms

The second half of Section 2 is geometry and structure. An inner product turns a vector space into a space with lengths and angles, and the Gram–Schmidt process manufactures an orthonormal basis from any basis. The special classes of matrices — symmetric, skew-symmetric, Hermitian, skew-Hermitian, normal, orthogonal and unitary — are each defined by one identity and each pin their eigenvalues to a line or a circle. The deepest result is the spectral theorem: a complex matrix can be diagonalised by a unitary matrix exactly when it is normal. For matrices that cannot be diagonalised at all, the Jordan canonical form is the next best thing, and its block structure is read off from the minimal polynomial and the dimensions of kernels. The section ends with bilinear and quadratic forms, whose classification by Sylvester’s law of inertia is a question about signs of eigenvalues.

1. Inner products, orthogonality and Gram–Schmidt

An inner product on a real or complex space is linear in one slot, conjugate-symmetric (⟨u, v⟩ = conj ⟨v, u⟩) and positive definite. It gives a norm ‖v‖ = √⟨v, v⟩, the Cauchy–Schwarz inequality |⟨u, v⟩| ≤ ‖u‖‖v‖ (equality iff u, v are dependent) and the triangle inequality. For a subspace W of an n-dimensional inner product space, V = W ⊕ W⊥ and dim W⊥ = n − dim W; the orthogonal projection onto W sends v to Σ⟨v, eᵢ⟩eᵢ for an orthonormal basis (eᵢ) of W.

Gram–Schmidt. From independent u₁, u₂, … set v₁ = u₁ and vₖ = uₖ − Σj<k (⟨uₖ, vⱼ⟩/⟨vⱼ, vⱼ⟩) vⱼ, then normalise. Each vₖ is orthogonal to all earlier ones and span(v₁, …, vₖ) = span(u₁, …, uₖ). Example: u₁ = (1, 1, 0), u₂ = (1, 0, 1) give v₂ = (1, 0, 1) − ½(1, 1, 0) = (½, −½, 1), with ‖v₂‖² = 3/2. In matrix form this is the QR factorisation A = QR, Q with orthonormal columns and R upper triangular.

2. Symmetric, Hermitian, normal, orthogonal and unitary matrices

Each class, its defining identity and where its eigenvalues lie
ClassDefinitionEigenvalues
real symmetric / HermitianAᵀ = A / A∗ = Areal
real skew-symmetric / skew-HermitianAᵀ = −A / A∗ = −Apurely imaginary or 0
orthogonal / unitaryQᵀQ = I / U∗U = Imodulus 1; det Q = ±1, |det U| = 1
normalAA∗ = A∗Aanywhere; eigenvectors of distinct eigenvalues are orthogonal
  • A real skew-symmetric matrix of odd order is singular: det A = det Aᵀ = det(−A) = (−1)ⁿ det A forces det A = 0.
  • Every square complex matrix is H + K uniquely, with H = (A + A∗)/2 Hermitian and K = (A − A∗)/2 skew-Hermitian; A is normal iff H and K commute.
  • Orthogonal and unitary matrices preserve inner products and lengths, ‖Ux‖ = ‖x‖; the eigenvalues of the rotation [[cos θ, −sin θ], [sin θ, cos θ]] are e±iθ, not ±1.

3. Diagonalisation by a unitary matrix

Schur’s theorem: every complex square matrix is unitarily similar to an upper triangular matrix, U∗AU = T. Spectral theorem: A is unitarily diagonalisable (U∗AU diagonal for some unitary U) if and only if A is normal. For real matrices: A is orthogonally diagonalisable (QᵀAQ diagonal with Q real orthogonal) if and only if A is symmetric.

🎯 Why a normal triangular matrix is diagonal
If T is upper triangular and TT∗ = T∗T, compare the (1, 1) entries: |t₁₁|² + |t₁₂|² + … + |t₁ₙ|² = |t₁₁|², so the first row is zero off the diagonal; induct down the rows. Schur plus this gives the spectral theorem in one line, and it also shows that a normal nilpotent matrix is 0 and a normal matrix with real eigenvalues is Hermitian.
⚠️ Diagonalisable is weaker than unitarily diagonalisable
[[1, 1], [0, 2]] has distinct eigenvalues 1 and 2, so it is diagonalisable, but its eigenvectors (1, 0) and (1, 1) are not orthogonal and AA∗ ≠ A∗A. So it is diagonalisable and not unitarily diagonalisable. [[0, 1], [−1, 0]] is real, not symmetric, and still unitarily diagonalisable over ℂ, because it is normal (it is orthogonal).

4. The Jordan canonical form

Over ℂ every square matrix is similar to a block-diagonal matrix of Jordan blocks Jk(λ) — λ on the diagonal, 1 on the superdiagonal — unique up to the order of the blocks. For each eigenvalue λ: the number of blocks is the geometric multiplicity dim ker(A − λI); the sum of their sizes is the algebraic multiplicity; the largest size is the power of (x − λ) in the minimal polynomial. The number of blocks of size ≥ k is dim ker(A − λI)k − dim ker(A − λI)k−1.

Counting Jordan forms: each eigenvalue is a partition
Characteristic / minimal polynomialBlock sizes allowedNumber of forms
(x − 2)⁴ / (x − 2)²{2, 2}, {2, 1, 1}2
(x − 2)⁶ / (x − 2)³{3, 3}, {3, 2, 1}, {3, 1, 1, 1}3
(x − 1)³(x + 1)² / (x − 1)²(x + 1)λ = 1: {2, 1}; λ = −1: {1, 1}1
⚠️ Minimal and characteristic polynomials do not always fix the form
For n ≤ 3 they do. At n = 4 they stop: J₂(0) ⊕ J₂(0) and J₂(0) ⊕ J₁(0) ⊕ J₁(0) both have characteristic polynomial x⁴ and minimal polynomial x², yet their kernels have dimensions 2 and 3, so they are not similar. The geometric multiplicity is the extra invariant that separates them.

5. Bilinear and quadratic forms

A bilinear form on ℝⁿ is B(x, y) = xᵀAy; it is symmetric iff A is. A quadratic form is Q(x) = xᵀAx with A symmetric (replace any A by (A + Aᵀ)/2): Q = x² + 4xy + y² has A = [[1, 2], [2, 1]]. By the spectral theorem Q = Σλᵢyᵢ² in orthonormal eigen-coordinates. Sylvester’s law of inertia: in every diagonalisation Q = Σ dᵢzᵢ² by an invertible change of variables, the numbers p of positive and q of negative coefficients are the same; rank = p + q, and the signature is p − q.

  • Q is positive definite ⇔ all eigenvalues of A are > 0 ⇔ all leading principal minors are > 0 (Sylvester’s criterion). Positive semidefinite needs all principal minors ≥ 0, not only the leading ones: [[0, 0], [0, −1]] has leading minors 0, 0 and is not semidefinite.
  • On the unit sphere, min Q = λmin and max Q = λmax (Rayleigh quotient). For Q = 5x² + 4xy + 2y², A = [[5, 2], [2, 2]] has eigenvalues 6 and 1, so on x² + y² = 1 the form ranges over [1, 6].
  • The conic Q(x, y) = 1 is an ellipse if both eigenvalues are positive, a hyperbola if they have opposite signs; x² + 4xy + y² = 1 (eigenvalues 3, −1) is a hyperbola.

Key takeaways

  • Gram–Schmidt subtracts projections onto earlier vectors; dim W + dim W⊥ = n.
  • Hermitian ⇒ real eigenvalues, skew-Hermitian ⇒ imaginary, unitary ⇒ modulus 1; an odd-order real skew-symmetric matrix is singular.
  • Unitarily diagonalisable ⇔ normal; orthogonally diagonalisable over ℝ ⇔ symmetric; diagonalisable alone is weaker.
  • Jordan blocks for λ: number = geometric multiplicity, total size = algebraic multiplicity, largest = exponent in the minimal polynomial.
  • A quadratic form is classified by the signs of the eigenvalues of its symmetric matrix; rank and signature are invariant (Sylvester).

Practice questions (13)

Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.

  1. Which statement about eigenvalues is correct?

    1. a Hermitian matrix has real eigenvalues and a unitary matrix has eigenvalues of modulus 1
    2. a unitary matrix has real eigenvalues and a Hermitian matrix has eigenvalues of modulus 1
    3. a skew-Hermitian matrix has real eigenvalues
    4. every eigenvalue of a real orthogonal matrix is 1 or −1
    Show answer

    Answer: A — a Hermitian matrix has real eigenvalues and a unitary matrix has eigenvalues of modulus 1

    If Hv = λv with H∗ = H, then λ‖v‖² = ⟨Hv, v⟩ = ⟨v, Hv⟩ = conj(λ)‖v‖², so λ is real; if U∗U = I, ‖v‖ = ‖Uv‖ = |λ|‖v‖, so |λ| = 1. Skew-Hermitian eigenvalues are purely imaginary, and a rotation by 90° is real orthogonal with eigenvalues ±i.
  2. Let Q be a real n × n orthogonal matrix. Which statements are always true?

    1. det Q = 1 or det Q = −1
    2. every eigenvalue of Q is 1 or −1
    3. ‖Qx‖ = ‖x‖ for every x in ℝⁿ
    4. Q⁻¹ = Qᵀ
    Show answer

    Answer: A — det Q = 1 or det Q = −1; C — ‖Qx‖ = ‖x‖ for every x in ℝⁿ; D — Q⁻¹ = Qᵀ

    (1) det(QᵀQ) = (det Q)² = 1. (3) ‖Qx‖² = xᵀQᵀQx = xᵀx. (4) is the definition QᵀQ = I for a square matrix. (2) is false: the rotation [[0, −1], [1, 0]] is orthogonal with eigenvalues ±i; the real eigenvalues, if any, are ±1, but complex ones of modulus 1 occur.
  3. Gram–Schmidt applied to u₁ = (1, 1, 0) and u₂ = (1, 0, 1) in ℝ³ (standard inner product), without normalising, gives v₁ = u₁ and v₂. The value of ‖v₂‖² is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 1.5

    v₂ = u₂ − (⟨u₂, v₁⟩/⟨v₁, v₁⟩)v₁ = (1, 0, 1) − (1/2)(1, 1, 0) = (1/2, −1/2, 1). Check v₂·v₁ = 1/2 − 1/2 + 0 = 0. ‖v₂‖² = 1/4 + 1/4 + 1 = 1.5. Forgetting to divide by ⟨v₁, v₁⟩ = 2 gives (0, −1, 1) and 2.
  4. The dimension of the orthogonal complement in ℝ⁴ of span{(1, 1, 1, 1), (1, −1, 0, 0)} is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 2

    The two vectors are independent (indeed orthogonal), so W has dimension 2 and dim W⊥ = 4 − 2 = 2. Concretely W⊥ = {x : x₁ + x₂ + x₃ + x₄ = 0, x₁ = x₂}, spanned by (0, 0, 1, −1) and (1, 1, −1, −1).
  5. Which of these complex matrices are unitarily diagonalisable?

    1. [[0, 1], [−1, 0]]
    2. [[1, 1], [0, 2]]
    3. [[1, i], [i, 1]]
    4. [[1, 1], [0, 1]]
    Show answer

    Answer: A — [[0, 1], [−1, 0]]; C — [[1, i], [i, 1]]

    Unitarily diagonalisable ⇔ normal. (1) is real orthogonal, hence normal. (3): A∗ = [[1, −i], [−i, 1]] and AA∗ = A∗A = 2I, so normal (it is I + iS with S real symmetric). (2) is diagonalisable (eigenvalues 1, 2) but AA∗ = [[2, 2], [2, 4]] ≠ A∗A = [[1, 1], [1, 5]]. (4) is not even diagonalisable.
  6. For every real 3 × 3 skew-symmetric matrix A,

    1. det A = 0
    2. all eigenvalues of A are real
    3. A is invertible
    4. trace A ≠ 0
    Show answer

    Answer: A — det A = 0

    det A = det Aᵀ = det(−A) = (−1)³ det A, so det A = 0. The eigenvalues are 0 and a pair ±iθ, which are not real unless A = 0. The diagonal of a skew-symmetric matrix is zero, so the trace is 0.
  7. A complex normal matrix all of whose eigenvalues are real must be

    1. Hermitian
    2. unitary
    3. skew-Hermitian
    4. nilpotent
    Show answer

    Answer: A — Hermitian

    By the spectral theorem A = UDU∗ with U unitary and D diagonal holding the eigenvalues. If D is real, A∗ = UD∗U∗ = UDU∗ = A. It need not be unitary (2I is normal with real eigenvalues and not unitary), and a normal nilpotent is 0.
  8. Let A be an n × n complex matrix. Which statements are true?

    1. A is unitarily similar to an upper triangular matrix
    2. if A is normal and nilpotent then A = 0
    3. every diagonalisable matrix is normal
    4. eigenvectors of a normal matrix belonging to distinct eigenvalues are orthogonal
    Show answer

    Answer: A — A is unitarily similar to an upper triangular matrix; B — if A is normal and nilpotent then A = 0; D — eigenvectors of a normal matrix belonging to distinct eigenvalues are orthogonal

    (1) is Schur’s theorem. (2) A normal A = UDU∗; nilpotent forces every eigenvalue, hence D, to be 0. (4) For normal A, Av = λv implies A∗v = conj(λ)v, and then λ⟨v, w⟩ = ⟨Av, w⟩ = ⟨v, A∗w⟩ = μ⟨v, w⟩. (3) is false: [[1, 1], [0, 2]] is diagonalisable and not normal.
  9. A 5 × 5 complex matrix A has characteristic polynomial (x − 3)⁵, minimal polynomial (x − 3)², and dim ker(A − 3I) = 3. The number of Jordan blocks of size 2 in its Jordan form is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 2

    There are 3 blocks (the geometric multiplicity), the largest has size 2 (the minimal polynomial), and the sizes sum to 5. With sizes in {1, 2}: a + b = 3 blocks and a + 2b = 5 give b = 2 blocks of size 2 and a = 1 of size 1, i.e. {2, 2, 1}.
  10. Up to the order of the blocks, the number of distinct Jordan canonical forms of a 6 × 6 complex matrix with characteristic polynomial (x − 2)⁶ and minimal polynomial (x − 2)³ is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 3

    Each form is a partition of 6 whose largest part is exactly 3: {3, 3}, {3, 2, 1}, {3, 1, 1, 1}. That is 3. Counting partitions with largest part at most 3 (7 of them) forgets that the minimal polynomial forces a block of size exactly 3.
  11. For the real quadratic form Q(x, y) = x² + 4xy + y², which statements are true?

    1. Q is indefinite
    2. Q has rank 2 and signature p − q equal to 0
    3. Q(1, −1) < 0
    4. Q is positive semidefinite
    Show answer

    Answer: A — Q is indefinite; B — Q has rank 2 and signature p − q equal to 0; C — Q(1, −1) < 0

    The matrix [[1, 2], [2, 1]] has eigenvalues 3 and −1: one positive, one negative, so p = q = 1, rank 2, signature 0, indefinite; (4) fails. Directly Q(1, −1) = 1 − 4 + 1 = −2 < 0 while Q(1, 1) = 6 > 0.
  12. The rank of the real quadratic form Q(x, y, z) = (x + y + z)² + (x − y)² is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 2

    Q = ℓ₁² + ℓ₂² with ℓ₁ = x + y + z and ℓ₂ = x − y linearly independent linear forms, so its matrix is a sum of two rank-one matrices with independent vectors: rank 2, positive semidefinite, with Q = 0 on the line x = y, z = −2x.
  13. The minimum value of 5x² + 4xy + 2y² subject to x² + y² = 1 is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 1

    The symmetric matrix is [[5, 2], [2, 2]], with trace 7 and determinant 6, so eigenvalues 6 and 1. On the unit circle a quadratic form ranges between its smallest and largest eigenvalues, so the minimum is 1, attained at the eigenvector (1, −2)/√5: 5 − 8 + 8 = 5 over 5 = 1.