The spectral theorem and the singular value decomposition

Given a linear operator LL on a finite-dimensional vector space VV, we often want to find a basis of VV that gives the simplest possible matrix representation for LL. The ideal case is when there is a basis for VV consisting of eigenvectors of LL, in which LL can be represented by a diagonal matrix and is called “diagonalizable”. Suppose v1,,vnv_1,\ldots,v_n is a basis for VV where each viv_i is an eigenvector of LL with corresponding eigenvalue λi\lambda_i (not necessarily distinct), then

L[v1vn]=[Lv1Lvn]=[v1vn]diag(λ1,,λn).L\begin{bmatrix} v_1 & \cdots & v_n \end{bmatrix} = \begin{bmatrix} Lv_1 & \cdots & Lv_n \end{bmatrix} = \begin{bmatrix} v_1 & \cdots & v_n \end{bmatrix} \diag(\lambda_1,\ldots,\lambda_n).

In other words, we have L=VΛV1L=V\Lambda V^{-1}, where V=[v1vn]V=\begin{bmatrix}v_1 & \cdots & v_n\end{bmatrix} and Λ=diag(λ1,,λn).\Lambda=\diag(\lambda_1,\ldots,\lambda_n). This is called the eigendecomposition of LL.

A linear operator LL is diagonalizable if and only if both the following conditions hold:

  1. The characteristic polynomial p(x)p(x) of LL splits (i.e., factors into product of linear polynomials).
  2. The algebraic multiplicity and geometric multiplicity of each eigenvalue λ\lambda of LL are the same.1

The geometric multiplicty is always at most the algebraic multiplicty; equality occurs for all eigenvalues if and only if all geometric multiplicities sum to the degree of p(x)p(x). More succintly, LL is diagonalizable if and only if VV is the direct sum of the eigenspaces of LL, in which case

L=λ1IEλ1++λkIEλk,L = \lambda_1 I_{E_{\lambda_1}} + \cdots + \lambda_k I_{E_{\lambda_k}},

where IEλiI_{E_{\lambda_i}} is the identity on EλiE_{\lambda_i} (and zero outside of it).


  1. The algebraic multiplicity of λ\lambda is the number of times (xλ)(x-\lambda) appears in the factorization of p(x)p(x), while its geometric multiplicity is the dimension of its eigenspace Eλ={vV ⁣:Lv=λv}E_\lambda = \{v\in V\colon Lv = \lambda v\}, the space of all eigenvectors of λ\lambda.