Spectral Theorem For Symmetric Matrices

Spectral Theorem

Eigenvalues and eigenvectors of symmetric matrices
The symmetric eigenvalue decomposition theorem
Rayleigh quotients

Eigenvalues and eigenvectors of symmetric matrices

Let be a square, n times n symmetric matrix. A real scalar lambda is said to be an eigenvalue of if there exist a non-zero vector $u in mathbf{R}^n$ such that

A u = lambda u.

The vector is and then referred to as an eigenvector associated with the eigenvalue lambda . The eigenvector is said to be normalized if |u|_2 = 1 . In this case, we have

u^TAu = lambda u^Tu = lambda.

The estimation of is that it defines a management forth behaves but similar scalar multiplication. The amount of scaling is given by lambda . (In High german, the root ''eigen'', means ''cocky'' or ''proper''). The eigenvalues of the matrix are characterized by the feature equation

det(lambda I - A) = 0,

where the note det refers to the determinant of its matrix argument. The function with values t rightarrow p(t) :=det( t I - A) is a polynomial of degree chosen the characteristic polynomial.

From the key theorem of algebra, whatsoever polynomial of degree has (perchance not distinct) complex roots. For symmetric matrices, the eigenvalues are real, since lambda = u^TAu when Au = lambda u , and is normalized.

Spectral theorem

An important event of linear algebra, called the spectral theorem, or symmetric eigenvalue decomposition (SED) theorem, states that for whatsoever symmetric matrix, in that location are exactly (perchance not distinct) eigenvalues, and they are all real; further, that the associated eigenvectors can be called and then as to class an orthonormal basis. The result offers a unproblematic way to decompose the symmetric matrix as a product of simple transformations.

Theorem: Symmetric eigenvalue decomposition

Here is a proof. The SED provides a decomposition of the matrix in simple terms, namely dyads.

Nosotros check that in the SED above, the scalars lambda_i are the eigenvalues, and u_i 'due south are associated eigenvectors, since

$Au_j = sum_{i=1}^n lambda_i u_iu_i^Tu_j = lambda_j u_j, ;; j=1,ldots,n.$

The eigenvalue decomposition of a symmetric matrix can be efficiently computed with standard software, in fourth dimension that grows proportionately to its dimension as n^3 . Here is the matlab syntax, where the first line ensure that matlab knows that the matrix is exactly symmetric.

Matlab syntax

>> A = triu(A)+tril(A',-1); >> [U,D] = eig(A);

Example:

Eigenvalue decomposition of a symmetric matrix.

Rayleigh quotients

Given a symmetric matrix , we can limited the smallest and largest eigenvalues of , denoted $lambda_{rm min}$ and $lambda_{rm max}$ respectively, in the and so-called variational class

$lambda_{rm min}(A) = min_{x} : left{ x^TAx ~:~ x^Tx = 1 right} , ;; lambda_{rm max}(A) = max_{x} : left{ x^TAx ~:~ x^Tx = 1 right} .$

For a proof, see hither.

The term ''variational'' refers to the fact that the eigenvalues are given as optimal values of optimization problems, which were referred to in the past as variational problems. Variational representations be for all the eigenvalues, but are more complicated to state.

The interpretation of the above identities is that the largest and smallest eigenvalues is a measure of the range of the quadratic function x rightarrow x^TAx over the unit Euclidean ball. The quantities above can be written as the minimum and maximum of the then-chosen Rayleigh caliber x^TAx/x^Tx .

Historically, David Hilbert coined the term ''spectrum'' for the set of eigenvalues of a symmetric operator (roughly, a matrix of infinite dimensions). The fact that for symmetric matrices, every eigenvalue lies in the interval $[lambda_{rm min},lambda_{rm max}]$ somewhat justifies the terminology.