Eigenvalues-and-Eigenvectors

• Definition：
  Let A be an n×n matrix.
  A scalar is said to be an eigenvalue of A if there exists a nonzero vector such that
  Such is said to be an eigenvector for A belonging to .
  • The pair (,x) is sometimes called an eigen-pair for A.
  • Eigenvalues and eigenvectors are only defined for square matrices
  • Eigenvectors must be nonzero
• Property
  • A is singular ⇔ =0 is an eigenvalue of A.
  • If A is nonsingular, then is an eigenvalue of , and x is an eigenvector of belonging to ^(−1), i.e. .
  • For , is an eigenvalue of , and x is an eigenvector of belonging to , i.e.
  • If , what can be? Hint: .
  • If , what can be? Hint:

Find eigenvalue and eigenvector

• : eigenspace 特征空间 corresponding to the eigenvalue .
  This has a nontrivial solution x \ne 0 if and only if we pick so that the square matrix (A−I) is singular.That is: First use det⁡(A−I)=0 to find all eigenvalues for A, and then use N(A−I) to find all eigenvectors for A.

• Property:
  Let A be an n×n matrix and be a scalar. The following statements are equivalent:

  • is an eigenvalue of A.
  • has nontrivial solutions.
  • A−I is singular.
  • det⁡(A−I)=0

• is a degree n polynomial in variable , called the characteristic polynomial 特征多项式.

• : characteristic equation for A

If are distinct eigenvalues of an n×n matrix A with corresponding eigenvectors , then the vectors are linearly independent.

An n×n matrix A is said to be diagonalizable if there exists a nonsingular matrix X and a diagonal matrix D such that
We say that X diagonalizes A.

An n×n matrix A is diagonalizable if and only if A has n linearly independent eigenvectors.

• If A is diagonalizable, then the column vectors of the diagonalizing matrix X are eigenvectors of A and the diagonal elements of D are the corresponding eigenvalues of A.
• The diagonalizing matrix X is not unique. Reordering the columns of a given diagonalizing matrix X or multiplying them by nonzero scalars will produce a new diagonalizing matrix.
• If A is n×n and A has n distinct eigenvalues, then A is diagonalizable. If the eigenvalues are not distinct, then A may or may not be diagonalizable, depending on whether A has n linearly independent eigenvectors.
• If A is diagonalizable, then A can be factored as a product
  

Let A be a symmetric n×n matrix:

• Property:
  • If x is an eigenvector of A and y⊥x, then Ay⊥x. ^[1]
  • If are eigenvectors of A with , then . ^[2]

A square matrix A is orthogonally diagonalizable if for some diagonal matrix D and some orthogonal matrix Q.

• n×n matrix A is orthogonally diagonalizable if and only if A has n orthonormal eigenvectors (columns of Q)
• From the previous slide, if a symmetric matrix A has n distinct eigenvalues, then A has n mutually orthogonal eigenvectors.
• Surprisingly, symmetric n×n matrix A ALWAYS has 𝒏 orthonormal eigenvectors, and ALL eigenvalues are real!

If A is a real symmetric matrix, then there is an orthogonal matrix Q that diagonalizes A; that is, , where is diagonal, with real entries.

Properties

• Eigenvalue: nonnegative , real
• Eigenvector: Orthoganal

Suppose a m×n matrix A Then

1. and are symmetric. are orthogonally diagonalizable
2. and are orthogonally diagonalizable

has m orthonormal eigenvectors
has n orthonormal eigenvectors

3. All eigenvalues of and are non-negative.^[3]

Assume that A is an m×n matrix with m ≥ n. is SVD of A
Factorize A into a product , where

• U is an m × m orthogonal matrix
• V is an n ×n orthogonal matrix
• \sum is an m ×n matrix whose off-diagonal entries are all 0’s and whose diagonal elements satisfy: Properties
• The first r columns of V is an orthonormal basis for
• The rest of columns of V is an orthonormal basis for
• The first r columns of U is an orthonormal basis for
• The rest of columns of U is an orthonormal basis for

If A is an m × n matrix, then A has a singular value decomposition.

If A is a real symmetric n×n matrix, then there is a change of variables , where Q is an orthogonal matrix, such that
where D is a real diagonal matrix.

Let A be a real symmetric n×n matrix. Then A is positive definite if and only if all its eigenvalues are positive.
A is

• Positive definite iff all its eigenvalues are positive
• Negative definite iff all its eigenvalues are negative
• Positive semidefinite iff all its eigenvalues are nonnegative
• Negative semidefinite iff all its eigenvalues are nonpositive
• Indefinite iff all it has both positive and negative eigenvalues

Let A be a real symmetric n×n matrix. The following are equivalent:

• A is positive definite.
• The leading principal submatrices all have positive determinant.

Leading principal submatrices:upper-left 1×1, 2×2, 3×3,…
If some determinants are positive, and others negative: A is indefinite.
If these determinants are all ≥0: positive semidefinite.