eigen decomposition

[gusl]

Here's something I should have learned in Linear Algebra class, if it is true:

For every linear transformation T that is not projecting into fewer dimensions, there exists an *orthogonal* basis B, such that T takes each basis vector to a multiple of itself, i.e. forall b_i in B, T(b_i) = lambda_i (b_i). In this basis, T is represented by a diagonal matrix, namely the one containing lambda_i. (I can't prove this)

If this is right, there should be something called the "orthogonal eigenbasis algorithm", but I can find no such thing... (doesn't PCA do this?)

(Also, a linear combination of two eigenvectors is an eigenvector iff the two eigenvalues are the same)

I trust that my geeky readership will correct me if this is wrong.

I should read about characteristic polynomials.

Comments

[smandal.livejournal.com]

Well, it's certainly true (http://mathworld.wolfram.com/EigenDecompositionTheorem.html).

If T is square and fully of rank N, where N is the dimension of the space, then the eigenvalues and eigenvectors are easy to find. Examples here (http://en.wikipedia.org/wiki/Eigenvalue). (PCA is a generalization to the Factor Analysis described in the article.)

[rdore.livejournal.com]

What you're talking about is called an orthoganal diagonalization. Being orthogonally diagonalizable is equivalent to being symmetric (equal to one's transpose).

[jcreed.livejournal.com]

Isn't it enough to demand that the matrix is nonsingular if you let the eigenvalues be complex?

[rdore.livejournal.com]

No.

1. All eigenvalues in the field iff there's a Jordan normal form.
   
2. Basis of eigenvectors iff diagonalizable.
   
3. Symmetric and all eigenvalues in field (missed before) iff orthogonally diagonalizable.
   
4. (For complex matrices) Hermitian iff unitarily diagonalizable.
   

[mauitian.livejournal.com]

The eigenvectors are only orthogonal if T is symmetric (or Hermitian), but even if it's not there's still a spectral decomposition (eigen decomposition).

Here are my notes on it:

http://deferentialgeometry.org/#eigen