Eigenvalue Analysis

Summary:

• Motivation: PC are new variables, uncorrelated built from the old ones.
• Coefficients are obtained through eigenvalues of variance-covariance or correlation matrix.
• Eigenvalues represent variance explained.
• Also useful discriminant analysis, canonical variate analysis.
• Generalized eigenvalue problem $\det(A-\lambda B) \: AB^{-1} \: B^{-\frac{1}{2}}A \: B^{-\frac{1}{2}}$.
• Statistics $\rightarrow$ symmetrical eigenvalue problems.
• $A$ and $Z^\prime AZ$ same eigenvalues $\rightarrow$ $Z$ tranforms $A$'s structure.
• $A=LR\qquad\qquad RL-L^\prime AL$ similarity.
• Power method: $\lambda_1$ simple $v_1,x_0$ non $\bot \: v_2\qquad X_i=A\frac{x_{i-1}}{\parallel x_{i-1}\parallel}$.
• $x_n = \alpha_n A^n_? x_0 = w_i\lambda^n_i \left( v_i + \left(\frac{\lambda_2}{\... ...w_1} \: v_2 + \cdots \right),\quad\parallel x_n \parallel \rightarrow \lambda_1$.

• Advantageous for $A$, easy to compute, only provides $v_1,\lambda_1$.
• $X_0$ orthogonal columns, re-orthogonalizing at each step.
• $T=A\,X_{i-1}\qquad T=X_iR_i\qquad X_i\rightarrow [v_1\cdots v_k], R_i\rightarrow [\lambda_1\cdots \lambda_k]$.
• Rate $\lambda_{k+1}/\lambda_k$.
• $QR$-algorithm         $A_{i+1} = Q^\prime_i A_i Q_i$ where $A_i=Q_iR_i$.
• Improvement 1: start with $A_0=U^\prime AU$ tridiagonal (Householder).
• Tridiagonal form is not lost at each step, subdiagonal elmination through Givens to give the $QR$ decomposition.
• Improvement 2: convergence depends upon $\lambda_{p+1}/\lambda_p$. $A-\mu I$ and $A$ same eigenvectors and eigenvalues differ by $\mu$.
• Choose $\mu$ so that $(\lambda_{k+1}/\lambda_k)$ small ($\mu$ close to $\lambda_{k+1}$).
• $\lambda_{k+1}$ not known, $Q_{pp}$ good or eigenvalue of lower block of $A$.
• $A_{i+1} = R_iQ_i + \mu I$.
• Cubic convergence.
• Implicit lemma: A sym. non-singular $B=Q^\prime AQ,\: Q\bot$, $B$ positive off-diagonal elements then $Q$ and $B$ are determined as soon as the first col. of $Q$ is.
• $A_{i+1} = R_iQ_i + \mu_i I = Q^\prime (A_2-\mu_i I) Q_i+\mu_i I = Q^\prime_iA_iQ_i$.
• $R_k = Q^\prime_k(A_k-mu_k I)$.
• $Q_k = J_1\cdots J_p,J_k=G_{k+1} k$.
• First row of $Q^\prime_k$ is that of $J^\prime_1$ because only one to touch first row.