Question 5 of 5: Effectiveness of Principal Component Analysis

Question 5: Effectiveness of Principal Component Analysis (20 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Given. The $4\times4$ variance-covariance matrix $\Sigma$ of a four-band image (diagonal = band variances; off-diagonals = inter-band covariances, all large and positive).

Find. Whether PCA will be effective — i.e. whether a small number of principal components captures nearly all of the image variance — with a quantitative justification.

Approach. PCA rotates the band axes onto the eigenvectors of $\Sigma$; the eigenvalues are the variances of the resulting principal components. PCA is "effective" when the bands are highly correlated, so that the first eigenvalue dominates and the leading component alone explains most of the total variance (the trace of $\Sigma$).

1. Total variance = trace. The sum of the band variances is invariant under the PCA rotation: $$\operatorname{tr}\Sigma = 34.89+105.95+104.02+21.35 = 266.21.$$
2. Inspect the correlations. Normalizing the covariances, e.g. $r_{23}=99.58/\sqrt{105.95\times104.02}=0.949$ and $r_{12}=55.62/\sqrt{34.89\times105.95}=0.915$; the six inter-band correlations range from $0.83$ to $0.97$, so all four bands are very strongly correlated. Strong correlation is precisely the redundancy PCA exploits.
3. Eigenvalues of $\Sigma$. Solving $\det(\Sigma-\lambda I)=0$ (done numerically) gives the principal-component variances $$\lambda_1=\boxed{253.44},\quad \lambda_2=7.91,\quad \lambda_3=3.96,\quad \lambda_4=0.90,$$ which sum to $266.21$, confirming they reproduce the trace.
4. Proportion of variance explained. Dividing each eigenvalue by the trace: $$\text{PC1}=\frac{253.44}{266.21}=95.2\%,\quad \text{PC1+PC2}=98.2\%.$$ The first principal component alone captures $\approx95\%$ of the total scene variance, and the first two together capture over $98\%$.

Conclusion — yes, PCA is highly effective here. Because the four bands are strongly correlated, the transform concentrates almost all information into a single component: PC1 carries 95.2% of the variance, so the four correlated bands can be compressed to essentially one (at most two) principal components with negligible information loss. PCA would be an excellent dimensionality-reduction / de-correlation step for this image.