Principal Component Analysis

• A method to obtain axes along which data exhibits the most variance.
• Helps in dimensionality reduction by reducing the data to lesser number of dimensions along these axes such that maximal amount of information from the data is retained.
• Helps in visualisation as high dimensional data can be optimally reduced to 2 or 3 dimensions to be effectively visualised.
• Method of feature extraction.

Principal components are a sequence of projections of the data, mutually uncorrelated (ie orthogonal )and ordered in variance. [1]

Equivalent definition:

Goal is to identify the most meaningful basis to reexpress a data set. ie. Is there another basis that is a linear combination of the original basis that best reexpresses the data set. Reexpressing the data as a linear combination of it's basis vectors. [2]

Input: Data with p features
Output: Maps data to p new features, such that covariance between features is 0.

• Standardise the data. Make mean 0 and std 1 for each feature.
• Find the eigen vectors(should have unit norm) or perform SVD of the covariance matrix.

[1]: Elements of Statistical Learning
[2]: A Tutorial on PCA