Projection Matrix

A projection matrix is a matrix used in linear algebra to map vectors onto a subspace, typically in the context of vector spaces or 3D computer graphics. It has the following main applications:

A matrix P is a projection matrix if:

1. P² = P (idempotent property).
2. P is square (n × n).

This means applying the projection matrix twice is the same as applying it once. Essentially, the projection does not change after the first application.

Examples of Projection Matrices

Some common examples of projection matrices are:

Projection onto a Subspace

Given a vector v ∈ Rⁿ and a subspace U ⊂ Rⁿ, the projection of v onto U can be computed using the projection matrix P. If A is a matrix whose columns form an orthonormal basis for U, the projection matrix P is:

P = AA^T

• For example, if A = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}, then P = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}, which projects every vector onto itself (identity matrix).

Orthogonal Projection in Linear Regression

In Ordinary Least Squares (OLS) regression, the projection matrix P projects the vector of observed values y onto the column space of the design matrix X:

P = X(X^TX)⁻¹X^T

Here:

• X is the n × p matrix of predictors (independent variables).
• y is the n × 1 vector of responses (dependent variable).

The projected vector \hat{y} = P y is the vector of predicted values.

Projection in Principal Component Analysis (PCA)

In PCA, high-dimensional data is projected onto a subspace spanned by the top k eigenvectors (principal components) of the covariance matrix:

P = U_kU_k^T

Where:

• U_k​ is a matrix containing the top k eigenvectors as its columns.

This projection reduces the dimensionality of the data while retaining maximum variance.

Types of Projection Matrices

Some of the common types of projection matrices are:

• Orthogonal Projection Matrix
• Perspective Projection Matrix
• Oblique Projection Matrix

Let's discuss them in detail.

Orthogonal Projection Matrix

Projects a vector onto a subspace along directions orthogonal to the subspace.

If A is a matrix whose columns form a basis for a subspace, the orthogonal projection matrix onto the column space of A is:

P = A(A^TA)⁻¹A^T

Perspective Projection Matrix

Perspective projection matrix is used in computer graphics to project 3D points onto a 2D plane while preserving the perspective view.

A common example for a perspective projection matrix in 3D is:

P = \begin{bmatrix} f & 0 & 0 & 0 \\ 0 & f & 0 & 0 \\ 0 & 0 & (z_{\text{far}} + z_{\text{near}}) / (z_{\text{near}} - z_{\text{far}}) & 2 \cdot z_{\text{far}} \cdot z_{\text{near}} / (z_{\text{near}} - z_{\text{far}}) \\ 0 & 0 & -1 & 0 \end{bmatrix}

where f is the focal length.

Oblique Projection Matrix

Oblique projection matrix projects vectors onto a subspace along a direction that is not orthogonal to the subspace.

Properties of Projection Matrices

Some of the common properties of projection matrices are:

• Idempotence: P² = P
• Symmetry (for orthogonal projections): P = P^T
• Eigenvalues: The eigenvalues are 0 and 1, where 1 corresponds to the subspace being projected onto.
• Trace: The trace of a projection matrix equals the dimension of the subspace onto which it projects.
• Rank: The rank of P equals the dimension of the subspace it projects onto.

Applications of Projection Matrices

Projection matrices have various application in many fields, some of the common application are:

• Linear regression: Projecting data onto the column space of predictor variables.
• Computer graphics: Transforming 3D scenes onto a 2D plane.
• Signal processing: Isolating components of signals in subspaces.
• Principal Component Analysis (PCA): Projecting data onto the principal components.

Read More,

• Vector Algebra
• Vector and Scaler
• Scalar and Vector Projection Formula
• Matrices
• Types of Matrices