The inner product, also called the dot product, is one of the most fundamental operations in linear algebra. It measures the similarity between two vectors and plays a crucial role in geometry, physics, machine learning, and numerical computations. Unlike the outer product, which produces a matrix, the inner product results in a scalar value.
Given two vectors:
\mathbf{u} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix}, \quad\mathbf{v} = \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix}
the inner product (also called the dot product) is defined as:
\mathbf{u} \cdot \mathbf{v} = \sum_{i=1}^{n} u_i v_i
which results in a scalar value.
Geometric Interpretation
The inner product of two vectors can also be expressed in terms of the angle θ between them:
\mathbf{u} \cdot \mathbf{v} = \|\mathbf{u}\| \|\mathbf{v}\| \cos\theta
where ∥u∥ and ∥v∥ represent the magnitudes (norms) of the vectors, given by:
\|\mathbf{u}\| = \sqrt{\sum_{i=1}^{n} u_i^2}, \quad \|\mathbf{v}\| = \sqrt{\sum_{i=1}^{n} v_i^2}
If the inner product is zero (u⋅v=0), the vectors are orthogonal, meaning they are perpendicular to each other.
Numerical Example
Consider two vectors:
\mathbf{u} = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}, \quad\mathbf{v} = \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix}
The inner product is calculated as:
\mathbf{u} \cdot \mathbf{v} = (1 \times 4) + (2 \times 5) + (3 \times 6) = 4 + 10 + 18 = 32
Thus, the result is 32.
Python Implementation
Using NumPy’s dot() Function
import numpy as np
# Define vectors
u = np.array([1, 2, 3])
v = np.array([4, 5, 6])
# Compute inner product
inner_prod = np.dot(u, v)
# Display result
print("Inner Product:", inner_prod)
Output:
Inner Product: 32
Using Explicit Summation
# Compute inner product manually
inner_prod = sum(u[i] * v[i] for i in range(len(u)))
print("Inner Product (manual computation):", inner_prod)
Output:
Inner Product: 32
This produces the same result as np.dot().
Using Matrix Multiplication (@ Operator)
inner_prod = u @ v # Equivalent to np.dot(u, v)
print("Inner Product (matrix multiplication):", inner_prod)
Output:
Inner Product (matrix multiplication): 32
Applications of the Inner Product
1. Machine Learning and Neural Networks
The inner product is used in the computation of cosine similarity, a key measure in recommendation systems and clustering. It also plays a major role in gradient descent for optimizing model parameters.
2. Physics and Mechanics
The work done by a force F moving an object by displacement d is given by:
W = \mathbf{F} \cdot \mathbf{d}
If the vectors are perpendicular, no work is done (W=0).
3. Computer Graphics and 3D Rendering
In lighting models, the inner product determines how much light hits a surface. The dot product between the light direction vector and the surface normal vector helps in shading calculations.