Torch 自动求导

Torch 自动求导

一、链式求导法则

1.1 原理

标量链式法则：

$$y=f(u),u=g(x),\frac{\partial y}{\partial x}=\frac{\partial y}{\partial u}\frac{\partial u}{\partial x}$$

扩展到向量：

∂ y ∂ x ⃗ 1 × n = ∂ y ∂ u ∂ u ∂ x ⃗ m × n ∂ y ∂ x ⃗ 1 × n = ∂ y ∂ u ⃗ 1 × m ∂ u ⃗ ∂ x ⃗ m × n ∂ y ⃗ ∂ x ⃗ m × n = ∂ y ⃗ ∂ u ⃗ m × k ∂ u ⃗ ∂ x ⃗ k × n \begin{array}{l} \frac{\partial y}{\partial \vec{x}}_{1\times n} =\frac{\partial y}{\partial u}\frac{\partial u}{\partial \vec{x}} _{m\times n} \\ \frac{\partial y}{\partial \vec{x}}_{1\times n} =\frac{\partial y}{\partial \vec{u}} _{1\times m}\frac{\partial \vec{u}}{\partial \vec{x}} _{m\times n} \\ \frac{\partial \vec{y}}{\partial \vec{x}}_{m\times n} =\frac{\partial \vec{y}}{\partial \vec{u}} _{m\times k}\frac{\partial \vec{u}}{\partial \vec{x}} _{k\times n} \\ \end{array} ∂x ∂y​1×n​=∂u∂y​∂x ∂u​m×n​∂x ∂y​1×n​=∂u ∂y​1×m​∂x ∂u ​m×n​