CMU 11-785 L23 Variational Autoencoders_013001620011-785-CSDN博客

本文深入探讨了PCA中的EM算法及其局限性，然后介绍了线性和非线性高斯模型，强调了在不完全信息情况下的处理方法。进一步地，文章详细阐述了变分自编码器（VAE）作为非线性高斯模型的扩展，以及其在捕获数据潜在结构中的作用，尽管VAE不能直接计算数据的概率。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

在这里插入图片描述

$x = A z + E$

$\mid z)=N(A z, D)$

Given complete information $(x1,z1),(x2,z2)\left(x_{1}, z_{1}\right),\left(x_{2}, z_{2}\right)$

$\underset{A, D}{\operatorname{argmax}} \sum_{(x, z)} \log P(x, z)=\underset{A, D}{\operatorname{argmax}} \sum_{(x, z)} \log P(x \mid z)$

$=\underset{A, D}{\operatorname{argmax}} \sum_{(x, Z)} \log \frac{1}{\sqrt{(2 \pi)^{d}|D|}} \exp \left(-0.5(x-A z)^{T} D^{-1}(x-A z)\right)$

Initialize the plane
Complete the data by computing the appropriate $z$ for the plane
- $P (z ∣ X; A)$ is a delta, because $E$ is orthogonal to $A$
Reestimate the plane using the $z$
Iterate

PCA assumes the noise is always orthogonal to the data
- Not always true
The noise added to the output of the encoder can lie in any direction (uncorrelated)
We want a generative model: to generate any point
- Take a Gaussian step on the hyperplane
- Add full-rank Gaussian uncorrelated noise that is independent of the position on the hyperplane
  - Uncorrelated: diagonal covariance matrix
  - Direction of noise is unconstrained

$x$