论文标题
基于近似值的差异降低了重聚梯度的差异
Approximation Based Variance Reduction for Reparameterization Gradients
论文作者
论文摘要
灵活的变分分布可改善变异推理，但很难优化。在这项工作中，我们提出了一个控制变体，适用于具有已知均值和协方差矩阵的任何可重配分布，例如具有协方差结构的高斯人。控制变量基于模型的二次近似，其参数是通过将梯度估计器的方差最小化来使用双重周期方案设置的。我们从经验上表明，这种控制变化会导致梯度方差和优化收敛的大幅改善，以推理非物质变异分布。
Flexible variational distributions improve variational inference but are harder to optimize. In this work we present a control variate that is applicable for any reparameterizable distribution with known mean and covariance matrix, e.g. Gaussians with any covariance structure. The control variate is based on a quadratic approximation of the model, and its parameters are set using a double-descent scheme by minimizing the gradient estimator's variance. We empirically show that this control variate leads to large improvements in gradient variance and optimization convergence for inference with non-factorized variational distributions.
