论文标题
通过非线性连续加权有限自动机进行顺序密度估计
Sequential Density Estimation via Nonlinear Continuous Weighted Finite Automata
论文作者
论文摘要
加权有限自动机（WFA）已广泛应用于许多领域。 WFA的经典问题之一是离散符号序列的概率分布估计。尽管WFA已扩展到处理连续输入数据（即连续WFA（CWFA）），但由于使用基于WFA的模型的连续随机变量的序列，由于模型的表现以及近似密度通过CWFAS的障碍性限制了连续随机变量的序列，仍然尚不清楚如何将密度函数近似于连续的随机变量序列。在本文中，我们提出了对CWFA模型的非线性扩展，以提高其表现力，我们将其称为非线性连续WFA（NCWFA）。然后，我们利用所谓的RNADE方法，该方法是基于神经网络的众所周知的密度估计器，并提出了RNADE-NCWFA模型。 RNADE-NCWFA模型通过设计计算密度函数。我们表明，该模型比CWFA无法近似的高斯HMM模型更具表现力。从经验上讲，我们使用高斯HMM生成的数据进行了合成实验。我们专注于评估模型估计长度序列的密度（长度长于训练数据）的能力。我们观察到我们的模型在比较基线方法中表现最好。
Weighted finite automata (WFAs) have been widely applied in many fields. One of the classic problems for WFAs is probability distribution estimation over sequences of discrete symbols. Although WFAs have been extended to deal with continuous input data, namely continuous WFAs (CWFAs), it is still unclear how to approximate density functions over sequences of continuous random variables using WFA-based models, due to the limitation on the expressiveness of the model as well as the tractability of approximating density functions via CWFAs. In this paper, we propose a nonlinear extension to the CWFA model to first improve its expressiveness, we refer to it as the nonlinear continuous WFAs (NCWFAs). Then we leverage the so-called RNADE method, which is a well-known density estimator based on neural networks, and propose the RNADE-NCWFA model. The RNADE-NCWFA model computes a density function by design. We show that this model is strictly more expressive than the Gaussian HMM model, which CWFA cannot approximate. Empirically, we conduct a synthetic experiment using Gaussian HMM generated data. We focus on evaluating the model's ability to estimate densities for sequences of varying lengths (longer length than the training data). We observe that our model performs the best among the compared baseline methods.
