论文标题
数学上对新兴语言的词典熵进行建模
Mathematically Modeling the Lexicon Entropy of Emergent Language
论文作者
论文摘要
我们制定了一个随机过程，即Filex，作为在基于深度学习的新兴语言系统中的词典熵的数学模型。在数学上定义模型允许它生成可以直接和果断测试的清晰预测。我们凭经验验证了FileX预测超参数（训练步骤，词典大小，学习率，推出缓冲区大小和gumbel-softmax温度）与新兴语言在20个环境 - 热播放参数组合中的20分之间的熵之间的正确相关性。此外，我们的实验表明，不同的环境显示了其超参数和熵之间的不同关系，这表明了对模型的需求，该模型可以在精确的粒度水平上做出明确定义的预测。
We formulate a stochastic process, FiLex, as a mathematical model of lexicon entropy in deep learning-based emergent language systems. Defining a model mathematically allows it to generate clear predictions which can be directly and decisively tested. We empirically verify across four different environments that FiLex predicts the correct correlation between hyperparameters (training steps, lexicon size, learning rate, rollout buffer size, and Gumbel-Softmax temperature) and the emergent language's entropy in 20 out of 20 environment-hyperparameter combinations. Furthermore, our experiments reveal that different environments show diverse relationships between their hyperparameters and entropy which demonstrates the need for a model which can make well-defined predictions at a precise level of granularity.
