论文标题
Signnet：使用公制嵌入式学习的单个通道标志生成
SignNet: Single Channel Sign Generation using Metric Embedded Learning
论文作者
论文摘要
真正的解释代理不仅理解手语和翻译为文本，而且还理解文本并转化为标志。迄今为止，手语翻译中的大部分AI工作主要集中于从字符到文字的翻译。对于后一个目标，我们提出了一个文本到签名的翻译模型Signnet，该模型在翻译时利用了视觉符号的相似性（和相似性）的概念。介绍的这个模块只是涉及文本到符号（T2S）以及符号对文本（S2T）的两个双学习过程的一部分。我们目前将Signnet作为单个通道体系结构实现，以便可以在连续的双学习框架中将T2S任务的输出输入S2T。通过单个通道，我们指的是单个模态，即身体姿势关节。 在这项工作中，我们提出了使用新型度量嵌入学习过程的T2S任务，以保持符号嵌入相对于它们的相似性之间的距离。我们还描述了如何为相似性测试选择符号的正面和负面示例。从我们的分析中，我们可以观察到使用BLEU分数评估时，基于学习的模型的性能比传统损失的其他模型的表现要好得多。在姿势掩饰的任务中，Signet以及其最先进的（SOTA）的执行效果以及在文本的任务中超过了它们的表现，在BLEU 1-BLEU 4分中表现出了值得注意的增强（BLEU 1：BLEU 1：39-> 39; 〜26％; 〜26％; 〜26％; 〜26％; RWTH Phoenix-Weather-2014T基准数据集
A true interpreting agent not only understands sign language and translates to text, but also understands text and translates to signs. Much of the AI work in sign language translation to date has focused mainly on translating from signs to text. Towards the latter goal, we propose a text-to-sign translation model, SignNet, which exploits the notion of similarity (and dissimilarity) of visual signs in translating. This module presented is only one part of a dual-learning two task process involving text-to-sign (T2S) as well as sign-to-text (S2T). We currently implement SignNet as a single channel architecture so that the output of the T2S task can be fed into S2T in a continuous dual learning framework. By single channel, we refer to a single modality, the body pose joints. In this work, we present SignNet, a T2S task using a novel metric embedding learning process, to preserve the distances between sign embeddings relative to their dissimilarity. We also describe how to choose positive and negative examples of signs for similarity testing. From our analysis, we observe that metric embedding learning-based model perform significantly better than the other models with traditional losses, when evaluated using BLEU scores. In the task of gloss to pose, SignNet performed as well as its state-of-the-art (SoTA) counterparts and outperformed them in the task of text to pose, by showing noteworthy enhancements in BLEU 1 - BLEU 4 scores (BLEU 1: 31->39; ~26% improvement and BLEU 4: 10.43->11.84; ~14\% improvement) when tested on the popular RWTH PHOENIX-Weather-2014T benchmark dataset
