论文标题
在电子商务中使用有限库存的假设检验的基本问题
A fundamental problem of hypothesis testing with finite inventory in e-commerce
论文作者
论文摘要
在本文中，我们提请人们注意一个问题，该问题通常被在线环境中练习假设测试（A/B测试）的公司所忽视或忽略。我们表明，对实验中变体之间共享的有限库存进行实验可能会导致高误报率，因为违反了两组之间独立性的核心假设。我们在简化的设置中提供了对问题的详细分析，该设置的参数通过现实的场景告知。我们认为的设置是半限定条带的2美元二维随机步行。它足够丰富，可以考虑有限的库存，但同时简单地允许封闭形式的假阳性概率。我们证明可能会发生高的假阳性率，并开发适合在后续工作中设计适当测试的工具。我们的结果还表明，可能发生高的假阴性率。证明依赖于在半限制条中进行$ 2 $维的随机步行的功能限制定理。
In this paper, we draw attention to a problem that is often overlooked or ignored by companies practicing hypothesis testing (A/B testing) in online environments. We show that conducting experiments on limited inventory that is shared between variants in the experiment can lead to high false positive rates since the core assumption of independence between the groups is violated. We provide a detailed analysis of the problem in a simplified setting whose parameters are informed by realistic scenarios. The setting we consider is a $2$-dimensional random walk in a semi-infinite strip. It is rich enough to take a finite inventory into account, but is at the same time simple enough to allow for a closed form of the false-positive probability. We prove that high false-positive rates can occur, and develop tools that are suitable to help design adequate tests in follow-up work. Our results also show that high false-negative rates may occur. The proofs rely on a functional limit theorem for the $2$-dimensional random walk in a semi-infinite strip.
