论文标题
探索视觉提示以适应大型模型
Exploring Visual Prompts for Adapting Large-Scale Models
论文作者
论文摘要
我们研究了视觉促使在视觉中适应大型模型的功效。在迅速调整和对抗性重编程中的最新方法之后，我们学习了单个图像扰动，以便促使这种扰动的冷冻模型执行了一项新任务。通过全面的实验，我们证明了视觉提示对于剪辑和稳健的分配转移特别有效，可以通过标准线性探针实现性能竞争。我们进一步分析了下游数据集的属性，有关适应性性能的及时设计和输出转换。视觉提示的令人惊讶的有效性为在视觉中适应预训练的模型提供了新的观点。代码可在http://hjbahng.github.io/visual_prompting上找到。
We investigate the efficacy of visual prompting to adapt large-scale models in vision. Following the recent approach from prompt tuning and adversarial reprogramming, we learn a single image perturbation such that a frozen model prompted with this perturbation performs a new task. Through comprehensive experiments, we demonstrate that visual prompting is particularly effective for CLIP and robust to distribution shift, achieving performance competitive with standard linear probes. We further analyze properties of the downstream dataset, prompt design, and output transformation in regard to adaptation performance. The surprising effectiveness of visual prompting provides a new perspective on adapting pre-trained models in vision. Code is available at http://hjbahng.github.io/visual_prompting .
