Transformer 中的距离泛化：位置编码究竟有多重要？

My Translation Process

Alright, let's break this down systematically. First, I need to understand the request: I'm given an abstract about Transformer research, specifically about distance generalization, out-of-distribution length generalization, and related concepts like RoPE, ALiBi, and delay copy tasks. My task is to translate this abstract into Simplified Chinese, keeping the paragraph structure and ensuring I only output the translation, without any additional commentary.

I'll tackle this sentence by sentence. "Out-of-distribution length generalization" - I'll translate that as "分布外长度泛化" (fēnbù wài chángdù fànguà). This is a standard term. "Distance generalization" needs careful consideration. I'll go with "距离泛化 (jùlí fànguà)" - it is the most direct and accurate. I'll need to translate "inter-token distances", I'm thinking about "标记间距离 (biāojì jiān jùlí)" or just "token 间距离(token jiān jùlí)". I think "token" is well-understood, so I'll stay consistent with that. For "delay copy tasks", I'll translate that as "延迟复制任务 (yánchí fùzhì rènwù)". And for "source and recall", I'll go with the more natural "源与召回 (yuán yǔ zhàohuí)". Terms like "distance resolution" will be translated as "距离分辨率 (jùlí fēnbiànlǜ)". For "positive or negative transfer", I'll use "正迁移或负迁移 (zhèng qiān yí huò fù qiān yí)".

Now, I'll put it all together. I'll make sure each sentence is clear and flows well in Chinese. I'll keep the paragraph structure intact, ensuring the final output is concise and professional.

分布外长度泛化，即将任务从短上下文外推到更长上下文的能力，已在 Transformer 模型中得到了深入研究。本文聚焦于距离泛化，即在保持固定上下文长度的同时，探究在训练和推理之间改变 token 间距离时的模型性能。我们构建了两个合成延迟复制任务，两者均涉及源（source）与召回（recall）之间的有限距离，其中 token 会被完全或选择性地复制，并在训练期间未见过的延迟距离上测试了模型。我们探讨了三个问题：(A) 相比于无位置编码（NoPE），RoPE 和 ALiBi 等位置编码方案能否提高距离分辨率？(B) 数据多样性（即训练中见过的 token 间距离的数量）如何影响性能？(C) 距离迁移学习何时产生正向或负向效果？我们进行了深入的研究，发现加深对其底层机制的理解至关重要。