CoRA-NAS：融合粗排序与锚点残差微调的高效神经架构搜索

CoRA-NAS Framework Summary

My thinking starts with the challenge: zero-cost proxies offer inexpensive architecture ranking, but their reliability is inconsistent across different search spaces. I'm addressing this with CoRA-NAS (COarse Ranking + Anchor-residual), a novel two-stage framework. This framework uniquely combines a static ranking prior with a low-cost learning-curve refinement process. In the first stage, CoRA-Rank aggregates capacity and structure-at-initialization proxies using an equal-weight log-rank consensus and a target-free consensus gate. The second stage, CoRA-Refine, then takes over. It strategically samples anchors based on the ranking prior, extrapolates their early validation curves, and propagates a learned residual correction using an ExtraTrees model. The beauty of this is that the refinement step uses only about 1% of the computational cost needed to fully train the candidate architectures. Crucially, I'm not using fully trained architecture-accuracy data to train the ranker, which is quite an advantage in many cases. A key design choice is that I utilize a single configuration across different search spaces, but with space-specific architecture encodings. The resulting CoRA-Refine achieves mean Spearman correlations of 0.946, 0.715, 0.786, and 0.894 on NAS-Bench-201, NAS-Bench-101, TransNAS-Bench-101, and NATS-SSS, respectively. Even in the worst-performing search space, the correlation of 0.715 is the highest among compared methods. Furthermore, on NAS-Bench-201/CIFAR-100, the selected architecture achieves 73.32% accuracy, close to the reported ground-truth best of 73.37%. In a pure size space, CoRA-NAS refinement successfully recovers the static prior's underperformance, relative to a parameter count, while remaining equivalent to the strongest capacity proxies within noise. Finally, this framework successfully merges cross-space ranking robustness with low-cost architecture selection.

零成本代理能够以极低成本对网络架构进行排序，但其可靠性在不同搜索空间中存在较大差异。我们提出了 CoRA-NAS（COarse Ranking + Anchor-residual），这是一个将静态排序先验与低成本学习曲线细化相结合的两阶段框架。CoRA-Rank 通过等权对数排名共识和无目标共识门控，聚合了模型容量代理与初始化结构代理。CoRA-Refine 在该先验中采样锚点，外推其早期验证曲线，并利用 ExtraTrees 模型传播学得的残差修正。该细化过程仅消耗完全训练候选集约 1% 的计算成本。拟合排序器时未使用完全训练后的架构-准确率标签。该方法在不同空间中采用统一配置，并配合特定于空间的架构编码。在 NAS-Bench-201、NAS-Bench-101、TransNAS-Bench-101 和 NATS-SSS 上，CoRA-Refine 分别取得了 0.946、0.715、0.786 和 0.894 的平均斯皮尔曼相关系数。其在表现最差空间上的相关系数仍达 0.715，为所有对比方法中最高。在 NAS-Bench-201/CIFAR-100 上，其选出的架构准确率达到 73.32%，接近所报告的真实最优值 73.37%。在纯尺寸空间中，细化阶段弥补了静态先验相较于参数量指标的不足，同时在误差范围内与最强的容量代理保持持平。最终的框架兼具了跨空间的排序鲁棒性与低成本的架构选择能力。