ATLAS: Automated Approximation of Transformers for Efficient Homomorphic Inference in One Hour

ATLAS targets the high cost of transformer inference under CKKS fully homomorphic encryption (FHE), where softmax, normalization, and activations must be replaced by polynomial approximations. It formulates layer-specific approximation selection as a multi-objective optimization over latency and predictive accuracy. The paper reports configuration spaces of roughly 10^84 for 12-layer BERT/ViT models and 10^225 for 32-layer LLaMA3, with 35–50% of candidates producing numerically invalid outputs. Its two-stage search progressively relaxes layer constraints and uses surrogate models to reduce evaluation cost.