Can Dialects Be Steered Like Languages? Sparse Neurons and Distributed Directions in Arabic LLMs

This paper studies why Arabic LLMs tend to overproduce Modern Standard Arabic (MSA) instead of dialectal varieties. It evaluates two inference-time approaches that require no dialect-specific fine-tuning: identifying sparse neurons associated with dialectal features and amplifying or suppressing them, and extracting dialect-specific activation directions for vector steering. The authors present both methods as interpretability probes and control mechanisms, aiming to clarify how dialectal knowledge is geometrically represented inside Arabic LLMs while improving dialect-targeted generation.