Little Brains, Big Feats: Exploring Compact Language Models

This study evaluates small language models during the generation stage of retrieval-augmented generation (RAG). The authors benchmark models using open-source and proprietary datasets spanning multiple subject areas and question types. According to the paper’s abstract, RAG systems built with small language models can run directly on-device without GPU hardware within a reasonable amount of time. Experimental code and supplementary-material links are available in the associated GitHub repository, SLM-RAG-EVAL.