UEmbed: Unified Sparse and Dense Multimodal Embeddings

UEmbed is a decoder-only multimodal embedding model that produces dense and learned sparse representations in one causal forward pass. It appends N learnable special tokens, partitions the vocabulary into N disjoint subsets, and uses each token’s causal hidden state to predict sparse weights for its assigned subset. The authors release 2B, 4B, and 9B models trained on public data. UEmbed-9B reportedly scores 71.8 on dense and 71.0 on sparse MMEB-v2, outperforming multimodal embedding models trained on public data such as RzenEmbed, while remaining competitive with dense and sparse baselines on BEIR.