Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generation

The paper introduces SpectraReward, a training-free reward function that uses a pretrained multimodal large language model as an image-generation reward model. Rather than asking the model to issue a judgment, it computes the average image-conditioned, teacher-forced log-likelihood of reconstructing the original prompt from the generated image. Self-SpectraReward applies the same idea within unified multimodal models, using the policy’s understanding branch to reward its generation branch. The study spans two diffusion models, three reinforcement-learning algorithms, nine MLLM backbones from four families, model sizes from 4B to 235B, and five out-of-distribution text-to-image benchmarks.