Data Pyramid for Embodied Manipulation

This paper organizes the embodied manipulation data ecosystem into five complementary sources: real-robot data, UMI-style data, egocentric and exocentric data, simulation data, and general vision-language data. It frames these sources around the trade-off between scalability and robot alignment, comparing data quality, diversity, reusability, and physical fidelity. The authors then examine data recipes for embodied brain models, vision-language-action models, and world-action models, linking data composition to perception, reasoning, planning, action generation, and world prediction. Six open challenges are identified, including tactile datasets, failure and recovery data, scalable collection, cross-embodiment action alignment, egocentric data for dexterous manipulation, and principled recipe design.