Rapid Learning of Dexterous In-Hand Pen Writing through Real-Time Jacobian Estimation

In-hand pen writing with anthropomorphic robotic hands has long been bottlenecked by the contact complexity that simulators struggle to replicate and imitation learning cannot easily capture. A new embodied control framework bypasses reinforcement learning and offline demonstrations entirely, relying instead on real-time task Jacobian estimation executed on a laptop CPU. Following roughly 18 seconds of initialization, the physical hand begins articulating a grasped pen, achieving a mean in-plane precision of 0.6 mm across letters and shapes drawn on paper. By demonstrating arbitrary single-stroke writing through in-hand finger motion alone, the work presents a lightweight, compute-efficient alternative to data-heavy manipulation pipelines.