Copy the real world into simulation, train and test robots there, and bring the result back. What is known, who argues what, and the dates that will decide it, with a source for every fact.
Real-to-sim-to-real (Real2Sim2Real) means copying the real robot, room and objects into a simulator, training and testing the robot there thousands of times, and taking the result back to the real robot. It moved from a few lab papers in 2024 to a tool chain teams can download in 2026. A phone or depth-camera scan now becomes a simulated scene in minutes: Re³Sim reports about 2.5 minutes of reconstruction per scene (arXiv 2502.08645), and NVIDIA ships Gaussian-splat scene capture into Isaac Sim through Omniverse NuRec (NVIDIA, 11 Aug 2025). The physics underneath became open: Newton, built by NVIDIA, Google DeepMind and Disney Research, went to the Linux Foundation and reached version 1.0 in March 2026 (NVIDIA, 16 Mar 2026). Results on real robots are real: Imperial College's ZeroBot reports 87% real-world success after about two minutes of training from one image (arXiv 2609.34010), and World Labs trained policies with zero real training data on five robot types (World Labs, 28 Jul 2026). Money followed: Lightwheel raised two rounds of RMB 1 billion each in 2026 (Eastmoney, 23 Jun 2026), and World Labs bought the simulation start-up SceniX before agreeing to join AMD (World Labs, 28 Sep 2026). The limits are stated by the builders: legs transfer from simulation routinely, hands and soft objects do not yet; most pipelines handle rigid objects and slow motions.
The loop has three steps: capture the real scene, objects and robot into a simulation with geometry, appearance and physics such as mass and friction; train or test the policy there with variations; run it on the real robot and feed failures back (our note).
The name comes from Ken Goldberg's Berkeley group, which published "Real2Sim2Real" for robot cable casting in a 2021 preprint, with 8% to 14% median error over 240 physical trials (arXiv 2111.04814).
For legs, sim-to-real is routine: Agility trains a whole-body controller in Isaac Sim that transfers zero-shot to Digit (Agility, 28 Aug 2025); Unitree publishes a train, check and deploy workflow for Go2, H1 and G1 (GitHub); ANYbotics rolls out locomotion trained in simulation to its fleet (ANYbotics, 30 Oct 2023).
NVIDIA's GR00T N1 used 780,000 synthetic trajectories, equal to about 6,500 hours of human demonstrations, generated in 11 hours; mixed with real data they raised performance by 40% over real data alone (company figures; NVIDIA, 18 Mar 2025).
Skild AI trains on "trillions of synthetic experiences using approximate simulations" alongside human video and teleoperation (company account; Skild AI, 14 Jan 2026).
Galbot pretrained its GraspVLA model on SynGrasp-1B, a billion-frame synthetic grasp dataset (arXiv 2505.03233).
Exact twins: MIT's RialTo scans the deployment scene and reports more than 67% gains in policy robustness over baselines, with about 15 minutes of active work per twin and about two days of wall-clock time per task end to end (arXiv 2403.03949).
Cousins: Stanford's ACDC generates many similar scenes instead of one exact copy and reports 90% zero-shot success against 25% for twins on a door-opening task (arXiv 2410.07408).
Look and physics: SplatSim's Gaussian-splat scenes reached 86.25% average zero-shot success against 97.5% for policies trained on real data (arXiv 2409.10161); MIT's Scalable Real2Sim lets a robot measure mass within 1.34% and centre of mass within 2.15%, but inertia is still 42.35% off (arXiv 2503.00370).
PhysTwin builds real-time twins of ropes, plush toys, cloth and packages from sparse depth video, with far better shape prediction than the previous method, but it fits physics from a single type of interaction and needs three depth cameras (arXiv 2503.17973).
EgoPhys learns soft-object physics from ordinary first-person video and plans with it on a real robot arm (arXiv 2606.16202).
The engines are catching up: Newton adds solvers for deformable and granular materials, and NVIDIA names Samsung as a user for cable handling in refrigerator assembly (company claim; NVIDIA, 16 Mar 2026).
SIMPLER found a Pearson correlation of 0.924 between simulated copies of Google robot and WidowX setups and the real world, over more than 1,500 paired evaluations (arXiv 2405.05941).
SimFoundry reports a mean correlation of 0.911 over 7 tasks and 5 architectures (arXiv 2606.28276); X Square Robot reports 0.84 on its X2Real benchmark (company figure; arXiv 2609.27449); World Labs ran 2,000 simulated against 100 real trials per checkpoint and states that rankings held, without a coefficient (World Labs, 28 Jul 2026).
Evaluation is getting fast: NVIDIA's Isaac Lab-Arena, built with Lightwheel, ran 10 tasks in 0.76 hours in parallel against 34.9 hours one by one (company figure; NVIDIA, 5 Jan 2026).
NVIDIA sells the whole chain: NuRec capture, Isaac Sim 6.0 (8 Jun 2026), Isaac Lab, Lab-Arena evaluation, Newton physics and Cosmos for visuals (NVIDIA forum, Isaac Sim 6.0).
Lightwheel sells simulation-ready assets, egocentric human data and the RoboFinals evaluation; its CEO Xie Chen is a voice (Eastmoney, 11 Mar 2026).
World Labs acquired SceniX on 21 July 2026 (World Labs), published its real-to-sim-to-real pipeline a week later, and signed an agreement to join AMD on 28 September 2026 (World Labs).
The Physical AI Voices by Herbert Scale Experts.