Apple's machine learning research team published a paper on arXiv on November 4 titled Perceptive Humanoid Motion from Simulation, describing HALO, an autonomous control system for humanoid robots, and validating it on a real Unitree G1. All eight authors are listed under Apple; the first author is Siyuan Hu and the last is Vladlen Koltun.
HALO stands for Humanoid Autonomy, Learned in simulation, running Onboard. Its motion data is generated mostly in simulation, and the robot is then dropped into real rooms it has never seen and follows language instructions to walk over to things, sit on chairs, crawl under tables and push boxes.
Three layers, two of them running Qwen
The system has three layers, running from slowest to fastest:
- Orchestration layer: the vision-language model Qwen3.6-35B-A3B (4-bit quantized), which runs roughly every 2 seconds, breaks an instruction into subtasks and decides when to advance and when to retry.
- Motion generation layer: a frozen Qwen3.5-2B vision encoder paired with a 49-million-parameter diffusion Transformer, running about 3 times per second and planning the next 5 seconds of whole-body motion.
- Tracking layer: a policy trained with reinforcement learning in Isaac Lab, running at 50 Hz. It reads depth images, proprioception and IMU data and outputs joint targets for the G1's 29 joints.
Every model runs on the robot itself. The G1 carries an M4 Pro Mac mini with 64 GB of memory on its back, which handles the first two layers, while the tracking layer runs on the G1's built-in NVIDIA Jetson Orin NX. Nothing is tethered.
For training data, the team used the motion diffusion model Kimodo to generate movements and Infinigen to procedurally generate 1,000 rooms, then rendered 162,698 clips in Isaac Sim, about 296 hours in total. No teleoperation data was used.
Results and shortcomings
The real-world tests covered 6 real environments and 6 tasks, with 10 trials per task per environment, for 360 trials in all. Average single-task success was 91%, matching the 91% seen in simulation: crawling under a table 100%, passing through a half-open door 93%, walking to an object 92%, pushing a box 88% and sitting on a chair 85%.
Sequential tasks were clearly harder. Across three two-step sequences the average was about 70%. Walking through a door and then approaching an object succeeded in all 10 trials, while crouching and then sitting managed only 60%.
The paper lists several limitations itself. Each new skill still requires a person to write prompts and keyframes, and in the authors' words the skill library "grows only as fast as a person can specify additional tasks." The robot has no sense of touch. None of the six skills involves grasping or manipulating objects with hands. And the paper does not compare against other published systems on the same tasks, so the 91% figure can't be set directly against anyone else's numbers.
Set against the other route in the industry
The mainstream way to get training data for humanoids is teleoperation and wearable motion capture, which the authors describe as taking "at least an hour of human effort to produce an hour of data." HALO takes the all-simulation route: not a single hour of its nearly 300 hours of motion data came from a human operator.
The trade-off between the two is plain. Teleoperated data sits closer to real physics, and it is the most dependable source for jobs done by hand, such as grasping or opening a drawer. Simulated data is plentiful and cheap, but so far it has only been shown to work for whole-body locomotion: walking, sitting, crawling and pushing. The six tasks HALO chose conveniently avoid the hardest parts of simulation, contact and grasping.
The code is public on Apple Machine Learning Research's GitHub, including data generation, generator and tracker training, and deployment code. The README, however, includes no trained weights and offers no download of the roughly 300 hours of data. Apple has not announced any robot product plans; this remains a research paper. For developers in China, one point is practical: both "brains" are open-weight Qwen models and the hardware is a Unitree G1, so the barrier to reproducing it comes mainly down to that Mac mini and the compute needed to generate your own data.
Sources: Apple's arXiv paper, Humanoids Daily, CocoLoop, Sina Tech; the three-layer model configuration, simulated data scale and 360-trial real-world success rates were checked against the paper, and the scope of the open-source release follows the project README.