Abstract
Streaming text-to-motion generates continuous motion from instructions that arrive one at a time, without access to future prompts. Training such a model should assign each piece of text to the motion it describes, yet many motion datasets caption entire clips without marking where each action begins and ends. We present GC-Motion, a streaming text-to-motion framework that learns from annotations of different temporal granularity. Its granularity-aware conditioning uses temporally aligned descriptions to supervise token-level diffusion forcing, while clip- and class-level captions train the same generator through whole-window denoising. A generated motion can still exhibit foot sliding when root displacement and local pose are decoded separately. Our contact-guided decoder uses predicted foot contacts and foot kinematics to adjust the horizontal root displacement before each block is emitted. Experiments show that granularity-aware conditioning reduces heading drift when incorporating additional motion data with mixed annotations. With the generator and latents fixed, contact-guided decoding lowers median stance-ankle speed from 5.08 to 1.44 cm/s. Compared with four streaming baselines, GC-Motion scores higher on text matching, smoothness, and physics while meeting our real-time targets. We further demonstrate real-time onboard generation on a full-size humanoid robot.
Interactive demo
Each session below is a stream of prompts played back in your browser in real time. The prompts were sent at the listed times to the final model, which saw each prompt only when it arrived. A prompt takes effect at the next 4-frame commit (133 ms), even when the previous motion is still running. After a prompt's duration the model returns to standing. The sessions use the same prompt sets as the tasks in the real-robot video. All sessions use seed 0, except the Fig. 1 sequence, which uses the seed of the figure. None of the sessions were selected from multiple runs.
The viewer shows the 22 body joints of the generated SMPL-X motion. Each session was generated with the final model (contact-guided decoder with causal foot locking, CFG 2.0, a 4 s window) and stored as joint positions, so no model runs in the browser.
Real-robot video
The final model generates motion onboard the robot's Jetson Orin NX while an operator types the prompts. A kinematic retargeter and a whole-body tracking controller drive the robot from the generated motion. All footage plays at 1× speed (9 tasks, 5 min 06 s in total); the tasks play one after another.
Method
Granularity-aware conditioning
Each caption is routed by the time span it describes. Temporally aligned descriptions condition individual tokens in diffusion forcing. Clip- and class-level captions train the same generator through whole-window denoising, so coarse annotations add data without teaching the model the wrong text-to-time alignment.
Streaming generation
A streaming diffusion transformer denoises a window of motion latents with a noise level that increases along the window. The oldest token is committed every 4 frames. A new prompt conditions every token that has not been committed yet, so the motion changes within one commit.
Contact-guided decoding
Decoding root displacement and local pose separately lets planted feet slide. The causal decoder predicts foot contacts and uses foot kinematics to correct the horizontal root displacement before each block is emitted. It keeps the latent space of the existing generator.
Results at a glance
- Compared with four streaming baselines, GC-Motion scores higher on text matching, smoothness and physics and meets the real-time targets.
- With the generator and latents fixed, contact-guided decoding lowers the median stance-ankle speed from 5.08 to 1.44 cm/s.
- Granularity-aware conditioning reduces heading drift when training on additional motion data with mixed annotations.
- Onboard a full-size humanoid (Jetson Orin NX), each 4-frame step takes about 58 ms, within the 133 ms budget.
BibTeX
@article{gcmotion,
title = {GC-Motion: Streaming Text-to-Motion with Granularity-Aware Conditioning and Contact-Guided Decoding},
author = {Fung, Leo and Jamie and Yu, Jack and Wang, Zhao},
journal = {arXiv preprint},
year = {2026}
}