GC-Motion: Streaming Text-to-Motion with Granularity-Aware Conditioning and Contact-Guided Decoding

Leo Fung    Jamie    Jack Yu    Zhao Wang†*
LimX Dynamics
†Project lead    *Corresponding author

The model generates motion onboard a humanoid robot while prompts are typed live, here a new prompt every 1.5 s. Left: the prompts. Middle: the robot. Right: the generated motion. Real time, no speed-up.

Abstract

Streaming text-to-motion generates continuous motion from instructions that arrive one at a time, without access to future prompts. Training such a model should assign each piece of text to the motion it describes, yet many motion datasets caption entire clips without marking where each action begins and ends. We present GC-Motion, a streaming text-to-motion framework that learns from annotations of different temporal granularity. Its granularity-aware conditioning uses temporally aligned descriptions to supervise token-level diffusion forcing, while clip- and class-level captions train the same generator through whole-window denoising. A generated motion can still exhibit foot sliding when root displacement and local pose are decoded separately. Our contact-guided decoder uses predicted foot contacts and foot kinematics to adjust the horizontal root displacement before each block is emitted. Experiments show that granularity-aware conditioning reduces heading drift when incorporating additional motion data with mixed annotations. With the generator and latents fixed, contact-guided decoding lowers median stance-ankle speed from 5.08 to 1.44 cm/s. Compared with four streaming baselines, GC-Motion scores higher on text matching, smoothness, and physics while meeting our real-time targets. We further demonstrate real-time onboard generation on a full-size humanoid robot.

Teaser: prompts typed one at a time, onboard a humanoid robot, seven-axis comparison
(a) Prompts typed one at a time, each unknown until it arrives. (b) The same model onboard a humanoid robot in real time (Jetson Orin NX). (c) Seven-axis quality compared with streaming baselines.

Interactive demo

Each session below is a stream of prompts played back in your browser in real time. The prompts were sent at the listed times to the final model, which saw each prompt only when it arrived. A prompt takes effect at the next 4-frame commit (133 ms), even when the previous motion is still running. After a prompt's duration the model returns to standing. The sessions use the same prompt sets as the tasks in the real-robot video. All sessions use seed 0, except the Fig. 1 sequence, which uses the seed of the figure. None of the sessions were selected from multiple runs.

loading
…
Loading…
left right foot in contact · drag to rotate, scroll to zoom, double-click to reset
0.0 s

The viewer shows the 22 body joints of the generated SMPL-X motion. Each session was generated with the final model (contact-guided decoder with causal foot locking, CFG 2.0, a 4 s window) and stored as joint positions, so no model runs in the browser.

Real-robot video

The final model generates motion onboard the robot's Jetson Orin NX while an operator types the prompts. A kinematic retargeter and a whole-body tracking controller drive the robot from the generated motion. All footage plays at 1× speed (9 tasks, 5 min 06 s in total); the tasks play one after another.

Frames from live robot sessions
Frames from live sessions on the robot. Below each prompt (arrow), frames at the given time after it, with the background removed. Top: locomotion. Bottom: upper-body prompts.

Method

Method overview

Granularity-aware conditioning

Each caption is routed by the time span it describes. Temporally aligned descriptions condition individual tokens in diffusion forcing. Clip- and class-level captions train the same generator through whole-window denoising, so coarse annotations add data without teaching the model the wrong text-to-time alignment.

Streaming generation

A streaming diffusion transformer denoises a window of motion latents with a noise level that increases along the window. The oldest token is committed every 4 frames. A new prompt conditions every token that has not been committed yet, so the motion changes within one commit.

Contact-guided decoding

Decoding root displacement and local pose separately lets planted feet slide. The causal decoder predicts foot contacts and uses foot kinematics to correct the horizontal root displacement before each block is emitted. It keeps the latent space of the existing generator.

Results at a glance

Seven-axis comparison
  • Compared with four streaming baselines, GC-Motion scores higher on text matching, smoothness and physics and meets the real-time targets.
  • With the generator and latents fixed, contact-guided decoding lowers the median stance-ankle speed from 5.08 to 1.44 cm/s.
  • Granularity-aware conditioning reduces heading drift when training on additional motion data with mixed annotations.
  • Onboard a full-size humanoid (Jetson Orin NX), each 4-frame step takes about 58 ms, within the 133 ms budget.

BibTeX

@article{gcmotion,
  title   = {GC-Motion: Streaming Text-to-Motion with Granularity-Aware Conditioning and Contact-Guided Decoding},
  author  = {Fung, Leo and Jamie and Yu, Jack and Wang, Zhao},
  journal = {arXiv preprint},
  year    = {2026}
}