Academic Paper · cs.RO

Predictive Action Diffusion for Steerable Onboard Humanoid Control

PredActor predicts future states for guidance and generates actions for direct execution, using only proprioceptive observations and optional task context.

  • Onboard G1 DeployProprioceptive Only
  • 50 HzJetson Orin
  • End to EndDirect Action Output
  • CG + CFGenabled
Method overview Qualitative demos

Anonymous Authors

Anonymous Institution
Deployable inputs
Proprioceptive historyPredActorSelected action
predicted states stay internal · selected action executes

Real robot footage · 0.5x / 1.0x

01 / OverviewMethod · Simulation · Hardware

Predictive action control

Guide future motion.
Execute actions directly.

Humanoid policies must follow task commands while adapting to physical feedback. Motion-reference generators rely on separate trackers for execution. Action-only diffusion generates actions directly, without an explicit future-state trajectory for look-ahead guidance. Joint state–action diffusion provides this predictive representation.

PredActor builds on the joint formulation to combine two steering interfaces: classifier guidance (CG) applies test-time objectives to predicted states, while classifier-free guidance (CFG) strengthens learned task conditions. Future states stay internal; selected joint actions execute directly, without a separate motion-reference tracker.

  1. 01

    Open-loop kinematic planning

    The generator cannot revise behavior from robot feedback, and its references still depend on a separate tracker for physical execution.

    Open loop
  2. 02

    Closed-loop kinematic planning

    Feedback can shape later plans, but kinematic generation remains separated from dynamics-level reaction, timing, and recovery.

    Closed loop
  3. 03

    Action-only diffusion

    Direct action prediction removes the tracking handoff but provides no future-state trajectory for look-ahead objectives.

    Action
  4. 04

    Joint state–action diffusion

    Internal future states enable predictive guidance. PredActor combines CG and CFG within this formulation, conditioned on proprioceptive history.

    Joint

Paper figure · control forms

Control designs, from references to actions.

Control-form comparison
Five-panel comparison of open-loop tracking, closed-loop tracking, action diffusion, joint diffusion, and PredActor. PredActor uses joint state-action diffusion with classifier-free and classifier guidance while outputting direct robot actions.
Control-form comparison.Panels (d,e) both use joint state–action diffusion. PredActor conditions prediction on proprioceptive history, keeps future states internal, and executes only the selected action.

On narrow screens, scroll horizontally to inspect the figure labels.

Method overview

Internal predictions.
Steerable actions.

PredActor jointly denoises future states and actions from proprioceptive history and optional task context. Predicted states support CG; conditional and null predictions support CFG. No externally estimated full-body state is required as a policy input.

Paper figure · architecture

Predictive action diffusion.

Method 01
Predictive action diffusion architecture. Proprioceptive observation and optional task conditioning form condition memory for a diffusion transformer that jointly denoises interleaved state and action tokens. Optional objective steering guides predicted states before the denoise update.
Architecture.Task conditioning and objective steering are separate interfaces around the joint state-action denoising core.

On narrow screens, scroll horizontally to inspect the SVG labels.

C1

Proprioceptive and steerable

Only proprioceptive observations are needed. CG guides internal predicted states; CFG strengthens learned task conditions, with direct action execution.

C2

Data and training pipeline

Automatic task labels, asynchronous perturbed teacher rollouts, and learner-state aggregation support the training pipeline.

C3

Onboard at 50 Hz

Rolling denoising, computation-preserving optimizations, and delay compensation enable deployment on a Unitree G1 with Jetson Orin NX.

C4

Task interface and implementation

An editable interface brings text commands, semantic interpolation, and whole-body steering to the same denoising core; the open-source implementation spans training through deployment.

Steering and onboard execution

Predictive control within the onboard budget.

In simulation, PredActor reaches all 15 destination targets and achieves 0.580 text retrieval versus 0.373 for conditional action diffusion, with similar observed push survival (0.535 versus 0.564).

In an offline, zero-actuation Jetson Orin NX benchmark, the complete callback takes 16.790 ms median and 19.383 ms p95; 593 of 600 measured callbacks meet the 20 ms period. This supports measured 50 Hz feasibility, not a hard-real-time guarantee. To our knowledge, PredActor is the first joint state–action diffusion policy deployed entirely on a Unitree G1's onboard Jetson Orin NX for 50 Hz control.

03 / PredActor demosFive standalone demos across simulation and hardware

Qualitative control evidence

Commands, interaction, and behavior transitions.

Across simulation and physical G1 evaluation, PredActor demonstrates text-conditioned motion, joystick steering, disturbance response, and behavior transitions. Two simulation sequences and three hardware demonstrations are followed by a paired simulation sweep of semantic targets at matched blend weights.

This PredActor behavioral reaction demo could not be loaded.

01 / External interaction

Behavior under physical interference.

This hardware sequence shows the policy following walk and stand commands while responding to external interference.

This PredActor walk, squat, and walk demo could not be loaded.

02 / Text control

Walk, squat, return to walk.

The commanded sequence transitions from locomotion into a squat and back to walking, with each text instruction shown alongside the hardware response.

This PredActor walk, jog, and squat demo could not be loaded.

03 / Text control

Walk, accelerate to jog, then squat.

A second commanded sequence changes locomotion speed before transitioning into a squat, preserving the text-to-behavior progression in one continuous view.

This comprehensive PredActor MuJoCo demo could not be loaded.

04 / Comprehensive simulation

Commands, interference, and semantic interpolation.

This MuJoCo sequence combines text commands and joystick steering, response to external interference, and continuous interpolation between behavior targets.

This PredActor joystick steering demo could not be loaded.

05 / Joystick steering

Direction and locomotion mode share one interface.

The simulation shows joystick-directed turns while text commands switch the locomotion mode between walking and running.

04 / Semantic interpolationQualitative comparison · two targets × five matched weights

Parallel behavior sweep

Semantic interpolation across two behavior targets.

Each column uses the same blend weight for both rows, showing the conditioning transition from standing to a gesture and from standing to locomotion.

(a) Gesture target

Stand to raise hand

Semantic blend
0%Stand
25%Blend
50%Blend
75%Blend
100%Raise hand
(b) Locomotion target

Stand to run

Semantic blend
0%Stand
25%Blend
50%Blend
75%Blend
100%Run

Authors and resources

Research team and project resources.

Anonymous Authors

Anonymous Institution

PredActor demosWatch simulation and hardware control sequencesView ↑