Abstract
Humanoid locomotion on granular terrain remains a significant challenge due to its complex foot-terrain interaction dynamics that are difficult to model. Existing approaches either ignore granular contact dynamics or incorporate simplified normal force models with heuristic tangential components. In this work, we present a physics-grounded granular contact model based on three-dimensional resistive force theory (3D RFT) and efficiently simulate granular terrain for reinforcement learning (RL) training. Unlike traditional rigid contact models and simplified granular contact models with ad-hoc heuristics, our contact solver produces physically accurate granular intrusion dynamics without resorting to heuristics. It captures realistic penetration and tangential drag during training, enabling the policy to learn behaviors that transfer reliably to real-world granular terrain where rigid contact models fail. To adapt to varying terrain conditions, we train a terrain-adaptive locomotion policy via teacher-student RL, using a variational autoencoder to encode terrain information into a compact latent representation. Simulation studies using material point method (MPM) with NVIDIA Newton demonstrate that our method generalizes to unseen granular terrains, achieves a significantly higher success rate than baselines, and demonstrates zero-shot terrain identification and adaptation. We further validate our approach through extensive hardware experiments across diverse real-world granular terrains including basalt, dry sand, and beach sand. To the best of our knowledge, this is the first demonstration of agile humanoid locomotion on real-world granular terrain.
Method
System diagram of our proposed terrain-adaptive humanoid locomotion. It has two-stage training where privileged teacher network is first trained with VAE terrain encoder and MLP policy. Then, this teacher policy is distilled to deployable student policy via a hybrid DAgger and PPO.
Granular Contact Modeling
Granular terrain contact model definition. The terrain model predicts contact force and moment based on the foot body's penetration depth, velocity angle, and orientation.
Per-Terrain Velocity Tracking
We deploy the trained RL policy on a Unitree G1 humanoid robot to validate our approach in the real world, tracking a range of commanded forward velocities across terrains of varying granularity.
Terrain 1: Basalt
FailFailFail
Terrain 2: Beach Volleyball
FailFail
Terrain 3: Beach
Terrain Transition
Because our policy conditions on estimated terrain stiffness, it adapts online as the robot crosses between terrains of different granularity within a single trajectory. Ours maintains stable locomotion through the transition, while the fixed-terrain baseline struggles to adjust its gait.
Ours
PPO (3D RFT)
Contact Model Ablation
3D RFT best captures three-dimensional granular intrusion dynamics
We evaluate the fidelity of our granular contact model with a single rigid body intrusion experiment. We compare (1) horizontal travel distance and sinkage, and (2) ground reaction forces against baseline contact models and a Material Point Method (MPM) reference simulation. 3D RFT achieves closest horizontal travel distance and sinkage to the MPM simulation, and GRF also presents the best agreement with MPM.
Policy Performance per Contact Model
3D RFT facilitate the best RL policy performance
We compare policies trained under different contact models across three simulated tasks spanning flat, rough, and deep granular terrain. As terrain becomes rougher and softer, 3D RFT policy maintains tracking performance while other baselines degrade and deviates a lot from command.
Task 1: Circular Walking on Flat Terrain
Task 2: Circular Walking on Rough Terrain
Task 3: Forward Walking on Deep Sand
Environment Design
Observation and Action Space
| Observation | Teacher Actor | Teacher Encoder | Student Actor | Student Encoder | Critic |
|---|---|---|---|---|---|
| Proprioceptive Observation ($\mathbf{o}_t$) | |||||
| Angular velocity ($\boldsymbol{\omega}_z$) | ✓ | ✓ | ✓ | ||
| Joint positions ($\mathbf{q}_j$) | ✓ | ✓ | ✓ | ||
| Joint velocities ($\dot{\mathbf{q}}_j$) | ✓ | ✓ | ✓ | ||
| Command velocity | ✓ | ✓ | ✓ | ||
| Projected gravity ($\mathbf{g}_b$) | ✓ | ✓ | ✓ | ||
| Previous action ($\mathbf{a}_{t-1}$) | ✓ | ✓ | ✓ | ||
| Proprioceptive History ($\mathbf{o}_{t-N:t}$) | |||||
| $N$-step history of proprioceptive states | ✓ | ✓ | |||
| Terrain Latent | |||||
| Teacher latent ($\mathbf{z}_t$) | ✓ | ||||
| Student latent ($\hat{\mathbf{z}}_t$) | ✓ | ||||
| Privileged Observation ($\mathbf{s}_t$) | |||||
| Base linear velocity ($\mathbf{v}_{xy}$) | ✓ | ✓ | |||
| Foot height, global frame ($\mathbf{p}_f$) | ✓ | ✓ | |||
| Foot contact state ($\mathbf{c}_f$) | ✓ | ✓ | |||
| Contact force ($\mathbf{F}_c$) | ✓ | ✓ | |||
| Swing time since lift-off ($\mathrm{T_s}$) | ✓ | ✓ | |||
| Media-dependent scaling coeff. ($\eta$) | ✓ | ✓ | |||
Action Space: The action $\mathbf{a}_t \in \mathbb{R}^{29}$ represents the desired whole-body joint position offsets from a nominal pose $\mathbf{q}_{0,j}$, i.e., $\mathbf{a}_t = \mathbf{q}_j^{\text{des}} - \mathbf{q}_{0,j}$.
Reward Functions
| Reward | Expression | Weight |
|---|---|---|
| Task | ||
| Track lin. vel. XY | $\exp\!\left(-\|\mathbf{v}_{xy} - \mathbf{v}_{xy}^*\|^2_2 / \sigma^2\right)$ | $4$ |
| Track ang. vel. Z | $\exp\!\left(-\|\boldsymbol{\omega}_z - \boldsymbol{\omega}_z^*\|^2_2 / \sigma^2\right)$ | $1$ |
| Track yaw quat. | $\exp\!\left(-\|\mathbf{q}_z - \mathbf{q}_z^*\|^2_2 / \sigma^2\right)$ | $4$ |
| Track height | $\exp\!\left(-(\mathrm{z} - \mathrm{z}^*)^2 / \sigma^2\right)$ | $0.1$ |
| Termination | $-\mathrm{n}_\text{termination}$ | $200$ |
| Collision penalty | $-\mathrm{n}_\text{collision}$ | $10$ |
| Regularization | ||
| Action rate | $\|\mathbf{a}_t - \mathbf{a}_{t-1}\|^2_2$ | $-5\times10^{-3}$ |
| Lin. vel. Z | $\mathrm{v}_\mathrm{z}^2$ | $-1\times10^{-2}$ |
| Ang. vel. XY | $\|\boldsymbol{\omega}_{xy}\|^2_2$ | $-1\times10^{-4}$ |
| Joint deviation | $\|\mathbf{q}_j - \mathbf{q}_{0,j}\|_1$ | $-1.0$ |
| Joint vel. | $\|\dot{\mathbf{q}}_j\|^2_2$ | $-2\times10^{-4}$ |
| Joint accel. | $\|\ddot{\mathbf{q}}_j\|^2_2$ | $-2.5\times10^{-7}$ |
| Joint energy | $\|\dot{\mathbf{q}}_j \odot \boldsymbol{\tau}_j\|^2_2$ | $-1\times10^{-3}$ |
| Style | ||
| Straight base | $\|\mathbf{g}_b\|^2_2$ | $-10$ |
| Feet orientation | $\|\mathbf{g}_f\|^2_2$ | $-1.0$ |
| Feet slide | $\mathrm{c_f} \cdot \mathbf{v}_f$ | $-0.01$ |
| Feet air time | $\max(\mathrm{T_s}, 0.5)$ | $0.5$ |
| No fly | $\mathrm{n}_\text{zero-contact}$ | $-0.5$ |
| Contact impact | $\mathbb{1}_\text{first-contact} \cdot \mathrm{F_z}$ | $-5\times10^{-3}$ |
| Feet distance | $\|\mathbf{p}_{f,L} - \mathbf{p}_{f,R}\|^2_{xy}$ | $-2$ |
| Feet clearance | $\|\mathbf{p}_f - \mathbf{p}_f^{\text{des}}\|_z^2$ | $-2$ |
Domain Randomization
| Parameter | Range | Unit |
|---|---|---|
| Robot | ||
| Added base mass | $[-5.0, 5.0]$ | kg |
| Center of mass offset | $[-3.0, 3.0]$ | cm |
| Actuator gain scale | $[0.9, 1.1]$ | — |
| Push robot | $[-1.0, 1.0]$ | m/s |
| Terrain | ||
| Internal friction | $[0.1, 1.0]$ | — |
| Surface friction | $[0.1, 1.0]$ | — |
| Material density | $[1000.0, 3000.0]$ | kg/m³ |
| Packing ratio | $[0.5, 1.0]$ | — |