TLDR: PGTT (Phase-Guided Terrain Traversal) is a deep reinforcement learning method for legged robots that improves locomotion on challenging terrains. It achieves this by using reward shaping to guide the robot’s gait, adapting foot trajectories to local terrain information without constraining the robot’s action space. This design reduces learning bias and enhances adaptability across different robot morphologies. Trained in simulation with curriculum and domain randomization, PGTT shows higher success rates, faster learning convergence, and successful real-world transfer on robots like the Unitree Go2 and ANYmal-C.
Legged robots hold immense promise for navigating complex, real-world environments, from disaster zones to human-made structures. However, teaching these robots to walk reliably on uneven and challenging terrain has remained a significant hurdle. Traditional methods often struggle with adaptability or anticipating obstacles, leading to brittle performance.
A new research paper titled PGTT: Phase-Guided Terrain Traversal for Perceptive Legged Locomotion introduces an innovative solution called Phase-Guided Terrain Traversal (PGTT). Authored by Alexandros Ntagkas, Chairi Kiourt, and Konstantinos Chatzilygeroudis, this approach leverages deep reinforcement learning to enable robots to traverse difficult landscapes with greater robustness and efficiency.
The Challenge of Legged Locomotion
Current state-of-the-art reinforcement learning controllers for legged robots typically fall into two categories. Some impose rigid gait patterns, often based on oscillators or inverse kinematics (IK), which can limit the robot’s movement options and make it less adaptable to different robot designs. Others operate without much foresight, struggling to anticipate terrain changes, especially for their hind legs, and are prone to errors when faced with real-world noise.
Introducing PGTT: A Smarter Way to Walk
PGTT addresses these limitations by enforcing gait structure purely through a clever reward system, rather than by directly restricting the robot’s actions. This means the robot’s learning process has less pre-programmed bias, allowing it to discover more natural and adaptable ways of moving. The core idea is to guide each leg’s movement using a ‘phase’ variable, which dictates whether the leg should be in contact with the ground or swinging through the air.
PGTT uses a cubic Hermite spline to define smooth, desired foot trajectories. Crucially, this trajectory adapts its swing height based on local terrain information, ensuring the robot lifts its feet high enough to clear obstacles. The policy, which is the robot’s learned brain, directly controls the joint movements, making it flexible enough to be deployed on various robot morphologies without significant re-tuning.
How PGTT Works Under the Hood
The system combines three key ideas: a compact perception module, phase variables with a unique reward function, and an asymmetric actor-critic learning architecture.
- Perception: PGTT uses a ‘robot-centric heightmap’ derived from LiDAR measurements. This map provides a local, real-time understanding of the terrain immediately around the robot, without needing complex global positioning.
- Gait Structure through Reward: Instead of forcing specific joint movements, PGTT’s reward function encourages the robot to follow the terrain-adaptive foot trajectories. It also penalizes the robot if its foot touches the ground during the swing phase, promoting clear obstacle clearance. This reward-based guidance provides structure without stifling the robot’s learning.
- Learning Architecture: The system uses an asymmetric actor-critic model, where the ‘actor’ (the part that decides actions) uses only observations, while the ‘critic’ (the part that evaluates actions) has access to more detailed, ‘privileged’ information during training. This helps the robot learn more effectively without needing perfect sensing in the real world.
The training takes place in the MuJoCo (MJX) simulation environment, which is GPU-accelerated for high efficiency. The robots are trained on procedurally generated stair-like terrains, with a curriculum that gradually increases difficulty and domain randomization to improve real-world transfer.
Impressive Results and Real-World Application
In extensive simulations, PGTT demonstrated superior performance. It achieved the highest success rates when facing push disturbances (7.5% better than the next best method) and on discrete obstacles (9% better). It also converged to an effective policy roughly twice as fast as other strong baseline methods, all while maintaining comparable velocity tracking.
The researchers validated PGTT on a real-world Unitree Go2 robot, using a real-time LiDAR system to generate heightmaps. The robot successfully demonstrated robust stair and obstacle traversal. Preliminary results on an ANYmal-C robot, obtained with the same training parameters, further highlight PGTT’s ability to generalize across different robot platforms.
Also Read:
- Bridging the Reality Gap: A New Approach to Sim2Real Reinforcement Learning
- Improving Robot Collaboration in Unseen Teams
Conclusion
PGTT offers a simple, general, and effective mechanism for robust perceptive locomotion. By guiding gait structure through reward shaping rather than rigid action constraints, it allows legged robots to learn highly adaptable behaviors. Its reliance on the lightweight MuJoCo simulation stack also makes advanced perception-aware locomotion research more accessible, enabling more researchers to contribute to this exciting field.
