Reinforcement learning (RL) has become a cornerstone of modern robotics research, yet many general-purpose RL libraries fall short when it comes to the unique demands of robotic systems—such as high-throughput simulation, sim-to-real transfer, and algorithmic simplicity for rapid iteration. Enter RSL-RL: an open-source reinforcement learning library purpose-built for robotics researchers and engineers. Developed by the Robotic Systems Lab at ETH Zurich in collaboration with NVIDIA, RSL-RL prioritizes a compact, readable, and easily extensible codebase that empowers users to adapt core algorithms without wrestling with framework complexity.
Unlike bloated, one-size-fits-all RL toolkits, RSL-RL focuses exclusively on the methods that work best in robotics: Proximal Policy Optimization (PPO) and Student-Teacher Distillation, enhanced with robotics-specific techniques like Random Network Distillation (RND) for exploration and Symmetry-based Augmentation for natural, coordinated motion. Optimized for GPU-only training, it delivers high-throughput performance in large-scale simulators and has been validated not just in simulation—but on real hardware.
If you’re developing learning-based controllers for legged robots, manipulators, or other embodied agents—and you value simplicity, speed, and real-world applicability—RSL-RL is worth your attention.
Why Robotics Needs a Specialized RL Library
General RL libraries often assume flexible compute environments, diverse algorithmic needs, and abstract reward structures. Robotics, by contrast, operates under strict constraints: sparse rewards, high-dimensional continuous control, safety-critical deployment, and a strong reliance on simulation. These challenges demand tools that are not only fast but also transparent and modifiable.
RSL-RL was designed from the ground up to address this gap. Its codebase avoids unnecessary abstractions, making it straightforward to inspect, debug, and extend. This philosophy enables researchers to iterate quickly on novel reward functions, observation spaces, or policy architectures—without being bottlenecked by framework limitations.
Core Features That Solve Real Robotics Problems
Proven Algorithms, Minimal Overhead
RSL-RL ships with robust implementations of PPO and Student-Teacher Distillation, two of the most widely adopted approaches in robotic learning. PPO provides stable, sample-efficient training for continuous control, while distillation enables knowledge transfer from a high-capacity “teacher” policy (e.g., trained in simulation) to a compact “student” policy suitable for onboard execution.
Curiosity-Driven Exploration with RND
In many robotic tasks—especially locomotion or manipulation—the reward signal is sparse or delayed. To prevent agents from getting stuck, RSL-RL integrates Random Network Distillation (RND), which adds an intrinsic curiosity reward based on prediction error. This encourages the agent to explore novel states, dramatically improving learning in environments where external rewards are infrequent.
Enforcing Natural Symmetry
Legged robots, in particular, benefit from symmetric gaits. RSL-RL includes Symmetry-based Augmentation, a technique that modifies the training process to encourage policies that produce mirrored behaviors across left and right limbs. This leads to more natural, energy-efficient, and robust motion patterns—without hand-crafted symmetry constraints in the reward function.
Ideal Use Cases: Where RSL-RL Shines
RSL-RL excels in GPU-accelerated simulation environments commonly used in robotics research:
- Isaac Lab (built on NVIDIA Isaac Sim)
- Legged Gym (built on NVIDIA Isaac Gym)
- MuJoCo-based environments like MuJoCo Playground and mjlab
These frameworks enable thousands of parallel simulation instances, and RSL-RL’s GPU-optimized design leverages this parallelism for rapid training. More importantly, controllers trained with RSL-RL have consistently demonstrated successful sim-to-real transfer—a critical benchmark for any robotics learning library.
Whether you’re training a quadruped to walk over rough terrain or a manipulator to grasp objects under uncertainty, RSL-RL provides the right blend of performance, simplicity, and robotics-aware features to accelerate your development cycle.
Getting Started Is Simple
Installation is straightforward:
pip install rsl-rl-lib
Or, for development:
git clone https://github.com/leggedrobotics/rsl_rl cd rsl_rl pip install -e .
Configuration is handled via intuitive YAML files (see example_config.yaml), and RSL-RL natively supports popular logging backends:
- TensorBoard
- Weights & Biases
- Neptune
This plug-and-play design means you can go from zero to a trained policy in minutes—not days—freeing you to focus on your research questions rather than infrastructure plumbing.
Limitations and When Not to Use RSL-RL
RSL-RL is not a general-purpose RL library. It intentionally supports only a narrow set of algorithms (PPO and distillation-based methods) and assumes a GPU-only, large-batch simulation setup.
It is not suitable for:
- Offline RL or model-based RL
- CPU-only environments
- Tasks requiring algorithms like DQN, SAC, TD3, or A3C
- Non-robotics domains (e.g., board games, NLP, or recommendation systems)
If your project demands broad algorithmic coverage or runs on limited hardware, a more general framework like Stable Baselines3 or RLlib may be more appropriate.
Community Trust and Research-Grade Quality
Maintained by leading researchers at ETH Zurich and NVIDIA, RSL-RL adheres to rigorous software engineering standards:
- Google-style docstrings
- Automated linting and formatting via pre-commit and Ruff
- Clear contribution guidelines
Its methods have been published in top-tier venues like ICRA and CoRL, and the library is actively used in both academic research and real-world robotic deployments. This combination of academic rigor and practical utility makes RSL-RL a trustworthy foundation for serious robotics work.
Summary
RSL-RL fills a crucial niche in the reinforcement learning ecosystem: a lightweight, transparent, and robotics-first library that prioritizes real-world performance over generality. By focusing on the algorithms and techniques that matter most in robotics—and eliminating unnecessary complexity—it empowers researchers to build, test, and deploy learning-based controllers faster and more reliably. If your work involves training embodied agents in simulation with an eye toward real hardware, RSL-RL deserves a place in your toolkit.