Deep RL for Robotics Continuous Control, Locomotion & Sim

Master robot RL: policy gradients, PPO, reward design and sim-to-real transfer for quadrupeds and drones
What you'll learn

---

Derive the policy gradient theorem and PPO's clipped objective from scratch, with every step worked on screen rather than quoted as a result.
Design reward functions for continuous control, and recognise reward hacking by reproducing it on a real simulated quadruped, quadrotor and tracked robot.
Diagnose why a policy that works in simulation fails on hardware, and apply domain randomisation to close the reality gap.
Read and debug a real PPO implementation: advantage estimation, parallel environments, and the implementation details that decide whether it trains at all.
Requirements

---

Comfortable reading Python.
Basic calculus and linear algebra
No reinforcement learning background needed.
Helpful but not required: exposure to classical control (PID) or robotics. If you have used CartPole or Atari RL before, expect those intuitions to be challenged.
Description

---

This course contains the use of artificial intelligence.
Most reinforcement learning courses teach you CartPole. Then you try to apply it to a robot and discover that almost nothing transfers.
This course starts where those stop. Every example is a quadruped, a quadrotor or a tracked ground robot. Every action is a continuous torque or thrust carrying real units and real limits - never a
discrete button press. If you have trained an agent to balance a pole and felt no closer to controlling a machine, this is the course that closes that gap.
Every derivation is worked on screen. The policy gradient theorem is built term by term, not quoted. Generalised advantage estimation is derived, not cited. PPO's clipped objective is assembled
piece by piece, and then we turn the clipping off to show exactly what it was preventing. You will finish able to read a real PPO implementation and know what each line is defending against.
Reward design gets a full section, because it is where robot RL actually fails. You will watch reward hacking reproduced rather than described: a velocity reward satisfied by vibrating in place, a
height reward satisfied by flipping onto its back with feet in the air, a goal reward satisfied by knocking the goal over. None of these are bugs in the algorithm. Each is the algorithm doing
precisely what the number told it to.
Two full sections anatomise the reality gap - why a policy that walks perfectly in simulation falls over on real hardware, and what domain randomisation can and cannot do about it.
One honest limit, stated up front: this course teaches the reality gap without crossing it. There is no physical robot. Every "real robot" on screen is a deliberately mismatched simulation, and it
is labelled as such every time.
Nothing is installed. No GPU. No hardware. Every notebook is CPU-only and runs top to bottom on a consumer laptop.
You will learn to: derive policy gradients and PPO from scratch · design rewards for continuous control and recognise when they are being gamed · diagnose sim-to-real failure · apply domain
randomisation · read and reason about a real PPO implementation.
Who this course is for

---

Robotics engineers who know classical control and want to understand what reinforcement learning adds, and where it genuinely does not belong.
ML engineers and researchers who have done RL on CartPole, Atari or gridworlds and now need continuous torque control on real robot morphologies.
Graduate students and self-taught learners preparing for locomotion, sim-to-real or Isaac Lab work, who want the derivations rather than a library tutorial.
Homepage

---