Train and deploy

Microduck’s policies are not trained in the robot’s own repository but next door, in microduck_rl — MuJoCo Warp with PPO, exported to ONNX, loaded by the runtime on the robot.

The whole path

Policies are trained at 50 Hz and the onboard control loop runs at 50 Hz. That is not a coincidence: matching both ends is one of the conditions under which timing learned in simulation still holds on hardware.

The shortest route

With a CUDA GPU, four commands:

git clone https://github.com/pollen-robotics/microduck_rl && cd microduck_rl

# ~1–2 h for a usable gait at 4096 environments
uv run train Mjlab-Velocity-Flat-MicroDuck --env.scene.num-envs 4096

uv run scripts/export.py Mjlab-Velocity-Flat-MicroDuck --wandb-run-path <...>

# drive the exported policy in CPU MuJoCo, no robot needed
uv run scripts/infer_policy.py --walking output.onnx

No GPU: add --hf-jobs to any train command and it runs on Hugging Face Jobs.

Running it needs no release

This is easier than it sounds. Put your .onnx on the board and point the config at it:

[policy]
walk = "/home/radxa/my_walking.onnx"

sudo systemctl restart robotd

Those paths survive updates — a release replaces the binaries and policies it ships, not the file that points elsewhere. Delete the lines to go back.

A policy that could not be loaded reports unhealthy, and both robotctl health and the bottom border of robotctl monitor name the reason.

What actually decides sim2real

A policy that walks beautifully in simulation and falls over on hardware is the normal outcome, not an accident. microduck_rl encodes the recipe in the repository rather than leaving it in someone’s head. Four things:

1. The actuator cannot be ideal. BAM’s M6 model for the Dynamixel XL330: voltage control law, back-EMF, Coulomb/Stribeck/load-dependent friction. Treat a servo as an ideal torque source and the learned gait relies on a response it will not get.
2. Randomise physics, not observations. Battery voltage, voltage sag under load, command delay, friction magnitude. This is not the same as adding Gaussian noise to observations.
3. Model the backlash on the correct side. The real encoder sits on the output side of the play, so the simulated observation must read through it. Modelled on the input side, the policy believes it knows the joint position exactly — an error that stays invisible until the joint reverses.
4. One 61-dimensional observation contract, which is what lets walk, recover and trick policies hand the robot to each other at any moment.

What you can train

The main task is Mjlab-Velocity-Flat-MicroDuck. Beyond it: fall recovery, standing up, sitting, ground pick, ball kick, the forward roll, and a full family of roller tasks for the passive wheels.

Every main task also has a Backlash twin trained with ±1° of gear play per joint.

Where to go next

• Set up training — requirements, the four commands, and what to do without a GPU
• The training task list — every task id, its terrain and its purpose
• The sim2real recipe — the four things that decide whether a policy survives the move