What sim2real is, and why it is the hard step

sim2real is the move from a policy trained in simulation to one running on real hardware.

It is a discipline of its own because a policy that performs beautifully in simulation and falls over on the robot is the normal outcome, not an accident.

Where the gap comes from

What a policy learned is: given these observations, emit these joint commands, and that result follows. Break any link in that chain on hardware and the policy fails.

Link	Simulation’s simplification	What hardware does
Actuator	Ideal torque source, command equals output	Friction, back-EMF, voltage sag
Transmission	Rigid	Backlash — free play on reversal
Sensing	Exact joint angles	The encoder sits after the play
Timing	Commands take effect at once	Communication and compute delay
Power	Constant	The battery drains, and torque with it

Each is a small error alone. Together they are enough to defeat a gait that learned to exploit ground reaction forces precisely.

What microduck_rl does about it

The recipe is encoded in the repository rather than left in someone’s head. Four things:

1. The BAM actuator model — the servo is not an ideal torque source
2. Domain randomisation — of physics, not of observations
3. Backlash modelling — on the correct side of the encoder
4. A 61-dimensional observation contract — so policies can hand over at any moment

How to measure the transfer

scripts/infer_policy.py takes --debug, --save-csv and --record for exactly this: run the same command sequence in simulation and on the robot, then compare per-joint traces.

From the robot’s side:

robotctl monitor --json --hz 50 > run.jsonl