Building a surgical thread-insertion robot in simulation

Two things made this possible: a harness and a brief. The harness was in place before this project began. It is a repository set up for simulation and learning work: MuJoCo for physics, PufferLib 5.0 for reinforcement learning, a locked Python environment, Git LFS for assets, a commit-based workflow between a laptop and a remote RTX 4090 workstation, earlier MuJoCo projects to build on, and written engineering rules for how simulations are modelled, tested, recorded and documented here. The brief states what to build and the gates the work must meet.

Everything specific to this robot was written by two coding agents working together inside that harness from the brief, one running GPT-6 Astra and one running Claude Opus 5.5. That includes the workcell and needle model, the unit rescaling, the C++ thread model and contact kernel, the C task core shared by training, tests, evaluation and replay, the disturbance layer, the PufferLib environment, the evaluation tools, and this page with its WebAssembly replay. The project's first commit was on October 5.

We call this improvement loop AutoResearch. The agents plan a step, implement it, run it, measure the result against the brief's gates, record what failed, and change one variable before the next attempt. The training runs in this journal show the pattern: a diverged run exposed a reward bug, two runs that blew up led to lower learning rates, and a stable but slow run led to a longer training budget. A person sets the brief, reviews the results and corrects course. Their corrections were decisive: train on the GPU workstation rather than fall back to a CPU, make the gates serve the mission rather than the reverse, and pursue robustness under disturbances. Their checks of this page in a browser found four display bugs.

"Self-improving" means the work improves, not the agents. The models behind them do not change. The code, the physics, the policies and the process get better through measured iterations, and every mistake stays on the record.

We built the robot as a static scene before anything moved: an XYZ gantry, a downward insertion slide with a needle, a mounted microscope, and a domed tissue phantom with six targets and vessel markings. Clearance and limits were checked at 120 sampled poses before any dynamics. The first tool also had a thread retainer slide and a cassette of thread samples; the alignment experiments of chapters 7–14 use it. The insertion tool replaces them with a thread tube beside the needle (chapter 15).

A 44 mm thread of 40 µm diameter, split into 32 segments, has rotational inertias near 10⁻¹⁹ kg m². MuJoCo rejects them at compilation. Rather than inflate the mass or radius, we rescaled the engine's units: millimetres and grams, with seconds unchanged. Every dimensional quantity converts at the model boundary, and results are reported back in SI.

The physical mass and dimensions are unchanged, and bending then matched the analytical cantilever within 2.4%. The same conversion applies to every scene we build, so the robot and the thread share one unit system.

We ported the published discrete-elastic-rod (DER) plugin for MuJoCo to version 3.12 as a separate library, leaving the engine untouched. An energy-gradient audit found that the published force mapping produced zero restoring torque for a purely twisted rod despite positive twist energy. We kept the original variant and added a corrected one that projects the same Cartesian forces through MuJoCo's own Jacobians, plus the material-frame end moments. Its forces match the energy gradient to 2.85 × 10⁻⁹.

Bending stayed within 2.4% of the analytical cantilever, an undamped bent rod relaxed with energy drift falling from 0.075% to 0.038% as the step halved, and forces were identical after copy, restore and reset. For sim-to-real this matters more than any single benchmark: a rod whose twist cannot push back would mislead every later thread-handling result.

MuJoCo's soft contact computes a reference acceleration a_ref = −B·v − K·d(r)·pos with an impedance d(r). With d held constant, as in our first law, the damping force switches on as a step at zero penetration, so a grazing contact receives a finite impulse. A penetration ramp makes the force continuous in the state. And because the default integrator is first order through a contact lasting a few tens of microseconds, we step the thread with MuJoCo's RK4, which evaluates collision and the constraint solve at four stages. Both are standard MuJoCo settings; the engine is unmodified.

Before relying on it we checked the engine against an independent calculation that never calls MuJoCo: same-state contact forces match a closed-form solution to 10⁻¹¹, and agree across unit systems to 10⁻⁹.

Each phase of the mission touches the thread in its own way, at millimetres per second. We built one fixture per phase, with a rigid 150 µm probe driven only by force-limited actuators, and required the thread's centreline to agree within 1 µm when the timestep is halved and within 0.01 µm across unit systems, with penetration checked at every step over every contact.

Every fixture passes at a 5 µs RK4 step with a 20 µs contact time constant, for both material presets. Release failed at 10 µs (1.58 µm), because its short fall lands faster than settling does; we refined the step rather than relax the gate, and kept the failure on record.

A programmed servo adds inertial, gravity, viscous and Coulomb feedforward and a bounded integral term to each force-limited position actuator, by offsetting its command. Two design facts surfaced on the first run: Z reaches only 8 mm down, so the needle slide reaches the surface; and both fine slides hang against gravity at their retracted stops, so they are carried 0.5 mm out.

One C core (native/surgical_core.h) holds physics stepping, servo, observations and rewards. It is compiled into the PufferLib 5.0 adapter for training and into a local library for tests and evaluation, so they cannot drift apart. Physics runs at 1 ms; the policy acts at 50 Hz (20 physics steps per action).

Each output is clipped to [−1, 1] and cubed, then scaled to needle-tip velocity: 40 mm/s in X and Y, 20 mm/s vertically. The cube gives 40 µm/s resolution at 0.1. The command drives the X and Y stages and the insertion slide; a 10 ms filter smooths the reference, which the programmed servo tracks.

The logarithmic potential makes going from 10 mm to 1 mm worth the same as going from 10 µm to 1 µm, so the policy gets useful signal at every scale. There is no curriculum: every episode is the full task.

The learned policy is a third faster and passes low over vessels less often, but settles closer to the tolerance than the scripted controller: an episode ends as soon as the 0.3 s hold is met. Compare both in the replay at the top.

With exact state, no delay and a still target, both the scripted controller and the learned policy succeed every time. A real stage gets none of that. We rewrote the brief: every mission phase is learned under one shared, documented disturbance layer; the programmed servo stays only as the motor drive; scripted controllers become yardsticks. Results are reported on held-out seeds at stated disturbance levels, including full strength.

The layer lives in the same C core used for training, evaluation and replay. One number, the level from 0 to 1, scales every magnitude, and in training each episode draws its own level uniformly from that range. The magnitudes are illustrative until replaced by measurements of the real stage, sensors and tissue.

The replay at the top draws every component in the scene. Each carriage carries force arrows at 5 cm per 0.1 N: amber for noise, cyan for friction, lilac for vibration. The table shows its acceleration and recent trace. The Micro camera shows, at true scale, the moving target with its 10 µm tolerance cylinder, the noisy target estimate and the late tip measurement. Values sit in panels beside each object. Airflow is not modelled yet; it matters for the thread, which this task does not carry.

This workload is CPU-bound. PufferLib steps 28 copies of the MuJoCo world on CPU threads, about 29 of the machine's 32 hardware threads, while the GPU runs only the 101k-parameter network and its PPO update. It idles near 10% utilisation and about 59 W of its 450 W limit, with under 1 GB of memory for the trainer. Runs 1–3 predate the resource logger, so their figures come from PufferLib's own dashboard, which reports device-wide GPU memory including the desktop. From run 4, a logger samples GPU power, temperature, the trainer's GPU memory and host CPU load once a minute.

Picking up a 40 µm thread is the hard mechanical part of this machine. We tried it three ways: a slotted needle with a latch over a 44 mm free thread (about 600 times slower than real time, and the eyelet rattled loose on landing); a needle with a ledge and a rotary pincher; then a cartridge with a cannula and a spring latch under the thread's loop. Each attempt was watched on video, and each taught something, but none of it was what this project sets out to prove.

After reviewing photos of a current insertion robot, the user set the goal: prove the robot's movements and abstract the thread handling. The thread waits in a tube on an arm beside the needle, angled 15° from vertical, its end on the needle's path. On its way down the needle sticks to that end, carries the thread into the tissue, and lets go at depth. The rest is physical: the robot's dynamics, the thread as an elastic rod, the tube's walls, the puncture and the tissue's grip.

The first thread ran at a 5 µs step: light, stiff contacts between the thread, the needle and the holding parts settle in microseconds. With the bond replacing those contacts, we coarsened the thread the way rope simulations do: a 50 µs RK4 step, contacts that settle in 200 µs, a looser solver, and added joint inertia and damping that slow the thread's fastest wiggles without changing its shape or sag.

A run places threads at sites 0, 5 and 1. After each site the tool lifts 8 mm, clear of the 6 mm of thread left standing; a spare thread appears in the tube; the tool moves over the next site, measures the target, makes one correction from the measured needle tip, and inserts. Placed threads stay simulated and gripped by the tissue, and the run checks that the tool's moves do not disturb them.

The insertion runs use the disturbance layer of chapter 11 at scale v2: force noise, extra friction and table vibration on every slide; a late, noisy needle-tip measurement; a target estimate with noise, bias and drift; and breathing and pulse. In alignment only the target point moved. Here the phantom itself is moved, and its surface, the targets, the puncture model and the grip on placed threads move with it. The replay at the top draws all of it: force arrows on the slides, the moving target's path, the estimate and the measurement while the robot measures.

Ten evaluation seeds (1001–1010) at levels 1 and 2, and one undisturbed run, three sites each. Placement is the distance from the thread's end to the target ring's centre. A learned insertion policy will be compared on these same seeds and levels.

The full simulation is the reference, but at about 20 s of computing per simulated second it cannot supply the hundreds of millions of steps that PPO needs. The training environment is a C core, insert_core.h, built on the alignment core: the same robot, servo, disturbance layer and needle–tissue force model, at a 1 ms step, with 2,048 copies stepped on 28 CPU threads. It does not simulate the thread. It models the two things about the thread that decide where it lands, both measured in the full simulation: where the thread's end waits relative to the needle's axis, and how far it jumps sideways when the needle lets go.

Run 1 learned within minutes to approach before striking, and showed a gap in the environment: after the stroke started, the stages still took velocity commands, which on the real machine would drag the needle sideways through tissue. Run 2 holds the stages still from the start of the stroke and times the stroke exactly as the full cycle does. It ran to 300 M steps, and its success on training episodes climbed from zero to 72%.

Two yardsticks frame the result. The approved cycle aims the needle at the target, as chapter 19 did. A second scripted controller, written for this comparison, reads the measured thread-end offset and aims the thread's end instead; it is the strongest hand-written controller we have. At nominal disturbances the learned policy places nearly twice as many threads within 10 µm as the approved cycle (20 against 11 of 30) and cuts the median from 23 to 8 µm; at the stress level the medians are 12 and 25 µm. Against the compensating script it is level in the full simulation at nominal disturbances, ahead in the training environment, and behind in the full simulation at the stress level.

Following each thread's end from the moment the policy strikes separates what the policy controls from what it does not. The policy's part is the aim: when the stroke starts, the thread's end is a median 6 µm from the target. The programmed stroke carries it into the tissue and moves it about 3 µm. The release usually adds another 3 µm, but now and then the end jumps: 8 of the 63 threads moved more than 15 µm sideways when the needle let go, and those set the 90th percentile.

Materials and tissue parameters are presets, not measurements, and the controllers read simulator measurements rather than camera images.