AlienWars Gym

AlienWars Gym is a reinforcement learning and game development harness. Coding, design and NPC agents use it as a world model: they explore procedurally generated 3D worlds in simulation and train policies inside the development loop. PufferLib 5 trains the policies on CUDA, and the same C simulation, sensors and trained policies run in the browser through WebAssembly.

Open Map Lab ↗Code on GitHub ↗Sensor lesson ↗Training observatory ↗

Map Lab runs the full simulation in the browser. It can take a while to start.

Temperate AlienWars map with bases, roads and lakes — Temperate map with raised bases, roads, lakes and an extended ocean. Seed 73.

Role: Author
Training: PufferLib 5 on one GPU
Runs: Natively and in the browser
Vehicles: Ground, boats, aircraft and submarines

Worlds

Wave Function Collapse connects terrain tiles, road grades and tunnel profiles. Each seed produces coastlines, hills, beaches, cliffs and lakes, with bridges and mountain passes fitted to the terrain. The ocean floor uses the same geometry that sonar and submarines navigate.

Maps can be symmetric or asymmetric, with bases up to ten floors high, in Temperate, Desert or Frozen palettes.

Asymmetric desert map — Asymmetric desert with different base heights.

Isolated tunnel network — The tunnel network shown without the terrain above it.

Agents

There are twelve vehicles: three ground vehicles, three boats, three aircraft and three submarines. Each has its own hull, speed, turning limits and sensor range. Fixed-wing aircraft hold forward airspeed; the quadcopter can hover and strafe; boats respect draft; submarines travel below the surface.

A* plans global routes and PPO controls local navigation. Five family policies train at the same time in shared worlds, and each vehicle has its own destination and recurrent memory. The browser loads the selected trained checkpoints.

Sensors

Units carry LiDAR, sonar, RF and depth cameras. Equipment changes the policy inputs; overlays only change the display. Generation, collision, navigation and sensing share one C model that does not depend on the renderer, and sensor sampling runs without graphics during training.

Sensor overlays for all units — Range fans, camera fields and radio links for every unit on the map.

Training and results

Twelve vehicles act in the same worlds while five PPO policies learn at once, one per vehicle family. Training ran 49.5 million agent steps across three seeds in 18.5 minutes on a single RTX 4090, using PufferLib 5's native C and CUDA path.

The deployed policies were tested on eight maps they never saw in training, three missions per map. Every controller follows the same A* route; the comparison measures what the learned policy adds on top of a plain route tracker.

Vehicle family	PPO policy	Route tracker
Ground vehicles	56/72 (78%)	49/72
Surface boats	61/72 (85%)	57/72
Quadcopters	19/24 (79%)	24/24
Fixed wings	43/48 (90%)	44/48
Submarines	56/72 (78%)	60/72

The policies beat the route tracker for ground vehicles and boats and fall short for the other families; the 95% arrival target is not met yet. A later candidate with new safety behaviors, tested for another 49.5 million steps, did not pass the reliability bar, so the demo keeps the September 16 policies.

Training run report · Reliability test · Reliability dashboard

Scenes to open

Each link opens Map Lab at a specific seed and view. Map Lab downloads the simulation and trained policies, so the first load takes a while on a laptop.

Video

The harness in use: a coding agent explores the world model and runs reinforcement learning in the loop.