In 2021 I worked on a multi-agent reinforcement learning system for tactical deconfliction: keeping drones separated after they were airborne and their planned routes had stopped being useful.

This is a retrospective engineering note, not a reproducible benchmark. The original flight logs and training artifacts are not public, so the measurements below are observations retained from our project notes rather than independently verifiable results.

We trained centrally and executed on each aircraft. Every agent ran the same policy, consumed signed position broadcasts from nearby aircraft, and optimized for separation and energy use in three dimensions.

Why we tried a shared policy

Two aircraft in a corridor can be handled with a small set of explicit rules. As density grows, so does the number of interactions that have to be anticipated, tested, and kept mutually consistent. We wanted one policy that could be trained across different traffic densities instead of maintaining a growing catalogue of pairwise cases.

The shared policy simplified training and deployment, but it did not remove the trust problem. In our implementation, a position message affected another aircraft's decision, so messages were signed and verified before the policy consumed them. We could not assume that every participant in shared airspace would run hardware or software we controlled.

What the simulator left out

The policy performed well in the simulator. Flight testing exposed how much of that result depended on a narrow environment model.

Wind appeared in the simulator as a parameter. In flight it changed across time and position.
In the test runs recorded in our notes, energy use under load and gusting conditions was roughly 20–30% above the specification-derived model.
Position-broadcast latency was deterministic in simulation. The network used in flight had a longer and less predictable tail.

Together, those differences put the aircraft into states that were uncommon in training. The training result did not tell us how the policy would behave there.

Where the work moved

After the first field tests, the important questions were less about the MARL algorithm and more about the environment distributions: how wind should vary, how much latency jitter to include, and which energy model matched the aircraft outside steady flight.

The simulator had started as training infrastructure. It became the main engineering artifact because every claim about the policy depended on what the simulator represented—and what it omitted.