Lunar Laya: From State and Text to Decisions on a Mac

I keep coming back to Lunar Lander. It gives me a very visible way to ask a simple question: what does a model actually decide? Turn too much, burn too late, or miss the pad, and the result is right there on the screen.

After exploring learning and control approaches to this problem, I wanted to try another angle: give a small model the current state and a text instruction, then ask it for a decision that the application can execute and measure. That experiment is now public as Lunar Laya, an Atari-inspired lander using Laya with local inference on Apple MLX.

I believe the input should be a state and a text, even a structured text. The output should always be structured and very deterministic, so it can be controlled and measured. This implementation is a representation of that idea. The fact that the model can be small enough to run locally in an MLX environment is proof that it can be done for this kind of bounded decision task.

An actual trained-model flight using FP16 MLX inference, shown at 4× simulation speed. The input includes explicit guidance requests; no post-inference overrides were applied. The smaller Q6 version was evaluated separately.

A small decision with a clear contract

The simulation exposes position, velocity, tilt, fuel and the landing target. A feedback controller calculates the requested correction. Those values and requests are serialized into text, and Laya answers two choice questions: rotate left, hold or right; and set the engine to off, half or full. Here, Laya is the decision model, not the LayaAir game engine.

The model scores the allowed choices. The runtime selects the highest-scoring option and returns a structured answer with probabilities. For example, the choices inside an answer can look like this, with probabilities omitted for readability:

{
  "rotation": {"choice": "hold"},
  "engine": {"choice": "half"}
}

The application maps those choices to a fixed command: zero rotation and 50% thrust. I can record the proposed command, what was executed, the next state and how long the decision took. That gives me something to inspect when a landing fails, and something precise to compare after changing the model.

That is what I mean by a deterministic interface: a fixed set of outputs, an explicit selection rule and a predictable mapping into application behavior. Neural scores can still change with precision or hardware. A well-defined output contract makes those changes testable; it does not make every prediction correct.

Training it, then making it smaller

I added supervised training on three CUDA GPUs, keeping the original checkpoint available. Local inference uses MLX on the Mac; Python runs the simulator and guidance. A VanillaJS application lets us replay the recorded flights, inspect probabilities and compare controllers. It displays the experiments rather than running the model in the browser.

Now the best part: the compact 6-bit version passed all 1,928 held-out validation examples and landed all 30 evaluated flights across three pads. Its 11,147 decisions agreed with the guidance requests, and its recorded state transitions matched the trained FP16 evaluation exactly.

On an Apple M4 Max, peak active MLX inference memory dropped from about 872 MiB to 370 MiB, a 57.6% reduction. The weight file went from 614 MiB to 249 MiB. Those are model-runtime measurements: total process peak memory was about 709 MiB for Q6, including the surrounding software.

The compact runtime loads packed MLX weights, removes an unused classifier and evaluates the two control questions sequentially. It also disables retained allocator caching to favor memory use. There is a tradeoff: median decision time increased from about 10.1 ms to 13.5 ms in this comparison.

I tried smaller precisions too. Four-bit missed 215 validation answers; five-bit missed two. Six-bit was the smallest configuration tested that passed every validation answer. A smaller file is only useful if the behavior survives the change! The optimization notes include the measurements and reproduction commands.

What this experiment demonstrates

The requested rotation and engine labels are already in the input. The trained model is learning to follow those requests. The feedback controller supplies the navigation knowledge, so the successful landings demonstrate guidance following, not independent discovery of a flight policy. The repository also includes a baseline that executes guidance directly, which is sufficient for this simulator.

For me, the interesting result is the working connection between state, text and a bounded action. I want that same discipline when a model selects a GIS operation or chooses the next step in a data workflow: define what it can return, validate the result, and measure what happens. The lander makes that idea easy to watch.

The next experiment I would like to try is varying the instruction while keeping the output contract fixed, then measuring where the behavior holds and where it breaks. For now, the source code, training details and animation instructions are available. Take a look, try it on your Mac, and let me know where you would use a state-plus-text decision interface. More to come. :-)