FELN Liquid: Fine-Tuning a GIS Model on a Mac

I have been experimenting with small language models that translate everyday questions into structured GIS queries. With FELN Liquid, I wanted to explore the training side on the Mac itself. Could I take a compact model, teach it the conventions of a North Sea catalog, and complete the experiment locally?

The first complete run used LiquidAI's LFM2.5-1.2B-Instruct with LoRA through MLX on an Apple M4 Max. Training took 48 minutes and 37 seconds. Including checkpoint evaluation and the final test, the workflow finished in just under an hour. That is a practical turnaround for trying an idea and inspecting what it learned.

The task is familiar from the other FELN projects: turn a request into a JSON plan containing layers, where, and relations. A compact Layers.json catalog accompanies each request, supplying the names, types, aliases, codes, and hints that make the data meaningful.

For example, “Show all wells with water depth > 350 meters” should produce:

{
  "layers": ["Wells"],
  "where": ["\"water_depth\" > 350"],
  "relations": []
}

The measurement belongs to the water_depth field, and its unit is already understood from the catalog. The SQL predicate needs the numeric threshold. There is no spatial join in this example, so the relations array is empty.

The completed experiment used 1,000 synthetic examples, split into 796 for training, 100 for validation, and 104 for testing. Related query families stayed together across those splits. I also froze the source files, prompt, split identifiers, and hashes for the run. As the catalog and generated questions evolve, those copies let us identify exactly what produced a result.

LoRA updated about 11.1 million parameters, roughly 0.95% of the base model, using rank-16 adapters. MLX runs this work on the Mac's Metal GPU. Peak reported MLX memory was about 30.7 GB, so the memory requirement matters even with a relatively small model. The selected adapter is about 44.5 MB and still needs the base weights to run.

The recorded results selected epoch two:

Measurement	Selected adapter
Canonical plan matches, validation	98/100
Canonical plan matches, test	99/104 (95.2%)
Strict FELN schema validity, test	104/104
Identifier and SQL compilation checks, test	104/104
Observed median generation time, test	0.784 seconds

Epoch three matched 97 of the 100 validation plans, so the final checkpoint was not the selected one. The test split was evaluated after selection. Canonical comparison normalizes the SQL representation before comparing plans; the stricter raw-output score was 96/104 on the test set. The timing is an observation from this Mac run, not a controlled, repeated inference benchmark.

Then a short human question exposed a useful problem: “Show all wells with depth > 350 meters.” The original adapter invented a depth column and left meters in the SQL expression. The available field describes water depth. A person's word “depth” could also mean drilled-well depth, which this catalog does not provide.

The guarded inference command now asks for clarification on that wording. For an explicit request, it supplies additional instructions and one example, validates the complete response against the catalog and permitted SQL expressions, and allows one retry with feedback. Its result can be clarification_required, validated, or rejected. It does not execute the query.

That distinction matters. All five errors in the original tuned test set passed the schema and SQL checks. A wrong stored code or an extra condition can still be valid SQL. Likewise, the six targeted checks that passed after adding the guard are development checks, not a new accuracy benchmark. The original scores describe synthetic query families, and execution-result accuracy has not been measured.

I am excited by how much of the training and inspection cycle can happen locally. The next step is to add more human phrasing while reserving independently written questions for evaluation. A matched RTX training comparison is still future work for this project.

The scripts, run notes, and data preparation details are available in feln-liquid. The setup currently uses workspace-specific paths, and the model weights and run artifacts are not included in a clone, so start with the README before reproducing it. Feedback on the questions your own GIS users actually ask would be especially useful. More to come!