Thunderhead Explorer

I want to point my Mac camera at a scene, click an object, and see a mask stay on that object while it moves. I do not want to train a model for each object. I also want to run the model on my own hardware, not on a hosted API. Segment Anything models can do this. You select an object one time, with a point, a box or a name. The model then finds the same object in each new frame and marks its pixels. This marked area is the mask. The project is TrackAnything . It runs three trackers (EfficientTAM, SAM 2.1 and SAM 3) on an NVIDIA DGX Spark. The Mac only shows the camera and the result. In this post, I show how the pieces fit together, what the numbers say, and where the method is wrong. Vehicles on a motorway, tracked with EfficientTAM-S. The detector runs again every 15 frames, so new vehicles get a mask. Video: "Motorway A40 - on bridge above the traffic" by Kathinka Engels / Sounds of Changes, CC BY 3.0 , via Wikimedia Commons. Changes: audio removed, trimmed to 10 se...

I use Claude Code for GIS work. When I ask it for a map, I get a URL or a file path. I want to see the map. So I built a plugin that draws it in the terminal. The idea came from claude-inline-images by kbrianps. It draws PNG files that Claude reads. I wanted the same result for map services. The plugin is inline-maps . What it is Claude Code loads the plugin at session start. Claude Code calls this type of plugin a "mod". The mod API is early access. You need Claude Code 2.1.287 or later. The plugin adds two things: A tool, show_map . Claude calls it when you ask for a map. A command, /map . You type it at the prompt. Both use the same function. Only the way they return the result is different. Which map services it reads The plugin reads the URL and identifies the type of map service. Then it makes one request for a map image, or one request for each tile. It tests the patterns in the order of the table. Map service URL pattern Request ...

In my last post, Where Is Jev? , I found storage tanks in an overhead image. I asked a vision-language model one fixed question about each tile, and SAM 3 found the tanks in the tiles that passed. This time, I want to try the opposite approach. I do not ask a question, and I do not write a prompt. I point at one tank, and the app finds the chips that look like it. The tool is Clay Embedding Explorer , a small web app that I built around the Clay v1.5 foundation model. It shows chip embeddings on a map, over the source image. In this post, I explain what an embedding is here, how the pipeline works, and what the tank numbers say. One of the numbers looked bad at first, and the reason was more interesting than the number. The image and the labels I use the same image as last time: one RGB GeoTIFF, 8,200 × 6,087 pixels at 0.5 m per pixel, in EPSG:3857. Web Mercator stretches distances, so the true ground size of one pixel at this latitude is 0.457 m. A FileGDB has 97 hand-drawn tan...

Remember Where's Waldo? You do not look at the whole page at once. You scan it left to right, top to bottom, one cell at a time. When you find the cell with Waldo in it, you look closely and spot him. I wanted to try that idea on an overhead image of an industrial site with a lot of storage tanks. In my last post , a Jev-style decision model answered a plain-English condition for each row of a table. Here, the rows are image tiles. I cut the image into tiles and ask a vision-language model one fixed question about each tile: does it contain a tank, part of a tank, or no tank? If the answer is no tank, I skip the tile. If not, the tile goes to SAM 3 , and SAM 3 finds the tanks. No model is trained for this task. The project is where-is-jev . In this post, I show how the pieces fit together, what the numbers say, and where the method is wrong. The data The test image is one RGB GeoTIFF: 8,200 × 6,087 pixels at 0.5 m per pixel, in EPSG:3857. A FileGDB has 97 tank polygons (th...

When DuckDB published Jev and DuckDB: Plain-English Conditions in SQL , I wanted to try it on my North Sea wells and pipelines right away. The idea is simple and powerful. You write a plain-English condition inside a SQL query, and a decision model answers it for each row. But those Jev functions send the row data to the hosted TypeSafe API, and that needs an API key. I already had Laya, an open-weight decision model, running on my Mac from Lunar Laya . So I asked a direct question: can the same SQL run on a local model, inside DuckDB, with no network at all? The result is duckdb-jev-spatial . It is a native DuckDB extension, written in Rust, that registers seven Jev functions. Laya Rust and ONNX Runtime run the model inside the DuckDB process. The DuckDB spatial extension does the geometry. In this post, I show how the pieces fit together, what the NorthSea data shows, and where the model is wrong. Two kinds of conditions A model that reads one well row cannot find the pipelin...

One photo, one sentence. “Locate the Sydney Opera House”, and a few seconds later the scene is in 3D, as Gaussian splats you can tilt around. Recently I came across a small project called Odyseus Spatial VLM by Mercurius Technologies. Two things grabbed me right away. The first was the name. I grew up on Greek mythology, and seeing Odysseus on a GitHub page brought back a lot of childhood memories. The second was the idea: take a vision language model (VLM), the kind of model that can look at a picture and answer questions about it, and ask it not just what is in the photo, but where it is, in 3D, in meters. I liked it so much that I forked it, played with it for a while, and kept adding things. It grew into its own project, which I named Apollo Spatial VLM . (Why Apollo? Mercury and Apollo have a lovely story together. I tell it at the end of the README.) How it works You upload a photo and type what you are looking for, someth...