An update to my post on DHH’s Omarchy 4.6 roadmap, where “out-of-the-box local AI” was on the list. Now we know how far it goes. “Omarchy 4.6 is not just leaning into local AI, but will let you share any model running on any Omarchy machine on your tailnet with another. Get a DGX Spark, then run inference from your XPS13 netbook,” DHH wrote on X, Omarchy Central reports.
What’s new
- Share:
omarchy-local-aiPR #7 addsshare <recipe>, which puts the model’s loopback endpoint behindtailscale serveat/local-ai/<recipe>. It’s tailnet-only with Tailscale TLS, and a key-freemodels.jsonlists what’s shared. - Keys stay put: Anyone can see a shared model, but using one needs the host’s key.
pair <host>fetches it over SSH (“the only way a key leaves a machine”) and stores it mode 0600. You can now pair and forget one model at a time. - No remote power button: Paired machines get Disconnect instead of start/stop. You use what the host is already running.
- Usage count: A local share proxy counts how many machines used your model in the last five minutes, and the panel shows it on a Tailscale icon.
- Agents: Engine 0.6.3 exposes shared models to pi under an
omarchy-localprovider and addsoverlay qwenfor Qwen Code. Engine 0.6.0 can also split a model across two DGX Sparks over ConnectX-7, with GLM-5.3 Flash as the first recipe.
Limits so far: the share button and user-count tooltip haven’t been tested on a real local model, and a headless sharing host “needs work” because the units are tied to the graphical session. No release date yet.
My take
This is the setup I actually want: one beefy box doing inference and every laptop in the house using it, without anything touching the public internet. Before you turn on sharing, check what your machine already serves on your tailnet. I wrote a quick how-to for that in this batch.
God Bless, and Use the Source!
