blog
Short-form notes. Things I'm thinking about, things I'm reading, occasional opinions. 6 entries.
✎ A small note on agent context doctor
Agent context doctor is for the mess that piles up in agent instruction files when teams keep adding rules and forget what changed.
✎ local-engine-router v0.5.0: pip install, a curl bootstrap, and a setup wizard
v0.5.0 closes the gap between cloning the repo and having a running router. It is now on PyPI as local-engine-router, there is a curl-pipe install.sh that builds a venv and sets up the systemd user service, and a new init wizard that port-probes your local engines and scaffolds a working config.yaml so you no longer hand-write YAML to start.
✎ here we go
I have been building a handful of projects quietly for a while. Some of them just went open source, and from now on I am going to post as I work instead of going silent.
✎ local-engine-router: one endpoint for a fleet of local inference engines
On a unified-memory machine only one heavy model fits at a time. local-engine-router fronts vLLM, llama.cpp, Ollama and friends behind a single endpoint and swaps the GPU to the engine that owns the requested model, waiting for the kernel to actually reclaim freed memory before the next model loads.
✎ llmtop: an nvtop for local LLM inference
I usually have several inference engines running at once on my workstation: a vLLM server here, an Ollama daemon there, sometimes a llama.cpp build and a small router in front of…
✎ netpeek
A small terminal tool that shows which programs on your machine are actually using the network, in real time and completely offline. Open source soon.