Case Study
Open-Jet
Live demoSelf-hosted coding agent for local LLMs. Point it at an Ollama, LM Studio or llama.cpp server you already run, or let it profile your GPU, pick a GGUF that fits, and drive llama-server itself — startup, context sizing, GPU offload, KV-cache save and restore. Scores 59.3 on Terminal-Bench 2.0 with Qwen3.6-27B.
Outcomes
Impact
- Reads files, edits code, and runs commands fully offline. The Claude Code workflow without the cloud.
- Collapses a weekend of backend, runtime, and agent setup into a single install. Shipped on PyPI.
Stack
Tools
Decisions
Key decisions
- Most agentic coding systems still depend on cloud connectivity for orchestration or inference, which breaks in offline and restricted environments.
- Running an OS-level LLM agent directly on self-owned edge hardware enables local device control without sending code, logs, or shell state to external services.
- Security posture improves when execution, model weights, and tool outputs stay local, with explicit approval gates for mutating actions.
Approach
Technical approach
- A setup wizard profiles target hardware, tunes GPU layer offload, and recommends model sizing.
- Inference runs through local llama-server using quantized GGUF models from local files or Ollama pulls.
- The Textual TUI orchestrates chat, slash commands, file mentions, and tool execution requests.
- Session events and resource telemetry are written as structured logs for observability and replay.
Links