0063: An opt-in OpenAI-compatible endpoint
On this page
Status: accepted
Context
Editors and scripts on the same computer often speak the OpenAI API. Letting them use Short Leash's model saves a second download and a second runtime. Local model servers have a history of serious bugs: unauthenticated access, path traversal through model fields, DNS rebinding from web pages, and memory errors from unchecked parameters.
Decision
- Off by default. Settings › Models › More › Let other apps use the model serves
/v1/chat/completions,/v1/completionsand/v1/modelson127.0.0.1atdefaults::AI_API_PORT, next to the dump and OpenTelemetry ports. If the port is busy, Short Leash names the process holding it rather than picking another. - The key. It is at least 256 bits from the operating system's random source, starts with
dk-local-, and is stored insecrets/ai.jsonat 0600. It is shown only through its own route and can be replaced. It is compared in constant time, checked before the body is read, required on every route and never accepted in a query string. - Refused: a
Hostother than127.0.0.1orlocalhostwith the port; any request with anOriginorSec-Fetch-*header; unknown fields; numbers out of range. No CORS headers are sent. - Limited:
- No route or field names a file, a digest or a model path, and there are no routes that manage models.
- Body and header sizes are capped. Slow or idle clients time out, and generation stops when the client disconnects.
- One request runs at a time, below chat in priority, and the rest get 429.
- Only the model already loaded is served, and Short Leash's tools are never exposed.
- While the model loads the endpoint answers 503 "The model is loading. Try again in a moment." (0060).
Consequences
- Invariant 2 holds: the listener is loopback only.
- Another app can use the model only with the key the user copied into it, and a web page in a browser can't use it at all.
- Apps written for other local servers need Short Leash's port and key; the docs list both.