0061: Bring your own model

On this page

Status: accepted

Context

Short Leash recommends one model per job for each kind of machine, but some people already have models they trust or want one the catalog doesn't carry. Each way of adding one has its own risks: a file Short Leash never saw, a repository that can change after the fact, or a "local" server that forwards to the cloud.

Decision

  • A GGUF file is referenced where it is, after its header is checked. A known model family gets Short Leash's own chat template by default; the file's embedded template is shown and opt-in. Removing it takes it out of Short Leash and leaves the file.
  • A file from Hugging Face:
    • Only from ungated repositories, so no token is stored (invariant 9).
    • Short Leash resolves the branch to a full commit, uses the file's lfs.oid as its SHA-256, refuses files with no LFS entry, and saves the file as <sha256>.gguf under Short Leash's models folder rather than under its repository path.
    • Redirects to any https host are followed, and a resumed download asks for a fresh link.
    • Short Leash never describes such a file as scanned or safe.
  • A local server: an OpenAI-compatible endpoint at a literal loopback address or localhost, with no redirects. Short Leash's tools are turned off for it in chat, because a local server may forward to the cloud.
  • Models not from the catalog are marked "Not tested by Short Leash", run sandboxed (0060), and are marked "Failed to load" after two crashes.
  • Updates. A catalog model shows Update available when the signed manifest pins a newer version. The user starts the update, and the old file is removed only once the new one is verified.

Consequences

  • People can use models Short Leash doesn't vet, and the UI says plainly which ones those are.
  • A Hugging Face hash comes from the same site as the file, so it proves the download is intact, not that the file is good.