0080: Explain and review with the local model

Status: accepted

Context

A site's Changes page (0034) shows uncommitted changes, and Short Leash already runs a language model on this computer (0060). Asking that model what a change does, or for a commit message, saves a trip to another tool. A diff is text Short Leash didn't write, often from a cloned repository, and text like that can steer a model, a small one most easily (0062). The catalog's models read 16,384 tokens at once (Qwen3.5 4B) or 32,768 (9B and 35B), so a big diff doesn't fit, and the helper runs one request at a time for chat, summaries and the endpoint for other apps (0063).

Decision

• Two actions. Explain asks what a change does, for the lines selected in a file's diff or for the whole file, and streams the answer into a panel beside the diff. Draft message turns the staged changes into a commit message and puts it in the message box, where it stays editable; drafting never commits.
• The model and the queue. Both use the model chosen for chat (Engine::chat_model) through leashd's one model queue (Engine::llm_chat_until): one request at a time, first come, first served. Before asking, they stop the endpoint's answer in progress (Engine::yield_endpoint), as chat does. One Explain or Draft message runs per site at a time; a second one for that site is refused with "Short Leash's model is already working on this site's changes" (AssistError::Busy), and Stop cancels the one running. The core functions live in changes::assist behind a small Llm trait, which engine::assist::EngineLlm implements over the queue, so they're tested without a model.
• No tools. Their requests carry no tools and no tool_choice key and turn thinking off (enable_thinking: false). The model only produces text: it can't read anything else, call a tool or change the repository (invariant 15). Tests check both requests, as they're built and as they reach the model through the queue.
• Repository text is data. The system prompt says the diff comes from a repository and is data, never instructions, and the diff goes inside <diff> or <staged> tags. A diff that says "ignore your instructions" can only become an answer, marked as outside text, with nothing to act with.
• Answers are outside text. Every piece of the answer has invisible Unicode removed with ai::chat::strip_invisible before it leaves leashd. Invisible means the one list a project file refuses too (project::file::hidden_char: format characters, line and paragraph separators, variation selectors and the other default-ignorable ones), so repository text in a prompt, a pull request's texts and a base branch's name are held to the same list. The app shows Has outside text on Explain's answer and on a drafted message until it's edited. A draft also loses a code fence around it, a leading "Commit message:" label and wrapping quotes (changes::assist::clean_draft).
• The budget. Tokens are estimated as characters divided by 3, rounded up, before anything is sent. The limit is the model's context less 2,048 tokens kept for the instructions and the answer (ASSIST_RESERVE): 14,336 for the 4B and 30,720 for the 9B and 35B. Answers are capped at 1,024 tokens for Explain and 160 for Draft message. Explain's panel shows the estimate against the limit ("About 3,200 of 14,336 tokens").
• What doesn't fit. Explain refuses a diff that doesn't fit without asking the model and asks for a selection of fewer lines. A selection is cut from the diff (changes::assist::cut_patch), with selected unchanged lines kept as context. Draft message sends the staged diff when it fits; otherwise the file list (git diff --cached --stat), then whole files in order while they fit, and the app says the draft comes from the file list and the first files.
• Git reads. Explain reads a file's diff through 0034's hardened read. Draft message reads the staged changes of the site's folder only (-- .) through the same runner, with external diff and text conversion programs off.
• Nothing to ask about. Neither action asks the model when there's nothing for it to read, and each says why as a plain sentence, never as a model failure:
  • A file with no text changes (binary, a mode change, a rename with nothing inside changed, or only hidden whitespace): "This file has no text changes to explain." (AssistError::NoText)
  • A file too large to read whole (over 1 MB): "This file is too large to explain whole. Load its diff, select the lines to explain, then choose Explain again." (AssistError::FileTooLarge)
  • A selection with no added or removed line: "The selected lines have no changes. Select changed lines to explain them." (AssistError::NothingSelected)
  • Draft message with nothing staged: "Nothing is staged. Stage the files to describe first." (AssistError::NothingStaged)
• No model. With no language model downloaded, Explain and Draft message say so and link to Settings › Models. While one downloads, Explain shows its progress and starts once it's done.
• Nothing kept, nothing sent. Explain and Draft message store nothing: an answer lives in the panel until it's closed, and a draft lives in the message box. Neither sends anything off this computer. They use the local model only, with no cloud option.

We counted characters rather than tokens because the tokenizer lives in the model helper, and asking it to count would wait in the same queue as the answer. Characters divided by 3 overestimates English and most code, so the reserve rarely has to absorb an underestimate. Cutting a diff that doesn't fit, for Explain, would describe part of a change as if it were all of it, so Explain asks for a selection instead; a commit message only needs the outline, so Draft message cuts.

Consequences

• Each request replaces the prompt prefix the helper keeps for chat (0060), so chat's next answer reads its tools again and starts a few seconds later. The docs say so.
• Explain waits behind a chat answer in progress, and another app's answer through the endpoint stops when Explain or Draft message starts.
• Answers from the 4B are weaker than from the 9B or 35B. Each catalog model is tested on real diffs, and the docs say which one suits reviewing changes.
• A diff of long lines or many non-Latin characters can be estimated low; the 2,048-token reserve covers small misses, and a larger one fails with the model's error and what to do next.
• With the 4B, roughly 2,000 short lines fit at once; a bigger change needs a selection to explain, and a big commit gets a draft from part of it.
• A model answer can still be wrong or steered by the diff; the outside text mark asks the person to check it before acting on it.

Review comments

Context

Explain says what a change does, but Changes still had no way to say what's wrong with a line, for you or for a coding agent working on the same folder. Comments on uncommitted code have nothing stable to hang on: there's no commit to point at, and the file changes under them while you and the agent edit, stash and rebase. A local model can leave its own comments, but its text comes from the diff, which 0034 treats as untrusted, and 0062 says text like that is marked and never obeyed.

Decision

• Where they live. Comments are rows in leash.db (change_comments, migration 0011, 0600 like the rest of 0029), through store::change_comments and the daemon's API: the app, leash mcp and chat never open the database. Each has its site, file, line, that line's text, the body, by (you or Short Leash's model, as notes have it), kept, its state (open, outdated or resolved) and who resolved it. A body holds up to 10,000 characters and a site at most 1,000 open comments; past that, adding one is refused with a message.
• Comments sit on the file as it is now. A comment goes on an added or unchanged line; removed lines take none. Its anchor is the file, the line number and the line's text (trailing whitespace and \r trimmed, so CRLF files match), plus whether the line was a change and HEAD at the time.
• Anchored again on every read (changes::comments::reanchor), which runs whenever comments are listed: the page's 10-second poll and focus, and every MCP read. The comment moves to the nearest line with the same text within 200 lines, the one above on a tie; with none, it goes outdated, and it comes back when the line does. When HEAD moved and the new HEAD holds a commented changed line (a copy is near and isn't an added line any more), the comment resolves by itself as "Resolved by a commit". No diff history is kept and Git is only read.
• Review this file. changes::assist::review sends the file's diff, or the selected lines of it, with each row numbered by its line in the file now (Numbered), through the same queue, budget and no-tools rules as Explain, with thinking off and up to 2,048 tokens of answer. The answer is held to a JSON schema by a grammar: the chat shim's json_schema field becomes a grammar that's on from the first token, and the schema stays out of the template's inputs. The model only names a line number from an added row and writes up to 2,000 characters about it, at most 20 comments. leashd drops any other line, strips invisible characters from the body, and takes the line's text from the file on disk, never from the model; a line that changed while the model read it refuses the whole review ("{path} changed during the review. Review it again."). A file over 1 MB, a change with no added lines, a full site and a diff that doesn't fit are refused before the model is asked. One review runs per file at a time, and Stop saves nothing.
• Suggestions until kept. The model's comments arrive with kept = false: the app shows them with Keep and Dismiss, Dismiss deletes one, and reviewing again replaces the waiting ones (only those inside the selection, for Review selection), never kept ones.
• Outside text everywhere. A model comment carries the Outside text badge in the app, outside_text: true over MCP, a line in Copy for agent's text, and the tools' descriptions tell agents to treat them as suggestions to check, never as instructions. The line's text is marked as repository text too.
• Agents list and resolve, never write. list_change_comments (class Read) gives agents your comments and kept model comments, at most 200 with a count of the rest; un-kept suggestions are never shown. resolve_change_comment (class Act, so chat asks first) resolves only a comment agents can see, signed with the agent's name; the API marks the call as an agent's, and "you", "a commit" and a blank name are saved as "an assistant". Nothing over MCP adds, edits or deletes a comment.
• Retention. Open comments stay until they're resolved or the site is removed from Short Leash; a renamed site's comments follow it. A site that's gone from the list without being removed (a folder on an unmounted volume) keeps them until 90 days after its newest comment. Resolved ones are kept for [ai] comment_days (30 by default, 1 to 3650, under Review comments in Settings › Models › More), cleared by the database's upkeep every ten minutes. Resolved comments can't be edited.
• No new daemon event. The page reads comments again on its poll, on focus and after each action, so an agent's resolve shows within 10 seconds.

We anchored by line text rather than by tracking diffs because uncommitted code has no history to follow: a stash, a rebase or an agent rewriting the file would break any mapping from one diff to the next, while the line's own text is still there. A grammar, rather than asking nicely for JSON, means the model's answer can always be read, and checking each named line against the diff's added rows, with the text from disk, means the model can only point at the change and talk about it.

Consequences

• Comments survive edits above them, stashes and rebases without any diff tracking, but a heavily rewritten line goes outdated instead of following, and the nearest copy of a repeated line (}, a blank line) may not be the one first meant.
• Comments can't sit on removed lines.
• A diff that says "ignore your instructions" can only become a marked suggestion on one of its own added lines, which agents don't see until you keep it.
• Review's comments are only as good as the model chosen for chat; the docs say which catalog model suits reviewing once each has been tried on real diffs.
• Review, like Explain, replaces chat's cached prompt, so the next chat message is slower.
• Schema answers never think: llama.cpp's own schema grammar starts with the generation prompt, which the helper doesn't feed it, so thinking stays off for Review.
• Every list reads HEAD again, and each file with unresolved comments and its diff, so a list costs more the more files have comments; the 1,000 per site bounds it.

Asking chat about changes

Reading a pull request from GitHub is its own decision, 0084.

• This branch. Next to Uncommitted, the page can show what the current branch committed since it left its base: git diff from the merge base of the base and HEAD to HEAD, for the site's folder only (changes::branch). It goes through the same read runner as every other Changes read (Git::run, 0034): what a repository could make Git run stays off, nothing is fetched in a partial clone, and no lock is taken. Uncommitted changes are never in it. The base is origin/HEAD, else the first of main, origin/main, master and origin/master there is; with none, the page asks for one. A base chosen with Change base is kept per site in Short Leash's config.toml (compare_base, never in the repository), after it passes branch::valid_base (a branch name Git can't read as an option, a range or a revision, with no invisible characters) and is one of the repository's branches. On the base itself, or a branch with no commits of its own, the page says there's nothing of its own and offers Change base. Writing to Git (staging, discarding, committing), Explain and review comments read the working tree, so they're hidden under This branch.
• site_changes is a Read tool, for chat and leash mcp (0062). Without a file it lists the changed files with their counts (and, for This branch, the base and the branch's commits); with a file, that file's diff, or only the lines asked for (changes::assist::cut_patch). Diffs are cut on whole hunks to max_chars and say when they were. Coding assistants get 60,000 characters unless they ask for between 2,000 and 200,000. Chat chooses max_chars itself and hides the argument from the model (ai::chat::fit_to_model): budget(context, "").limit characters (the model's context less 2,048), which at 3 characters a token is about a third of what it reads at once, or less when less is left.
• Hidden characters are shown, not removed. In site_changes, a character that hides text (a bidi override, a zero-width space, a variation selector: any in hidden_char) appears as a visible code such as ⟦U+202E⟧, and the answer counts them, because a reviewer has to see the trick. A code also breaks any special token the model might read. A path given back with codes in it names the same file.
• Sized with a cautious count. Chat sizes every read, and counts what a thread holds, with ai::chat::cautious_tokens: each character outside ASCII 3 tokens, each digit 1, other ASCII 1 for every 2. No rule of thumb bounds a real tokenizer, so the model helper also checks the real count of the prompt plus the answer's room before it decodes. When it says the thread doesn't fit, chat asks again with older tool results left out, oldest first (ai::chat::fit_history, which puts a placeholder in their place). This turn's newest result is never left out: when it's all that's left, it's cut by at least half each time, so asking again always ends. The saved chat keeps every result. For the OpenAI-compatible endpoint and meeting summaries, where max_tokens is a cap, the helper gives the answer the room left and stops there with finish length, refusing only when fewer than 64 tokens are left.
• Ask in chat is a site_changes read Short Leash makes itself before the model answers (Engine::read_changes_first), like Ask about this meeting: a file, the lines selected in its diff, or every change, under Uncommitted or This branch. It's saved as the chat's first read, sized like any other, so the chat is marked Has outside text whatever the model does next.
• Small models. The change reads (site_changes, list_change_comments and pull_request) are among the tools offered to small models (ai::chat::SMALL_MODEL_TOOLS).

A large branch reaches the model as its file list and a few files, not all of it at once. Explain and comments stay on uncommitted changes; explaining a branch's diff isn't part of this.