0057: Voice runs in a helper inside the app

On this page

Status: accepted. It follows 0039's shape for Pair.

Context

Dictation listens for a key gesture anywhere on the system, records the microphone, turns speech into text with a local model and pastes it where the cursor is. Meeting notes record the microphone and the other side of a call. On macOS each of these needs a privacy grant: Microphone, Accessibility (and Input Monitoring if it's turned off), and System Audio Recording. macOS credits a grant to the app responsible for the process that asks. leashd runs under launchd, outside the user's session, so it can't hold these grants and must never listen.

Decision

  • A helper the app owns. leash-voice is a sidecar that Short Leash.app starts while it runs and dictation or meeting notes are on. The app spawns it directly and never disclaims responsibility, so macOS credits every grant to Short Leash.app. The helper reads a stdin pipe from the app and exits when the pipe closes, so a crashed app never leaves a live event tap behind.
  • What it holds:
    • The key gesture: a CGEventTap on macOS, a low-level keyboard hook on Windows.
    • The microphone.
    • System audio: Core Audio taps from macOS 14.4, process loopback on Windows.
    • The speech model, and posting the paste.
  • What leashd holds: state, settings and stored transcripts. The two talk over /v1/voice/helper, like Pair's helper.
  • Recording starts on the second tap of the double-tap, so a stray tap never lights the microphone indicator.
  • Rules the helper keeps:
    • It listens only after the gesture or after the user chooses Take notes, and a lamp shows while it listens.
    • It writes no audio to disk.
    • It never posts keys into secure input (macOS), password fields (Windows) or a window running at a higher integrity level; it copies the text and says so instead. When Windows can't tell the window's integrity level, it counts as higher. A password field is found through UI Automation; when UI Automation itself can't start, the field counts as an ordinary one, since copying every dictation would make dictation useless on that PC.

Consequences

  • Dictation works whenever Short Leash runs, in the menu bar too, and stops when it quits. Setup offers "Open Short Leash when you log in".
  • A helper crash takes down dictation, not the app or leashd, and the app restarts it (at most three times a minute, as with Pair).
  • Grants survive updates only with a stable signature, as for Pair; ad-hoc local builds lose them on every rebuild.
  • Several of these grants fail silently: a tap that hears nothing, a capture that returns zeros. So the helper reports what it actually got, setup runs live tests, and the UI shows the results.