0045: Choosing Pair's microphone and camera
On this page
Status: accepted
Context
Pair used the system's default microphone and camera, with no way to change either but the system's own settings. People with a headset, a USB microphone, an external webcam or a virtual camera want to pick one, and change it during a call without leaving. A camera can also open and give nothing at all, such as a virtual camera whose app isn't running, which left a black tile. The voice pipeline sounds good, so a choice must not change how it's processed.
Decision
- Where it lives.
[pair] microphoneand[pair] camerainconfig.toml, each{ id, name }or unset for the system's default. They're per computer, not per call, and not secret. Settings › Pair and the call bar's menus write the same setting (POST /v1/pair/device), so a choice made during a call is kept for later ones. - Who lists devices. Only the pairing helper, which owns the audio device module and the camera (ADR 0039). In a call it lists them every 3 seconds, off the call's thread, and sends
FromHelper::Deviceswhen the lists change; leashd puts them in the Pair view. Outside a call the app runsleash-pair devices, which prints the same lists. Listing needs no permission. - Finding a choice again. A device matches by ID and name, then by name, then by ID (
pair::helper::choose), because a Mac gives a microphone a new CoreAudio ID when it's plugged in again. A choice that matches nothing means the system's default for now; it's kept, and used again once it's back. The call bar says when that happens. - Switching the microphone.
ToHelper::Devicescarries both choices. The helper stops recording, sets the device in libwebrtc's audio device module, and starts again if anyone's connected (otherwise the first connection starts it, as before). The tracks, codec, audio processing and connections stay as they are, so no one needs a new offer. On a Mac the module's first entry follows the system's default. Windows' module lists no default entry, but names the default communications device (the one a call starts with) when asked for index 0xFFFF, so System default goes back to that one. - Opening the camera. The camera thread tries the chosen camera, then the system's default, then the others, up to six. One that gives no picture within 3 seconds (8 for the one chosen, since an iPhone as Continuity Camera can take that long) gives way to the next. A camera that wasn't chosen also gives way when all it sends is a placeholder: one flat color, or darker on average than video black, as a virtual camera with nothing to show sends (kept as a last resort if no camera does better). The chosen camera is used whatever it shows, so a real camera in a dark room is never passed over, and the person hears which one was used instead. When the chosen camera is plugged in again while another stands in, the camera starts over with it. Once running, a camera with no picture for 5 seconds turns off with a message. Short Leash's nokhwa patch makes a frame wait at most a second, so the thread can always stop (vendor/nokhwa/SHORTLEASH.md).
Consequences
- A headset or webcam is picked once and stays picked. Unplugging it falls back to the default without ending the call, and it comes back when plugged in again.
- The call bar's window grows down while a device menu is open, since the menu draws in the bar's own window. The window is see-through around the bar, so nothing is covered that wasn't already.
- Listing every 3 seconds costs a system call for each list, on a background thread. Neither CoreAudio's nor Media Foundation's device-change notifications are used, which keeps both platforms the same but means a new device can take up to 3 seconds to appear.
- On Windows, System default is the device that was the default communications device at the last listing, by index: if you change Windows' default during a call, Pair follows it within a few seconds, but recording moves by index rather than following the system itself.
- On Windows, Media Foundation needs COM joined on each thread that lists or opens cameras; nokhwa joins it once per process, so the helper joins it on its own threads.
- An older helper ignores
ToHelper::Devicesand an older leashd ignoresFromHelper::Devices, so mixed versions keep working with the default devices.