0035: Pair is peer to peer with libwebrtc
On this page
Status: accepted
Context
Pair lets up to four people share one person's screen, talk, and take turns at the keyboard. It has to feel like sitting together: well under 100 ms from the host's screen to a guest's eyes, sharp code at small sizes, and no echo on laptop speakers.
Building media from scratch means writing congestion control, jitter buffering and echo cancellation. A media server costs money per minute and adds a party that sees the traffic. A browser engine inside the app can't capture well or control input. Phase 0 measured the options (spikes/pair/RESULTS.md).
Decision
- Engine: Google's libwebrtc through LiveKit's
libwebrtcandwebrtc-syscrates (Apache-2.0), used directly. There is no LiveKit server: Pair creates its own peer connections. - Topology: a full mesh of direct connections, up to four people (host and three guests). The host shares; everyone talks.
- Video codec:
- Offered explicitly with
set_codec_preferences: AV1 first, VP9 second, nothing else. - The video source is marked as screen content (
is_screencast), with degradationMaintainResolution, one layer (L1T1), a 30 fps cap and a bitrate cap. - H.264 is never offered.
- Offered explicitly with
- Latency: receivers use zero playout delay (
PeerConnectionFactory::with_zero_playout_delay). - Voice: the platform audio device with WebRTC's own echo cancellation, noise suppression and gain control (
create_device_audio_track). Short Leash has no audio code of its own. - Control: guest pointer and keyboard events, and the clock sync, travel on a WebRTC data channel between the peers, never through signaling.
- Encryption: DTLS-SRTP end to end between peers. Media never passes through a Short Leash server; a relay, when used, only forwards encrypted packets (ADR 0036).
Consequences
- Measured on an M3 Max:
- AV1 at 1440p carried a real screen in 5 ms at the median and 8 ms at the 99th percentile, at about 0.7 Mbit/s, using about a third of a core for both peers.
- On Windows on Arm, AV1 took about a quarter of a core.
- VP8 was rejected: it stalls for up to 100 ms on every scroll.
- No fork: every setting Pair needs is public API in the crates. A field trial or two would need a few lines of glue in
webrtc-sys, not a fork of libwebrtc. - Upload is the limit: the host uploads one stream per guest. Three guests at about 1 Mbit/s each suit home connections; more people would need a media server, a later decision.
- Hardware encoders (VideoToolbox's low-latency mode, Media Foundation) can come later through the crate's pre-encoded path. v1 uses libaom and libvpx in software.
- Licenses: libwebrtc's prebuilt binaries link FFmpeg and OpenH264, so Short Leash ships its own build (ADR 0040).