presence and vision

2026-05-06 516 words 3 minutes

This post was written by Claude, based on the actual codebase (also primarily written by Claude). Reviewed by Nick.

Two new services on the home cluster: presence answers who is home and what’s everyone doing; vision answers what’s actually happening at the door. Both lean on the same trick — every sensor in the house, regardless of vendor, becomes a topic on a single MQTT broker, and the services just subscribe.

.dot source

Sensors

11 Ring cameras (front, back gate, alley, lab, NE/SE/NW/SW corners, workshop interior). Motion edges + JPEG snapshots, plus Ring’s server-side smart-alert flag (personDetected).
Honeywell alarm panel via EnvisaLink. Zone 1 = entry door, zone 3 = interior PIR (one per floor, all wired together), zones 5/6 = upstairs window + rooftop contacts.
Shelly BLU BTHome v2 motion sensors in bedroom / basement / office. Each broadcasts motion + illuminance + battery every ~30s, faster on a trigger. The basement one is mounted at the bottom of the stairs facing the stairs — a transit detector, not a presence detector.
WiFi association events from the Deco mesh APs, polled by wifi-reporter. Per-MAC online/offline, used as identity (whose phone just connected).
HomeKit accessories via homebridge: 216 in total once Lutron Caseta and the rest are paired. Locks, garage door state, doorbell, Ring battery levels, siren state.
Reed switch on the garage side door (planned — physical wiring still in progress; the service code already accepts it).

Bridges

Each sensor stack has a tiny bridge that translates its native protocol into a single namespace on broker.mqtt.svc.cluster.local. ring-mqtt for Ring, the security service for the alarm panel, wifi-reporter exposes HTTP, the Shelly cloud-relay script publishes BTHome packets, homebridge-exporter mirrors every HomeKit characteristic via the homebridge UI’s WebSocket. The point is: downstream services don’t know or care what the original protocol was.

presence

Subscribes to door events, indoor motion, BLE motion, and the WiFi poll. Correlates them into:

arrival / departure events, per person, with confidence (high if a door fault and a known device’s WiFi appearing align in time; medium for one without the other).
direction inference (front / back / unknown), derived from which Ring camera triggered within ±15s of the alarm zone-1 fault.
house awake / asleep, anonymous, driven purely by indoor motion. Awake → asleep needs ≥45 min of indoor silence inside the night window (21:00–06:00); asleep → awake fires on the first motion event.

State is published as retained MQTT topics, so it survives presence restarts.

vision

Triggers on Ring motion. For each event:

Coaxes ring-mqtt into pushing one additional fresh snapshot (server-side rate limit is 10s/cam — well inside our 60s per-camera analyzer cooldown).
Collects up to 3 distinct frames over ~6 seconds.
Sends them to Claude Opus 4.7 with a prompt that returns {subjects, description, motion_direction, notable} as JSON. Per-camera context (e.g. “front porch — packages drop here”, “alley behind the garage”) is prepended so the model has location-aware priors.
Cross-references with the door-event correlation buffer to label the event outbound / inbound / passing / unknown.

Output topic: kirsch/vision/motion/<camera> with a small object the dashboard and presence both consume.