AI driveway intelligence

One Gemini vision call per driveway visit turns two front-of-house cameras into a timestamped, typed record. A Home Assistant template rule, with no second model call, decides what happens next.

How it works

Flowchart in three stages: a free gate that skips the model when the door just opened or Doorbot is busy, one Gemini call that returns a timestamped event sequence and typed labels, and a template rule that logs the event and starts a package reminder until the front door opens.
Three stages: a free gate, one model call, a deterministic rule. Click for full size.

A motion sensor on the driveway starts the run. Two free state checks stop it first if the front door just opened or Doorbot is already handling a visit. The automation seeds a fail-closed answer, then makes one call to Gemini with frames from two front-of-house cameras, capped at 10 frames, 1024 px wide, and low media resolution. A JSON schema makes the model write a timestamped event sequence first, reading the time burned into each frame so both cameras merge into one timeline. Then it fills fixed-list labels for origin, last action, item, and role, each with an "unclear" option. A Jinja template reads those labels: real activity goes on the event timeline, and a delivery pattern with the door still closed starts a reminder. After a 90-second grace period it announces once, then nags every two hours until the front door opens.

Notable engineering details

  • Evidence before verdict. The schema puts the event sequence first, so every label rests on a written, timestamped list of observations. On-frame timestamps, rather than frame order, merge the two cameras.
  • The decision is a rule. Dispatch is a deterministic template over the model's labels: a delivery role or a set-down item, combined with how the visitor arrived and left. No second model call, so it's free and auditable.
  • Fail-closed by construction. A seeded "no activity, all unclear" answer survives a failed call, the prompt tells the model to prefer "unclear", and truncated replies are logged for review.
  • Cost and noise controls. Single-run mode plus a 45-second cooldown, capped frames and resolution, and a cheaper pre-screen model running in a logged shadow trial before it's trusted to skip calls.
  • Plays well with Doorbot. A shared flag keeps a package from being announced twice, and opening the front door is the one off switch for the whole reminder cycle.

The source is private for security and privacy reasons, so the walkthrough quotes a redacted extract and its line numbers aren't linked to GitHub.

Code walkthrough

Step through the code, chapter by chapter

Generated by code-walkthrough, my Claude Code skill: it picks the order to read the code in, quotes it verbatim from a redacted extract, and ends each chapter with a check-yourself question.

Back to the portfolio