LLM Watches My Garden: How AI Vision Decides Whether to Tell Me to Cover the Furniture Before Rain

The author set up a smart home automation in Home Assistant to protect outdoor furniture from rain. The system monitors precipitation forecasts and nowcast data, and when rain or snow is imminent, it turns on a camera's floodlight, captures a snapshot, and sends it to a multimodal LLM via the native AI Task integration with a structured output schema to determine whether the furniture is covered. If the LLM returns covered=false, it sends a Telegram message with the photo and an animated rainfall radar, along with two inline buttons: 'Covered' and 'Snooze for 3 hours'. The automation respects a cooldown of 3 hours between real checks and a snooze mechanism to avoid spamming, and it does not call the LLM unless precipitation is actually close. The entire pipeline is built from native Home Assistant primitives: automations, helpers, and the built-in LLM integration, without custom components. The host is a Raspberry Pi 3 running Home Assistant in Docker, with an IP camera and a weather integration providing nowcast and radar imagery. The author emphasizes that the LLM returns structured data (covered and reason) rather than free text, and that attachments must be placed in the media-source directory. The final authority is the human, who can tap a button to confirm covering or delay the reminder. The automation was assembled by an AI agent via SSH and REST/WebSocket APIs, while the human only set the task and accepted the result.