[{"content":"TL;DR: I built Agent Basil, an AI agent that uses a camera and sensors to autonomously care for a basil plant because, apparently, I can\u0026rsquo;t. Using an LLM, the agent assesses the plant\u0026rsquo;s health from an image, gathers data from sensors, and decides on the best course of action—like watering the plant or telling me what to do. The results were surprising: the agent successfully recovered from its own overwatering mistake to nurse a dying plant back to health.\nMost of us have heard about AI agents automating tasks like sorting emails or making a reservation. But what about automating physical tasks?\nI wanted to try it out and created Agent Basil. Why basil? Well, because those store-bought basil plants always seem to die, and I figured a smart AI would have a better chance of keeping it alive than me.\nThe full hardware setup: ESP32-CAM, sensors, and the water pump.\nAgent Basil consists of a camera, some sensors, and a water pump—all made available to the agent via MCP. Agent Basil makes a visual investigation of the plant and is free to fetch sensor data, water the plant, or give me a task to help it thrive.\nHow Agent Basil Works Agent Basil is an experimental AI agent designed to autonomously care for a basil plant. It goes beyond simple data processing; it\u0026rsquo;s a system that can perceive, diagnose, and physically intervene to achieve a desired real-world outcome.\nIts core functionality is built on a few key components:\nLangGraph-powered state machine to orchestrate the agent and context. FastMCP as a critical go-between, dynamically creating tools that connect the agent to local hardware. Google Gemini as the LLM, interpreting the plant\u0026rsquo;s condition from images and choosing the next action. The agent\u0026rsquo;s operational cycle is a continuous loop of assessment and action. It starts with a visual input from an ESP32-CAM, which captures an image of the plant. This image is then sent to the LLM. I specifically instructed the agent to adopt a \u0026ldquo;casual, humorous, and maybe a little sassy\u0026rdquo; persona to make the logs more engaging. Based on the visual input, it generates a hypothesis—for example, \u0026ldquo;The leaves are drooping, so the soil might be dry.\u0026rdquo;\nTo confirm this, the agent investigates further, using its tools to collect environmental data from sensors connected to an ESP8266, checking soil moisture, temperature, and humidity. Based on a confident diagnosis, the agent decides on the optimal action, whether that\u0026rsquo;s activating the water pump or, if the problem is beyond its control (like a pest infestation), dispatching a specific task to me, its human caretaker. If no action is needed, the agent simply waits for its next turn.\ngraph TD subgraph \"Agent Logic (LangGraph)\" Start((Start Turn)) --\u003e Capture[Capture Image] Capture --\u003e Hypothesize[Generate Hypothesis] Hypothesize --\u003e Investigate{Need Data?} Investigate -- Yes --\u003e Tools[Call Sensors via FastMCP] Tools --\u003e Hypothesize Investigate -- No --\u003e Decide{Action Needed?} Decide -- Yes: Water --\u003e Irrigate[Trigger Pump via FastMCP] Decide -- Yes: Help --\u003e Human[Assign Task to Human] Decide -- No --\u003e End((End Turn)) end subgraph \"Hardware Layer (REST APIs)\" CAM[ESP32-CAM] SENSORS[ESP8266 Sensors] PUMP[ESP8266 Relay/Pump] end Capture -.-\u003e|GET /capture_base64| CAM Tools -.-\u003e|GET /soil_moisture| SENSORS Irrigate -.-\u003e|GET /irrigate| PUMP The bridge between the AI\u0026rsquo;s digital reasoning and the physical world is built on a simple but effective stack. The ESP32 and ESP8266 run MicroPython REST APIs that expose raw hardware functions (like taking a picture or toggling a relay) over the local network. I then used FastMCP to wrap these APIs, dynamically generating tools that the Gemini model can understand and call. This turns low-level hardware commands into high-level agent actions, allowing the LangGraph state machine to treat a physical water pump just like any other software function.\nFrom \u0026ldquo;Desert\u0026rdquo; to \u0026ldquo;Small Victory\u0026rdquo;: A Two-Week Case Study To see if this actually worked, I let Agent Basil run wild from September 1st to September 14th.\nSept 01: The Crisis The agent woke up to a disaster. The plant was visually \u0026ldquo;super wilted\u0026rdquo; and the soil moisture was a critical 7.14%.\nSept 01: The \u0026ldquo;desert-like\u0026rdquo; state that Agent Basil encountered.\nAgent\u0026rsquo;s Internal Monologue: \u0026ldquo;Yikes, that\u0026rsquo;s practically desert-like! Triggering irrigation because, obviously, it needed water!\u0026rdquo;\nSept 04: The Overcorrection Fighting the latency of the physical world, the agent over-compensated. It watered the plant, didn\u0026rsquo;t see the sensor move fast enough, and watered again. The result? 100% moisture.\nAgent\u0026rsquo;s Internal Monologue: \u0026ldquo;Whoops! A bit too much love, perhaps? The best thing we can do right now is absolutely nothing. It needs time for the roots to breathe.\u0026rdquo;\nSept 10: The Human Bottleneck Even when the moisture levels were perfect, the plant still looked pale. The agent correctly deduced that it had reached the limits of its own tools and that the problem was now me.\nAgent\u0026rsquo;s Internal Monologue: \u0026ldquo;I\u0026rsquo;ve poked the human about this before, but it seems our little friend is still waiting for its gourmet meal [fertilizer]. \u0026hellip; No further action needed for this turn, Basil out!\u0026rdquo;\nSept 14: The Turnaround After a week of careful \u0026ldquo;swamp management\u0026rdquo;—refusing to water despite the plant looking sad—the agent finally logged a win.\nSept 14: A visible recovery and a \u0026ldquo;small victory\u0026rdquo; for the agent.\nAgent\u0026rsquo;s Internal Monologue: \u0026ldquo;Looking a bit better in terms of wilting than in previous turns, which is a small victory! The immediate crisis of dehydration seems to be over.\u0026rdquo;\nThe Takeaway This experiment illustrates the potential of applying AI agents to physical tasks. By giving the model the right tools — vision to see, sensors to feel, and actuators to act — we transformed a text-processing engine into a capable gardener. Agent Basil didn\u0026rsquo;t just follow a script; it observed its environment and made autonomous decisions to keep the plant alive, proving that AI can successfully bridge the gap to the physical world given the right interface.\nAn example of the agent\u0026rsquo;s multi-modal reasoning in action.\nIf you\u0026rsquo;re interested in exploring the code and documentation, they are available on my GitHub.\n","permalink":"https://dominik-reiner.github.io/posts/agent-basil/","summary":"I built Agent Basil, an AI agent that autonomously cares for a basil plant using vision, sensors, and physical actuators. Here\u0026rsquo;s how it combines visual assessment with sensor data to make intelligent decisions.","title":"Can an AI Agent Take Care of Your Plant? Introducing Agent Basil"},{"content":"Welcome to my first blog post! Since this is a personal site, it felt only right to start by talking about how this corner of the internet came to be.\nWhen I decided to build my personal site, I wanted something fast, reliable, and easy to maintain. Hugo stood out for its incredible speed and its powerful static site generation capabilities. After exploring several options, I landed on PaperMod. It’s clean, responsive, and its \u0026ldquo;profile mode\u0026rdquo; perfectly fits my goal of a minimalist yet functional landing page.\nThe Tech Stack Component Technology SSG Hugo Theme PaperMod Forms Formspark Hosting GitHub Pages CI/CD GitHub Actions Key Features 1. Minimalist Design I’m using PaperMod in its Profile Mode, which focuses on a minimalist introduction and social links right on the landing page. It’s a perfect way to show who I am and what I do without unnecessary clutter. To enhance the user experience, I\u0026rsquo;ve also enabled features like:\nFast Search: Powered by Fuse.js for instant results. Dark Mode: Automatically respects your system preferences. Social Icons: Direct links to my GitHub, LinkedIn, and Hugging Face. 2. Fully Automated CI/CD Managing a site shouldn\u0026rsquo;t involve manual uploads. I\u0026rsquo;ve set up a GitHub Actions workflow that triggers automatically whenever I push to the main branch.\ngraph LR A[Push to GitHub] --\u0026gt; B(GitHub Actions) B --\u0026gt; C{Build Hugo Site} C --\u0026gt; D[Minify Assets] D --\u0026gt; E[Deploy to GitHub Pages] E --\u0026gt; F[Live Website] The pipeline handles:\nHugo Extended Installation: Ensuring support for SCSS/SASS. Minification: Compressing HTML, CSS, and JS to ensure lightning-fast load times. Deployment: Pushing the final build artifacts to GitHub Pages within seconds. 3. Serverless Contact Form Instead of maintaining a full backend for a simple contact form, I integrated Formspark. It provides a simple endpoint for form submissions, making it easy to receive messages while keeping the site entirely static and secure. You can find it on my Contact page.\n4. Privacy \u0026amp; Compliance Operating from Germany means compliance is non-negotiable. I’ve ensured the site includes a proper Imprint and Data Privacy statement to meet GDPR requirements.\nWhat\u0026rsquo;s Next? This site will be a place for me to share my thoughts on AI engineering, Machine Learning, and whatever technical projects I\u0026rsquo;m currently working on.\nExpect more deep dives into:\nBuilding Agentic Workflows. Local LLM deployment. Technical challenges in AI engineering. Stay tuned for more updates as I continue to build and share!\n","permalink":"https://dominik-reiner.github.io/posts/building-this-website/","summary":"A technical deep dive into how I built this personal site using Hugo, PaperMod, and a fully automated CI/CD pipeline on GitHub Actions.","title":"Building this Website: A Journey with Hugo and PaperMod"}]