BMO, an Embodied AI Agent
A local multimodal AI agent living in a BMO shell, in progress: a quantized Qwen 3 LLM on a Raspberry Pi 5 with Whisper voice input, TTS output, a live camera for real-world context, and Steam game streaming to double as a console.
BMO is my attempt to build the friendly computer from Adventure Time for real: a local, multimodal AI agent that lives on my shelf, sees the room, talks back, and, because every good roommate should, also plays video games. No cloud. Everything runs at home.
The brain
The core is a quantized Qwen3 4B Instruct model running on a Raspberry Pi 5, squeezed down to roughly 2GB of VRAM with 2-bit quantization, running at about 2.40 tok/s. I initially wanted to run Qwen3 14B, but decided I’d rather keep VRAM headroom free for future projects than max out the model size, so this turned into an exercise in quantizing as aggressively as I could while still getting usable output.
Hardware
Everything is designed to run on one board: the Pi 5 will host the Ollama runtime plus Whisper, Piper and openWakeWord, with a USB mic and speaker for voice I/O, a CSI camera for vision, and Wi-Fi for search and Steam streaming.
Ears, voice, and eyes
- Whisper handles voice input, so you talk to BMO like you’d talk to a person.
- TTS output gives it a voice back.
- A live camera feed provides real-world context cues, so responses can be grounded in what’s actually happening in the room.
Agent pipeline
Wait, Listen, Think, Speak. The Think stage is a small tool-calling loop: the same quantized model runs twice, once to decide whether a tool is needed, then again to generate the response, pulling in the camera, a system-time check, or a DuckDuckGo search as retrieval context.
The console party trick
BMO also streams Steam games over the local network, which turns the same little box into a couch gaming console. It’s the feature that makes people smile, and the reason the project earns its shelf space even between experiments.
Status
WIP. The quantized model and voice/tool loop work on my dev machine, but the Raspberry Pi 5 itself hasn’t arrived yet, so getting it running on the actual target hardware is the current blocker. It’s also my personal testbed for the same question my FSAE work asks: how much intelligence can you run on a small computer at the edge?