Engineering Log #001
Building a real-time AI Building Intelligence platform using Raspberry Pi 5, OAK-D stereo vision and local edge AI.
Mehdi "Masoud" Variji
7/29/20262 min read
Building a Real-Time AI Building Intelligence Platform
Introduction
This project started with a simple question: can a building understand how it is being used without sending video to the cloud?
I wanted to build a system that could observe occupancy, understand room-level activity, maintain a live digital twin and eventually support intelligent building automation. From the beginning, the goal was to process everything locally on affordable hardware instead of relying on cloud AI services.
This engineering log documents the first working version of that platform.
The Hardware
The current platform runs on a Raspberry Pi 5 using an OAK-D stereo depth camera as the primary perception sensor.
Supporting components include an ESP32 for hardware communication, MQTT for messaging, SQLite for local data storage and a PyQt6 application that provides the system dashboard and engineering tools.
Every component operates on local hardware. No camera stream leaves the device.چ
Software Architecture
The perception pipeline is written in Python using YOLOv8 and OpenCV.
The current system includes:
Multi-zone occupancy detection
Face recognition
Real-time digital twin
Anomaly detection
Decision engine
Local SQLite database
15-module PyQt6 dashboard
Each module was developed independently and later integrated into a single application.
Why Local Processing
Keeping inference on the Raspberry Pi was an intentional design decision.
Running locally reduces latency, keeps video inside the building and makes the platform independent of external AI services. It also allows the complete system to continue operating without an internet connection.
This approach influenced almost every engineering decision that followed.
Current Development Stage
The platform is currently running on real hardware inside an occupied residential environment.
At this stage the system observes occupancy, records events and evaluates decisions while continuing to collect engineering data.
Building control remains intentionally conservative during this observation phase so behaviour can be reviewed before additional automation is introduced.
Current Limitations
This project is still under active development.
Energy values displayed by the dashboard are currently simulated. Smart-meter integration has not yet been implemented.
Long-term learning continues as additional observations are recorded.
Several building-control capabilities are planned but are not yet enabled.
Technology Stack
Hardware
Raspberry Pi 5
OAK-D Stereo Camera
ESP32
Software
Python
YOLOv8
OpenCV
PyQt6
SQLite
MQTT
What’s Next
The next engineering log will describe the multi-zone occupancy engine and how room-level occupancy is converted into a real-time digital twin while maintaining separate perception, decision and observation layers.
Status
Status: running(
TechnovaMind
Independent engineering practice focused on Edge AI, Computer Vision, Robotics and AI Building Intelligence..
Contact
© 2025. All rights reserved.
Solutions
AI Building Intelligence
Assistive Robotics
Edge AI & Embedded Systems
AI Workflow Automation
Telegram Commerce Automation
Available for international AI engineering opportunities, technical collaborations and pilot projects.