I started to work on my DIY home automation a few years ago.

Quickly it became obvious that equipments had to be connected to a core system

So step by step I started to design my Home Automation Infrastructure that I want to share

What are right now the main achievements ?

All equipments are connected either with RF or WIFI
All equipments are monitored even some managed over Internet
Equipments actions can be triggered with a scheduler
One event on one equipment can schedule one action on an other equipement.
One alert system informs on troubles
Sensors data are stored for treatments and analysis
Thanks to standardization it is effortless to add new equipments
I have total control on my home automation system

In this instructable I will enter in details in this infrastruture.

Some microcontroller softwares can be used as is.

Regarding the HUB, I will be pleased to share the server software with people interested to work on it.

Supplies

One server that operates Linux

Some Arduino or ESP8266 microcontrollers

Some sensors

Step 1: Let's Illustrate With One Example Before Entering in Technic

With this example we can see that it is possible to have a global view of the comfort equipments and to manage them consequently locally or remotely

Step 2: Infrastructure Overview

The core of the infrastructure is the Hub. The Hub is made of a software that runs on a central server.

Sensors and actuators are connected thru microcontrollers.
Microcontrollers communicate with the Hub with a common protocols and some gateways.

Doing so, heterogen equipments can interact with each other.

They are monitored and managed with laptops and smartphones on the home LAN even over Internet.

It is possible to add a new station without any coding.

Step 3: Infrastructure Overview

What are the components of this infrastructure ?

One architecture
Some communication standards
Some software components
Some hardware components

Step 4: Communication Overview

From top to buttom communication relays on

UDP/IP over Internet
UDP/IP using Ethernet and WIFI over the home LAN
Communication between microcontrollers and upper levels can be:
- direct for those that supports WIFI
- indirect thru a serial/WIFI gateway

- indirect thru a RF433/Ethernet gateway
- Communication between microcontrollers and sensors are specific to the sensors

Step 5: The Software Components: 3 Gateways

Gateways softwares

Running on ESP8266 with a WIFI connection converts serial link frames to UDP frames and vis et versa
Running to Arduino with a Ethernet connection converts RF433 frames to UDP frames and vis et versa
Running to Arduino with a Ethernet connection converts UDP instructions to RF433/Home Easy and Somfy signals

RF stations share the gateways, so there are a maximum of 2 RF gateways for all the RF stations of one home.

Here you can find the ESP8266 WIFI Gateway instructable

Here you can find one use case of the ESP8266 WIFI Gateway

Here you can find an instructable for the RF433 UDP/IP gateway

Step 6: The Software Components: 3 Microcontroller Frameworks

3 frameworks are available that make it easy to develop new connected home stations. It allows to concentrate on data and services without taking care of communication.

2 based on Arduino platforms

One for radio frequency based communication. It supports unidirectional and bidirectional exchange. Messages go are converted thru RF Gateway between RF and UDP/IP frames.
One for WIFI based communication. It supports unidirectional and bidirectional exchange. Messages go are converted thru ESP8266 WIFI gateways between RF and UDP/IP frames.

1 based on ESP8266 plafroms

Use WIFI communication. It supports unidirectional and bidirectional exchange. Messages are directly send to and received from the HUB

Data and services that support by these frameworks are described further in this instructable

Trick: station identifiers are stored in Eeprom. So it is possible to upload the exactly same firmware inside multiple microcontrollers. The HUB will automatically differentiate them.

Step 7: Terminology

Station: one microcontroller with sensors and actuators

Indicator: a station parameter that values is stored in the HUB database. Typically the microcontroller send regularly the indicators values to the HUB. Eventualy parameters values can be set from the HUB to the station

Measurement: value of a sensor to store in the HUB database or in a Google sheet

Station, indicator and measurement are defined in the HUB database and identified with a unique identifier

Alert: an event sent from the station to the HUB that requires specific attention

Register: a microcontroller 8 bits register. It can be send from the station to the HUB and vis et versa. Registers are defined inside the microcontroller identifed with a unique identifier
Command: optionnaly it is easy to add command. A command byte sequence that is specific to the station

The HUB can forward registers and command to dedicated code via an UDP forward

Service: a functionality provided by the HUB to the stations

Action: something we want the station to do. Action are defined in the HUB Database

Step 8: Data Flow

The HUB keeps regularly stations and gateways informed of services availability and theirs UDP/IP addresses. By doing so it is possible move the HUB to an other server without any change on the stations and gateways side.

There are two kinds of data flow

Binary mode for Arduino stations, Arduino gateways and ESP8266 gateways
ASCII mode for ESP8266 WIFI station

Step 9: Data Flow: Alerts and Events

Depending on certain conditions a station can decide to send an 'Alert' message to the HUB. The message contains some characteristics as Category, Severity...
The HUB will acts depending on the charactristics. If needed it will schedul one or more tasks.

For instance it can immediatly update a register in an other station, and later send an email.

Tasks can be chained

Step 10: Let's Zoom on the HUB Side

The HUB software runs on Linux platform.

The code is 100% JAVA.

There are 2 kinds of code.

The "batches" one, standard, mostly running 24/7.

It communicates in real time with all the stations. It continuously stores data in Mysql tables.

It regurarly check for update "indicators" table and if needed informs stations.

Data purge is done once a day according to specifications set in a Mysql table.

The HMI one running over Tomcat application server
- Class and Servlet are specific and developed according to the needs
- It shares the Mysql tables with the "batches"
- One API allows to send commands to the HUB
- It is possible to act with a station by:
- updating an indicator target in the Database, that target will be automatically forward to the
- updating a microcontroller register
- creating an alert on the HUB, that will trigger event

Step 11: Measurments in a Google Sheet

Instead of storing measurement values in a Mysql table, it is possible store them in a Google Sheet.

A new line is added for each new set of values.

Doing so it is possible to get a dynamic google graph.

Step 12: Let's Zoom in the Indicators Dynamics

The above picture illustrates the indicators dynamics
In this example the station with identifier 100 send 3 indicators (1,2 and 3) values to the HUB.

Firstly the message is decoded based on the indicators characteristics table.

indicator 1 is an INT of one byte,

indicator 2 is an INT of two bytes and

indicator 3 is one Byte with a target value of 0x02.

Decoded values are added in the indicators values table with a timestamp.

As indicator 3 value sent by the station is not equal to the target, the BUB will automatically send a request update to the station.

Step 13: Let's Zoom in the Measurements Dynamics

The above picture illustrates the measurements dynamics

In this example the station with identifier 102 send 2 measurments of type 1 values (1 and2) to the HUB.

Firstly the message is decoded based on the measurements characteristics table.

the first measurment of type 0 for the station 102 is 'temperature'

the second measurment of type 0 for the station 100 is 'humidity'

2 records are added to the measurements values table

Measurements are INT16 (signed).

The station can send up to 10 measurements of the same type for a station

Step 14: Let's Zoom in the Registers Dynamics

From station to HUB

Registers provide the easiest way to develop specific code for a station.

When a station send one or more registers to the HUB , there are automatically forwarded to a specific defined UDP/IP address.

Doing so it is easy to add specificities wherever you eed over Internet, just by listening a UDP port.

From HUB to station

The registers mapping table is used to convert a requested action for a station into an registers update command to send to the station.

Actions can come from a scheduled task, a crontab entry or a HMI request.

Update registers is the easiest way to act on a station in real time

Step 15: Alerts & Events Dynamics

A station can send an "Alert" message. Alert message are used to notify that something happened that need human attention and/or that need the HUB to trigger actions. It does not mean something went wrong. For instance the end of a trouble is notified by an "Alert" message with a closed MesgType.

Status, MsgType, Urgency, severity and ResponseType are codified. Category can be defined according to the station need.

All "Alert" messages are stored in the HUB database and can be viewed on a web HMI.

When the HUB receive an "Alert" message it will look in the database for event to trigger depending on the stationId, MsgType and Category. If so the event will be activated. An event provides actions and even schedules an other event.

Step 16: Add a New Station Without Coding

For instance imagine you add already developed a weather station with the id 0xfe01 and want add another one.

To do so you will have:

Set the address inside the Arduino code to 0xfe02
Set the MOSI Pin of the new Arduino to ground
Upload the Arduino code inside the new station the station configuration will be written in eeprom
Remove the MOSI Pin from ground Restart the Arduino

That's it for the Arduino side