Introduction: Creating Your First IoT Ethernet Device. Part 5 : IoT, Home Automation

By www.youtube.com/channel/UCNDFrgp

This is a 'twofer' (two for one), the first part (A) is the Ethernet equivalent of it's WiFi ESP8266/DHT22 based predecessor with an added I2C LCD display for summary data (see second picture above). Then in part (B) we will extend the design (see first picture above) to include barometric pressure measurement, add a real time clock, graphical TFT display showing live data trends (see third picture above) and log all values to an SD card in CSV format for direct import to Microsoft Excel (TM).

Introduction

As mentioned, this Instructable is split into two parts. Part (A) documents how relatively simple it is to cross port to Ethernet, extend and further develop an MQTT IoT device based around the ESP8266-01 WiFi enabled module from the earlier Instructable 'Pimping your first IoT WiFi Device. Part 4 : IoT, Home Automation'

The target IoT device has a single led output, a button input and can read and display both local temperature and humidity levels via a DHT22 sensor/20x4 LCD display and calculate the resultant heat index and also display remote temperature and humidity levels communicated via topic subscriptions to an MQTT Broker. Control functionality and a network connection is achieved via the Arduino Ethernet shield.

Part (B) seeks to further extend this initial design by adding the following functionality;

  1. Barometric Pressure Sensing via BMP085,
  2. Realtime clock using a DS1307Z+,
  3. ILI9431 TFT Display for live trending of data,
  4. Data logging to an SD Card in CSV format.

To set the system up, use the source code provided and create the necessary circuitry as before you will need the following;

For Part A

  1. A minimal grasp of electronics,
  2. Knowledge of Arduino and it's IDE,
  3. Some Patience,
  4. Some understanding of your home network.

System entry requirements;

  1. Home network with Ethernet (and WiFi to receive remote data),
  2. An MQTT Broker correctly configured (See Part 1 and 2 : IoT, Home Automation),
  3. MQTTSpy installed and set up. (See Part 2 : IoT Home Automation),
  4. You have created a remote WiFi IoT device. (See 'Pimping' your first IoT Device. Part 4 : IoT, Home Automation). This is used as the source of the remote Temperature and Humidity levels displayed and recorded.

You will also need the following parts;

  1. 1 off Arduino Mega 2560
  2. 1 off Ethernet Shield
  3. 1 off 10K resistor
  4. 2 off 1K resistors
  5. 1 off white led
  6. 1 off DHT22 Temperature and Humidity Sensor
  7. 1 off SPST Button
  8. 1 off push fit perf board for prototyping
  9. 1 off 20x4 LCD display with I2C interface adaptor fitted. See Instructable www.instructables.com/id/Arduino-I2C-LCD-Driver-Library-and-PackMan
  10. Various interconnection wires.

The article will cover these topics;

  1. Brief overview of the circuit,
  2. Brief overview of the software,
  3. How to test your IoT device and see it in action (detailed),
  4. References used (grouped with Part B).

For Part B

  1. A reasonable grasp of electronics,
  2. Knowledge of Arduino and it's IDE,
  3. As always some patience and attention to detail (there are more components to connect up),
  4. Knowledge of your home network.

System entry requirements;

  1. Same as for Part A.

Parts list;

  1. 1 off Arduino Mega 2560
  2. 1 off Ethernet Shield
  3. 8 off 10K resistors
  4. 4 off 1K resistors
  5. 1 off 100R resistor
  6. 1 off DHT22 Temperature and Humidity Sensor
  7. 1 off BMP085 Barometric Pressure Sensor
  8. 3 off SPST Buttons
  9. 1 off White Led
  10. 2 off VN10LP Fets
  11. 1 off CD4050 level shifter
  12. 1 off 20x4 LCD display with I2C interface adaptor fitted (see above)
  13. 1 off ILI9341 TFT Display
  14. Various interconnection wires

Topics covered;

  1. Brief overview of the circuit,
  2. Detailed overview of the software,
  3. How to test your IoT device and see it in action (brief),
  4. References used (grouped with Part A).

Series Links

To Part 4 : 'Pimping' Your First IoT WiFi Device. Part 4 : IoT, Home Automation

To Part 6 : Setting Up and Configuring OpenHAB. Part 6 : IoT, Home Automation

Step 1: Circuit Overview - Part A

The simple circuit layout and circuit diagram are pictured above.

Output from the DHT22 Sensor is fed to the D2 input on the Arduino Mega via a 10K pull up resistor.

The input button status is fed to D7 on the Arduino Mega 2560 via a 1K pull up resistor. When pressed the D7 reads low and released read high. This status is inverted in software.

Output Led is controlled by a high or low level on the output of D6 via a 1K ballast resistor. Output high = led on, output low = led off. Consequently no software state inversion is required.

The I2C LCD display is connected to the SDA and SCL pins on the Arduino Mega 2560 and draws power directly from the +5v rail on the Arduino.

Step 2: Software Overview - Part A

Preamble

To successfully compile this source code you will need the following extra libraries;

PubSubClient.h

DHT.h

Bounce2.h

LiquidCrystal_I2C_PCF8574.h

Full details also given in source code below.

You will also need to know;

  1. The IP address of your MQTT Broker (in IPv4 format AAA.BBB.CCC.DDD) : IPAddress server()
  2. A free IP address to use for your IoT device if you choose not to use the 'USE_DHCP_FOR_IP_ADDR' #define in the code.

Code Overview

On startup the software connects to the Ethernet network using the pre-supplied connection details above.

Once connected to a LAN the IoT device will publish its details on the following MQTT topics;

  1. /EthernetDevice/SwVerConfirm : This contains a concatenated string, comma separated, formed by the generic device name, unique MAC address of device and the name of the Arduino file used to program the device. in this case 'WIZNet5100,de:ad:be:ef:fe:ed,EthernetTempHumiCD2.ino'.
  2. /EthernetDevice/Temp1Status : On initial connection to the network this topic publishes a 'Connected' message to the broker. Subsequently it is used to publish the local temperature in Degree Celsius accurate to 2 decimal places. Updates are only issued if there is a change in temperature to prevent this IoT from 'clogging' up the network.
  3. /EthernetDevice/Humd1Status : Similar to Temp1Status, but is a representation of the local humidity levels scaled as a percentage.
  4. /EthernetDevice/HeatInd1Status : Similar to both Temp1Status and Humd1Status, though in this instance the software computes the new heat index from the temperature and humidity readings.

It then subscribes to the following MQTT topics and responds to them via a 'callback' function;

  1. /EthernetDevice/SwVerCommand : When notification is received of this topic the device will respond with a /EthernetDevice/SwVerConfirm publication. This is a broadcast response. ie. all devices could respond.
  2. /EthernetDevice/de:ad:be:ef:fe:ed/SwVerCommand : When notification is received of this topic the device will respond with a /EthernetDevice/SwVerConfirm publication. This is a targeted response. ie. only this device will respond.
  3. /EthernetDevice/Led1Command : When notification is received of this topic the device led output is set accordingly. Payload '1' = Led on, Payload '0' = Led off. Once a command has been actioned the device will respond by publishing the Led status via the /EthernetDevice/Led1Confirm topic 'On' or 'Off'
  4. /EthernetDevice/Button1Status : When notification of publication is received on this topic the device will respond by publishing the current button status via a /EthernetDeviceButton1Command topic 'Pressed' or 'Released'. In this way the status of the button can be checked at any given time. Note also if the button state changes independently, via a button press then the device will automatically publish a /EthernetDevice/Button1Command topic.
  5. /WiFiDevice/Temp1Status : Similar to /EthernetDevice/Temp1Status but contains the remote IoT WiFi device Temperature
  6. /WiFiDevice/Humd1Status : As above in /EthernetDevice/Humd1Status but for Humidity levels at remote IoT WiFi device.

Once completed the software now drops into a loop monitoring the Broker connections along with any changes in input button status, local temperature or humidity levels or new publications of remote temperature and humidity. All temperature and humidity changes both local and remote are displayed on the system LCD in real time.

Attachments

Step 3: Testing Your IoT Device - Part A

To test our new Ethernet IoT device we will use the system set up outlined in the first slide above.

For the most part we will use the IoT devices themselves to make topic publications and view these with MQTTSpy on the PC which is connected to the Mosquitto MQTT Broker on the Raspberry Pi.

MQTTSpy will be used to test button presses and trigger led illumination.

The second diagram depicts the flow of publications and subscriptions in our system.

The video clip above shows the respective publications and subscriptions being exercised via MQTTSpy and interaction between local and remote IoT devices.

For further details on how to use MQTTSpy see Instructable Part 2 (Setting Up an MQTT Broker. Part 2 : IoT, Home Automation) and Part 3 (Creating your first WiFi IoT device. Part 3 : IoT Home Automation)

Step 4: Circuit Overview - Part B

This circuit design threw up some interesting problems to resolve before I could put all the sub-systems together. The main issues were interfacing to devices over SPI and I2C where those items were both 3v3 and 5v.

Specifically, the Ethernet Shield and SD card were accessed via SPI and are 5v compliant, the ILI9341 is SPI but 3v3 compliant (even though it is powered via +5v as it has on board +3v3 regulation but no level shifting at its inputs or outputs).

The LCD display and RTC are I2C and 5v compliant, while the BMP085 module was I2C with 3v3 rails. Again similar to the TFT display with on board +3v3 regulation.

To overcome this issue I used two methods of level shifting; for the SPI bus I used a CD4050 and for the BMP085 I used a 'home rolled' FET level shifter.

I used 10K pull ups for the I2C lines. You may notice there are two sets of 10K pull ups on pair for the BMP085 (R5/6) and the other for the RTC (R7/8). Actually the second set are not required, but the circuit I have published is faithful to my actual layout and my system uses a modular RTC I 'knocked up' for this instructable and it contains its' own pair of 10K pull ups so I can re-use later.

All inputs to the Arduino Mega 2560 are via tactile buttons, each of which is independently pulled high via a 1K resistor. This will give a logic high level when released and a logic low when pressed. This state is inverted in software.

For the output LED a 1K ballast resistor is used to limit current. The LED is turned on by a logic high on output pin D6. So no software state inversion is necessary.

A 10K pull up resistor is used for the Data I/O line to the DHT22 as per the manufacturers datasheet. Communications to this device is via D2 on the Mega 2560 and pretty much follows the Adafruit example.

The unused level shifter IC1F input is tied to 0V to prevent any oscillation from a floating input.

Step 5: Software Overview - Part B

Preamble

To successfully compile this source code you will need the following extra libraries (there are a few);

PubSubClient.h

DHT.h

Bounce2.h

LiquidCrystal_I2C_PCF8574.h

Adafruit_GFX.h

Adafruit_ILI9341.h

DS1307.h

Time.h

Adafruit_BMP085.h

Full details also given in source code below.

You will also need to know;

  1. The IP address of your MQTT Broker (in IPv4 format AAA.BBB.CCC.DDD) : IPAddress server()
  2. A free IP address to use for your IoT device if you choose not to use the 'USE_DHCP_FOR_IP_ADDR' #define in the code.

Code Overview

On startup the software connects to the Ethernet network using the pre-supplied connection details above.

Once connected to a LAN the IoT device will publish its details to the following MQTT topics;

  1. /EthernetDevice/SwVerConfirm : This contains a concatenated string, comma separated, formed by the generic device name, unique MAC address of device and the name of the Arduino file used to program the device. in this case 'WIZNet5100,de:ad:be:ef:fe:ed,MQTTEthernetAll6_5.ino'.
  2. /EthernetDevice/Temp1Status : On initial connection to the network this topic publishes a 'Connected' message to the broker. Subsequently it is used to publish the local temperature in Degree Celsius accurate to 2 decimal places. Updates are only issued if there is a change in temperature to prevent this IoT from 'clogging' up the network.
  3. /EthernetDevice/Humd1Status : Similar to Temp1Status, but is a representation of the local humidity levels scaled as a percentage.
  4. /EthernetDevice/HeatInd1Status : Similar to both Temp1Status and Humd1Status, though in this instant the software computes the new heat index from the temperature and humidity readings.
  5. /EthernetDevice/Barometric1Status : Similar to Temp1Status, Humd1Status and HeatInd1Status, but is a representation of the local barometric pressure level measured in hPa or mBar.

It then subscribes to the following MQTT topics;

  1. /EthernetDevice/SwVerCommand : When notification is received on this topic the device will respond with a /EthernetDevice/SwVerConfirm publication. This is a broadcast response. ie. all devices could respond.
  2. /EthernetDevice/de:ad:be:ef:fe:ed/SwVerCommand : When notification is received of this topic the device will respond with a /EthernetDevice/SwVerConfirm publication. This is a targeted response. ie. only this device will respond.
  3. /EthernetDevice/Led1Command : When notification is received of this topic the device led output is set accordingly. Payload '1' = Led on, Payload '0' = Led off. Once a command has been actioned the device will respond by publishing the Led status via the /EthernetDevice/Led1Confirm topic 'On' or 'Off'
  4. /EthernetDevice/Button1Status : When notification of publication is received on this topic the device will respond by publishing the current button status via a /EthernetDeviceButton1Command topic 'Pressed' or 'Released'. In this way the status of the button can be checked at any given time. Note also if the button state changes independently, via a button press then the device will automatically publish a /EthernetDevice/Button1Command topic.
  5. /EthernetDevice/SetTime : When notification of a publication is received on this topic the device will update the system clock (DS1307Z+) with this time value. Payload must be of the form 'HH:MM' (24HR clock).
  6. /EthernetDevice/SetDate : When notification of a publication is received on this topic the device will update the system clock (DS1307Z+) with this date value. Payload must be of the form 'DD/MM/YY', UK format but can be easily changed.
  7. /EthernetDevice/SetLoggingCommand : When notification of a publication is received on this topic the device will update the logging state machine. A Payload of 0 = Close current logfile, 1 = Open/Start new logfile. A new logfile name is formed by concatenating DD/MM with HH:MM to form 'DDMMHHMM.csv'. In this way a unique filename is formed on each opening. As the Arduino SD card library has the old DOS 8.3 filename convention limitation, this means that a period of at least one minute before new file openings must elapse. Otherwise it is possible to overwrite an earlier opened file (within the same minute). Not a major issue but one worth pointing out.
  8. /EthernetDevice/SetLoggingPeriodCommand : When notification of a publication is received on this topic the device will update the system variable iLoggingPeriodInMinutes, which is used to govern the period at which current Local Temp/Humidity/BarometricPressure, Remote Temp/Humidity is logged to SD card. Payload must be of the form 'HH:MM' (24HR clock).
  9. /EthernetDevice/GetLoggingStatus : When notification of a publication is received on this topic the device will respond by publishing the current status of logging control variable currentLoggingStatus via /EthernetDevice/GetLoggingStatusConfirm topic. A value of '0' = Logging Inactive (eLoggingInactive) and '1' = Logging Activated (eLoggingActive). As this enumerated type has specifically weighted values of 0 and 1 for eLoggingInactive and eLoggingActive respectively, by simply adding the numeric offset of Ascii '0' the returned value will be a '0' or '1'.
  10. /EthernetDevice/GetLoggingPeriodStatus : When notification of a publication is received on this topic the device will respond by publishing the current logging period in 'HH:MM' (24HR clock) format via a /EthernetDevice/GetLogginPeriodConfirm topic.
  11. /WiFiDevice/Temp1Status : Similar to /EthernetDevice/Temp1Status but contains the remote IoT WiFi device Temperature
  12. /WiFiDevice/Humd1Status : As above in /EthernetDevice/Humd1Status but for Humidity levels at remote IoT WiFi device.

Once completed the software now drops into a loop monitoring the Broker connections along with any changes in button status, logfile status, logging period changes, logging status updates (via MQTT topic publications or via a local button press to close or open the logging file), local temperature or humidity levels or new publications of remote temperature and humidity. A summary of local barometric pressure, temperature and humidity changes both local and remote, along with current .csv filename, date and time are displayed on the system LCD in real time and repeated along with plotted trends on the system TFT.

Handling logging data to file

Logging of all local temperature, humidity, barometric pressure and remote temperature, humidity is done in .csv form (comma separated variable) this is to allow for direct import into Microsoft Excel (TM) such that it becomes relatively simple to create nice graphs to visualise the recorded data.

Logging is controlled via the use of a 'Software State Machine' which records the various states that logging can occupy (these reflect the enumerated type eLoggingStatusType of eLoggingInactive, eLoggingActive, eLoggingInitialise and eLoggingFault). A diagram of this state machine has been included above. The blue circles are the various states and the lines or 'transitions' between them show any legitimate entry or exit into or out of those states. The text above each transition shows the condition for moving between states and the text below the transition show any action which may be required to move between states.

Changes to logging are achieved by setting the variable newLoggingStatus to the state of choice and then calling the updateLogging() function. Which is automatically called on each pass of the main loop().

The state machine logic inside the updateLogging() function will take care of the necessary state transitions and actions such as updating the TFT and LCD screens of a change of filename or clearing the log filename if logging is terminated.

Handling screen scrolling

Screen scrolling is a interesting topic so I thought I would add a few comments on how I achieved it.

From the second diagram above I have depicted a typical series of screen plots from pictures 1 ... 5. Here you can see the data trend line (yellow) progress from left to right on the screen until screen plot 7 is reached and we have run out of memory to save the 7th data plot point. Simple we could make an array a bit longer you say. Then just plot the new point. Well we just can't plot the new point as it will be off the display to the far right as in screen 6. A solution to this would be to use head and tail pointers (marked here as H and T), then just clear the screen and re-plot all points between Tail and Head. That would work for the scrolling right to left as in screen plot 7. However, there's a another fundamental issue with this approach, unless we possess a Turing machine (https://en.wikipedia.org/wiki/Turing_machine) we can't keep adding to our array indefinitely. We need a more efficient algorithm.

The solution is to move the Head (or Tail) pointer back round to the beginning of our array to store next plot once it 'falls off the end' of the array. In this way a data buffer of fixed size can be used. The screen is then updated in exactly the same way as described above. Actually the plotting is done from H-1 and T-1, but this is a minor detail. See screen plots 8 and 9.

We have in effect created a 'circular' buffer from a 'linear' one. See picture three above.

This is exactly the algorithm I used in function updateTrendScreen() to get the screen to scroll from right to left.

Of course I added a few tweaks to optimise the updating and make it flow a bit better, but its essentially the same.

I left out one small cosmetic optimisation, can you spot it?

Step 6: Testing Your IoT Device - Part B

To test our extended Ethernet IoT device we will use the system set up outlined in the first slide above.

As before, for the most part we will use the IoT devices themselves to make topic publications and view these with MQTTSpy on the PC which is connected to the Mosquitto MQTT Broker on the Raspberry Pi.

MQTTSpy is used to test button presses, trigger led illumination, set time/date, set/get logging period, and open/close logfile.

The second diagram depicts the flow of publications and subscriptions in our system from the trending perspective. (ie. it doesn't include those items tested with MQTTSpy directly).

For further details on how to use MQTTSpy see Instructable Part 2 (Setting Up an MQTT Broker. Part 2 : IoT, Home Automation) and Part 3 (Creating your first WiFi IoT device. Part 3 : IoT Home Automation)

I have also included a screen shot of actual barometric pressure data recorded from the BMP085 (above) along with the resultant .csv and .xls spread sheet files (below).

To check the functionality of the trend screen scrolling, there was no other way that to watch it do it's thing for at least one full screen, to speed this process up I reduced the trend screen update period to once every 10 seconds (#define DEFAULT_TREND_LOGGING_PERIOD_IN_SECONDS 10). If you are having trouble sleeping I included a video clip of one such test above.

One interesting point to note is that since actual live data spooled from the sensors was use to build the trends, it took at least three samples before the remote temperature sensor detected a change in local temperature sufficiently large enough to report via a publication. So it is useful to bear this in mind, if you choose to test in this way.

Final note : The trend screen video clip shows a wide variance in temperature and humidity. These were forced by blowing on the sensors and artificially heating them during filming. I wish to state up front, these are not representative of the ambient conditions in the South of England. It's actually a very nice and moderate climate. Honest. :-)

Attachments

Step 7: So What's Next?

Ok, so if you've been following the earlier parts of this series on IoT and home automation you are probably at the point where the 'fun' of using MQTTSpy to test and run your setup is 'wearing a little thin'. Now we have a few IoT devices we can start to do something useful with them. Therefore the next step is to add a user friendly 'front end'. In our case we will be using OpenHAB.

Step 8: References Used

I used the following sources to put this instructable together;

PubSubClient.h

DHT.h

Bounce2.h

LiquidCrystal_I2C_PCF8574.h

Adafruit_GFX.h

Adafruit_ILI9341.h

DS1307.h

Time.h

Adafruit_BMP085.h

General details regarding the ILI9341 TFT display

General details on SD card usage with the Arduino Ethernet Shield

Datasheets

Comedy Value