IoT Desktop Console. Part : 11 IoT, Home Automation

Preamble

This article, the 11th in a series on home automation documents how to create and integrate an IoT Desktop Device into an existing home automation system including all the necessary software functionality to enable the successful deployment within a domestic environment.

Picture 1 shows the completed IoT desktop device and Picture 2 shows all the component parts used in the prototype which were 'shoe horned' into the finished article.

Introduction

As mentioned above this Instructable details how to make an IoT Desktop Device whose primary purpose is to present the user with an 'at a glance' summary of temperature and humidity levels published by distributed sensors within an IoT network accompanied by local barometric pressure along with time and date.

The design seamlessly hooks into the MQTT/OpenHAB based IoT network detailed in this series on home automation (HA), building on reused code taken from here. Alternatively as the MQTT interface is fully documented throughout it can be simply be adapted to an existing HA system via MQTT publications/subscriptions.

At it's heart is an ESP8266-12E which is responsible for MQTT communications and controlling all functionality excepting LCD display back-lighting control and gesture sensing which use an ATTiny85 and ATMega328P respectively. The device is fully configurable via text files stored on a local SD card, though calibration and network security parameters can also be programmed via remote MQTT publications.

What parts do I need?

See the bill of materials attached below

What software do I need?

Arduino IDE 1.6.9,
Arduino IDE configured to programme the ESP8266-12E (same as this). Then configure the IDE as indicated in the detailed description provided in the software sketch,
Microsoft Excel if you want to make your own icons for the LCD,
Python v3.5.2 if you wish to use the automated test capability,
Sigrok PulseView 0.5.0-git-b732b48 if you want to capture the gesture sensor control logic in action.
Kicad v4.0.7 if you want to have the PCB manufactured.

What tools do I need?

Microscope at least x3 (for SMT soldering),
Molex connector crimping tool,
SMD soldering Iron (with liquid flux pen and flux cored solder),
Screwdrivers (various),
Heat gun,
Drills (various),
Countersink handtool,
Files (various),
Dremel (various bits),
Sturdy vice (small and large, like a black and decker work mate),
Scalpel,
Vernier calipers (used to measure fabrication and useful for sizing PCB components),
Spanners and Nut drivers (various),
Strong tweezers (for SMT soldering),
Junior Hacksaw,
Drill (with various drill bits),
Fine pliers (point and snub nosed),
Flush cutters,
DMM with audible continuity check,
24MHz 8 Channel USB Logic Analyzer Saleae 8 CH Logic Analyzer for MCU ARM FPGA (for use with Sigrok PulseView).

What skills do I need?

A lot of patience,
A great deal of manual dexterity and excellent hand/eye coordination,
Excellent soldering skills,
Excellent fabrication skills,
The ability to visualise in 3 dimensions,
Some knowledge of software development with 'C' (if you want to understand the source code),
- The project contains some interesting code using; variable function arguments (kind of implementation of fscanf, missing from ESP8266 library), linked lists, pointers to pointers for generic file reading, multiple timers with call back routines via function pointers, multiple state machines and a very simple cooperative RTOS, direct display memory reads from LCD over I2C, gradual fading of LCD back-light using easing function controlled PWM, integration of non-blocking implementation of gesture sensor.
Some knowledge of Python (how to install and run scripts, if you want to use the automated testing),
Knowledge of Arduino and it's IDE,
Good knowledge of electronics,
Knowledge of how to layout a PCB using a cad tool (Kicad), if you want to have the PCB manufactured,
Some understanding of your home network.

Topics covered

User Manual
Circuit Overview
PCB Manufacture
Software System Overview
Software Overview
Sensor Calibration
Gesture Control
MQTT Topic Naming Convention
OpenHAB Configuration
Testing the Design
Conclusion
References Used

Series Links

To Part 10 : IR REMOTE CONTROL VIA IOT. PART 10 IOT, HOME AUTOMATION

To Part 12 : Retro Speech Synthesis Part 12 IoT, Home-Automation

BOM.xlsx
Download

Step 1: User Manual

This section describes the functionality of the IoT Desktop Device in detail from a users perspective. The video above shows the system in use and demonstrates the various functionality in action.

Pictures 1 through 3 name the main component parts of the IoT Desktop Device

Buttons

From left to right.

1st button 'Open Logging' initiates a logging session. This functionality can be controlled remotely by issuing the following MQTT publication; 'WFD/THBSen/7/Logging/Command/1', Payload '1'
2nd button 'Close Logging' closes off the current logging session. This functionality can be controlled remotely by issuing the following MQTT publication; 'WFD/THBSen/7/Logging/Command/1', Payload '0'

Logging is made to the integral SD card in csv format. The name of the logging file is derived by concatenating the Day=DD, Month=MM, Hour=hh,Minute=mm of the local real time clock. ie DDMMhhmm.csv.

The following local parameters are logged; Date, Time, Temp Local, Humidity Local, Barometric Pressure, Ambient Light Level.

3rd button 'Button 0' is user programmable and has been designed to act in a bi-stable manner. ie. Press once for 'on' and press again for 'off'. For each change of state a user defined MQTT topic and payload is published.
4th button 'Button 1' is also user programmable however in this instance the associated user defined MQTT topic/payload is published as a one shot each time the button is depressed. No action results when the button is released.

Configuration of these buttons is achieved via editing a text file named 'conbutin.txt'. This file is resident on the integral SD card and must contain 6 entries as shown below;

1st : Button0 Pressed Topic (pressed). ie. /OHB/Req/StudyLights/Control
2nd : Button0 Pressed Payload. ie. 1
3rd : Button0 Released Topic (released). ie. /OHB/Req/StudyLights/Control
4th : Button0 Released Payload. ie. 0
5th : Button1 Pressed Topic (pressed). ie. /OHB/Req/StudyLights/Control
6th : Button1 Pressed Payload. ie. 0

Note : The entries are in the form of MQTT Topics and their respective payloads. If an entry is not required then the key word 'null' must be placed in both Topic and Payload positions

Ambient Light Sensor

Located on the top of the enclosure in the middle of the buttons, this device monitors local ambient light levels and publishes them via the topic 'WFD/THBSen/7/ALSStatus/1'. Units are in Lux.

The ambient light sensor is also used to trigger a change in System LCD illumination. As the local light levels increase above a user definable threshold the brightness of the display will also increase to give better contrast.

The converse is also true for decreasing light levels.

Configuration of these parameters is achieved via editing a text file named 'confvals.txt'. This file is resident on the integral SD card and must contain 7 entries as shown below;

1st Display Ambient Light Sensor Upper Threshold Value. In Unsigned Long form.
2nd Display Ambient Light Sensor Lower Threshold Value. In Unsigned Long form.
3rd Display Delay Before Change Value. In Unsigned Long form.
4th Display Back-light Upper Value. In Unsigned Long form.
5th Display Back-light Lower Value. In Unsigned Long form.
6th Display Delay Before Scroll Value. In Unsigned Long form.
7th Enable gesture control. In boolean form 0 = GC off, 1 = GC on

Note : Entries 6 and 7 are general configuration items used to set the delay before an enumerated sensor value will scroll and to enable gesture control respectively.

Gesture Sensor

The IoT Desktop Device is also fitted with a simple gesture detection sensor on the front of the enclosure which is capable of reconising the following; Left, Right, Up, Down and Near.

Left forces a left scoll of the list enumerated sensor values
Right forces a right scoll of the list enumerated sensor values
Up triggers a user configurable MQTT topic publication
Down triggers a user configurable MQTT topic publication
Near forces the display of the local temperature and humidity levels.

Configuration of these parameters is achieved via editing a text file named 'congesud.txt'. This file is resident on the integral SD card and must contain 4 entries such as shown below by way of example;

1st : Up Gesture Topic. ie. /OHB/Req/StudyLights/Control
2nd : Up Gesture Payload. ie. 1
3rd : Down Gesture Topic. ie. /OHB/Req/StudyLights/Control
4th : Down Gesture Payload. ie. 0

System Display

This display presents the user with 9 items of information as shown in picture 4 above;

Logfile name

When not logging the display shows the words 'NO LOGGING'. Once logging has been initiated the display generates file name as in the description for the 2nd button and starts logging local ambient values to a csv file on the SD card.

Connectivity Icons

These icons indicate the various states of IoT network connection. See picture 5 above. Here you can observe there are 5 icons ascociated with 3 connection states.

AP mode : If the IoT Desktop device fails to connect to your WiFi network it will become and access point on it's own network. By doing so it will allow you to redefine the parameters necessary to connect.

MQTT Broker : This shows two states, connected and not connected. Helpful in determining if there is an issue with the MQTT Broker on your IoT network

WiFi Network : This also shows two states, connected and not connected.

System time

This is the time as read from the on-board real-time clock presented in 24hr format

Barometric pressure

This is the local barometric pressure reading as taken from the internal BMP085 barometric pressure sensor in millibars (mB). The value of barometric pressure is published by the device via the topic 'WFD/THBSen/7/BarometricStatus/1'.

Barometric trend

This gives a simple indication of the barometric trend; rising, stable or falling. See picture 5 above for the actual icons.

System date

This is the date as read from the on-board battery backed real-time clock.

Enumerated sensor name

This is the user assigned name for a given temperature or humidity subscribed topic. Here the user makes the association with say, a temperature and the Hallway, or humidity and Master Bedroom. In each case, once the display cycles through it's list of subscribed temperature and humidity topics, the enumerated sensor name is also displayed via association.

By doing so it makes the sensor information more readable to the user. A typical entry in the 'consenin.txt' configuration file would be;

1st : Enumerated sensor name. ie Garage
2nd : Temperature Topic. ie. WFD/THSen/1/TempStatus/1
3rd : Humidity Topic. ie. WFD/THSen/1/HumdStatus/1

Temperature

The temperature of the associated subscribed topic paired with an enumerated sensor name. In degrees Celsius.

Humidity

The humidity of the associated subscribed topic paired with an enumerated sensor name. In %age of relative humidity.

SD Card

This holds all configuration data and web pages when in AP mode. Configurations files are as follows;

- calvals.txt : This file contains the calibration entries for the temperature, humidity and barometric sensors.
- secvals.txt : This file contains the parameters necessary to connect to your WiFi network and MQTT Broker.
- confvals.txt : This file contains the configuration parameters for sensing of ambient light and triggering the resulting display illumination levels.
- consenin.txt : This file contains configurable entries which enable the unit to subscribe to two topics Temperature and Humidity and link these topics to an enumerated sensor name.
- congesud.txt : This file contains configurable gesture entries which are matched to Up and Down gestures respectively.
- conbutin.txt : This file contains configurable button entries for Button 1 and 0 respectively.

Leds 1 and 0

These are controlled remotely and can be used as general indication.

Led 1 : This led is controlled by publishing a payload of '0' or '1' to the following topic 'WFD/THBSen/7/Led/Command/1'

Led 0 : This led is controlled by publishing a payload of '0' or '1' to the following topic 'WFD/THBSen/7/Led/Command/2'

Led 0 has special functionality. During the initialisation phase at start up, the led flashes to indicate the state of the connection or the failure code if a connection can't be made.

Note : Local Temperature, Humidity and Heat Index values are published via the following topics respectively;

WFD/THBSen/7/TempStatus/1
WFD/THBSen/7/HumdStatus/1
WFD/THBSen/7/HeatIndStatus/1

Step 2: Circuit Overview

A full circuit diagram of all the electronics is given in picture 1 above, along with a detailed PDF below.

The desktop device contains three processors;

IC4 : ESP8266-12E

As mentioned above this processor is responsible for controlling the main functionality of the desktop device along with the WiFi back-link into the network and onward connection to the MQTT Broker.

This processor is directly connected to the integral SD card reader and externally mounted DHT22 temperature/humidity sensor for ease and speed of operation. As I/O on the ESP8266 is limited, extra I/O is provisioned via the on-board I2C interface on GPIO Pins 5 (14) SCL and 4 (13) SDA.

This I2C bus connects the following devices;

IC1 : PCF8574A (address 0x3E), Generic I/O which connects to the 4 button inputs and two led outputs. This device is 3v3 compatible and as a consequence uses T5 and T6 to interface to the two blue led outputs which require 4.5v to illuminate off the 5v rail. Note T5 and 6 are FETs in order to allow for a ballast resistor to be used (R15, R16). A bipolar transistor would not work as there would be a Vbe of around 0.6v in saturation.
IC6 : PCF8574A (address 0x3F), Generic I/O which ultimately connects to U3 to form the non-blocking gesture detection system. Described in detail below.
IC2 : DS1307Z (address 0x68), This is a battery backed real-time clock, which is 5v compatible.
Barometric Pressure Sensor Module, J6 : BMP085 (address 0x77, 0x1E). Supplied with 5v, but is 3v3 compatible via an on-board 3v3 regulator.
LCD Module, J8 : PCF8574A (address 0x27), This generic I/O device converts serial (I2C) to parallel (4 bit parallel operation of the HD44780 LCD) which is 5v compatible. See here for further information.
Ambient Light Sensor Module, J9 : BH1750 (address 0x23). Supplied with 5v, but is 3v3 compatible via an on-board 3v3 regulator.
U1 : ATtiny85, PWM controller for LCD back-light control. Described in detail below.
U3 : ATMega328P, Non-blocking implementation of gesture sensor. Described in detail below.

U1 : ATTiny85

This processor forms an I2C interface (address 0x08) to the system LCD which allows for gradual fading of the back-light and is based on the two following instructables, here and here. I non-destructively reverse engineered the PCF8574A Serial to Parallel I2C Module design and discovered it was possible to control LCD back-light intensity by connecting directly to the two LED pins on the module (marked here as X-Y connector in picture 2 above, or J3 in pic 3). Transistors T1, T2 and resistors R18, R19 form the control circuitry which enable PWM of the LCD back-light.

Given smooth fading with easing functions is processor intensive I ported the control of fading to an ATtiny85.

U3 : ATMega328P

This combination of generic I/O IC6 connected to U3 forms the non-blocking gesture detection system (see here for further details of implementation) and is 3v3 compatible. Connection to the APDS9960 is via J14 which is supplied with 5v, but is 3v3 compatible via an on-board 3v3 regulator. This device also has an I2C adress of 0x39, though this is not exposed to the ESP8266, being directly connected to U3. Picture 4 above shows a logic analyser plot of the gesture sensor in action.

The capability to reset the gesture sensor electronics was included in hardware (R37, R42, R36, D2, C18 and T4), but has not been implemented in software as the subsystem has remained reliable since operation over some months.

Mixed supply rail interface (3v3, 5v)

FETs Q1 and Q2 provide the 3v3 - 5v interface for the I2C communications. Pull up resistors ROpt1...4 were included just in case, but not used in the final design as the modules provisioned sufficient coupling to the respective supplies.

In-circuit programming

The electronics was designed to allow for the in-circuit re-programming of the ESP8266 and ATmega328P for easier debugging;

IC4 ESP8266 : In-circuit programming is provisioned by J5, S5 and S6.
U3 ATMega328P : In-circuit programming is provisioned by J13, with automatic reset provided by R42, C18 and D2 which enables the Arduino boot loader to function directly from the Arduino IDE.

Mixed supply rails

The desktop device requires the use of both 5v and 3v3 supply rails.

Supply to the device is first conditioned via an SMS buck converter which efficiently drops a wide range of supply input (9v - 15v) to 7.6v, decoupled via C2, this is fed directly to;

IC3 an LD1117S50TR a 5v LDO linear voltage regulator, mounted on-board the main PCB. Supply rail decoupling is provided by C4, C6 and C20.
J7 an LDV1117v33 a 3v3 LDO linear voltage regulator, mounted externally to a 21.6 °C/W heat sink to ensure no heating effects are felt by the locally mounted sensors. Supply rail decoupling is provided by C1, C5 and C21.

IoTDeskTopDevice.pdf
Download

Step 3: PCB Manufacture

So far all of my electronics Instructables have been prototyped on solderless breadboard as proof of concept and constructed on veroboard in final design.

It's always been my intention to commit my designs to PCB but the block to this was lack of good CAD tools (by good I mean cheap) and the prohibitively expensive cost of PCB manufacture in the UK.

I had initially purchased a perpetual 'EAGLE HOBBYIST' user licence for around £100 with the intention to use this toolchain for both schematic capture and PCB design. Things were looking good, I noticed Adafruit and Farnell were creating footprints for all their parts so in theory it would be relatively easy to put together a design sourcing the components from Farnell and using a local (but expensive) PCB manufacture house I'd located.

Before writing this Instructable, all my schematics were done with Eagles, though I hadn't as yet gone so far as to create a PCB partly down to not fully understanding the design flow and there not being a vast amount of training collateral available. I did go so far as to purchase Mitchell Duncan's 'Learning to fly with Eagle' (despite it taking forever to get published) and Simon Monk's 'Make your own PCBs with Eagle', but to-date they remain unused, in pristine condition.

However, with the recent takeover of CadSoft by Autodesk all this changed, despite all the usual dishonest reassurances big business offer they reneged and moved over to a subscription based model, something I really dislike. Rightly or wrongly my view is when I purchase software I own it in perpetuity. It's up to the software company to come up with a model that fits, or I simply won't purchase your product.

Consequently I was back to square one.

However all this recently changed after I noted a great deal of interest in Kicad. After much 'Googling' I found a plethora of information, User Forums, YouTube videos (specifically the truly outstanding work by Chris Gammell at 'Contextual Electronics') etc. All my questions were answered and designing a PCB was actually quite easy.

I now use Kicad for both my schematic diagrams and PCB design. See Pics 1 and 2 above.

Ok, so now I had a good tool chain, plenty of instruction and training but I still needed to use an expensive UK manufacturer to get the PCB fab'ed.

That is until JLCPCB arrived on the scene...

This company seems to have come from 'left field', offering an insanely cheap price for a dual sided PTH PCB for $2 and is now within easy reach of the hobbyist.

Ok the $2 thing is for your first order and there are some restrictions on size, but still they offer outstanding value for money.

Pics 3...6 show my JLCPCB delivery. The box was a bit worse for wear but the packing protected the contents. As you can see I received a pack of 5 PCBs, double sided, plated through hole with silk screen and green solder mask. The thinnest trace I am using is approx 8mil to get the density I needed (this was my first board and it made routing easy).

I chose the basic board package and followed the instuctions and (video) on how to generate and upload your Kicad Gerber files. Once uploaded I was presented with a 'zoomable' image of my board which gave me a 'warm feeling' everything was ok. It was almost like they wanted my business (strange concept in the UK). I paid with paypal and watched how my boards were progressing through the production cycle via my account login.

Just over a week later my boards arrived through the post and the rest is history.

Pic 7 shows my set up for populating the PCB. I used a microscope as my eyes aren't as good as they used to be, and for the most part I used SMT components which are tiny.

Pics 8 to 10 show the board fully populated and the DIL components inserted.

Finally Pic 11 shows the board fully installed and wired up in it's enclosure.

Board Layout

All connections to the board are made via 0.1" Molex connectors to facilitate assembly/disassembly. The RTC and Barometric sensor are located away from the ESP8266 RF source to minimise the risk of interference. The on-board +5v regulator is soldered to a large thermal relief to give better dissipation under operation. I used oversized pads for the 16MHz Xtal on the ATMega328P to allow me to flow solder to all four corners of this SMT device.

Note : A full copy of the PCB and schematic are attached below.

IoTDeskTopDevice.zip
Download

Step 4: Software System Overview

This IoT Desktop device contains six key software components as shown in pic 1 above.

SD Card

This is the external SD SPI Flash Filing System and is used to hold the following information (see pic 2 above);

Icons and 'IoT Desktop Device Home Page' html: Served up by the IoT device when it is unable to connect to your IoT WiFi network (usually due to incorrect security information, or first time use) and provides the user with a means of remotely configuring the sensors without the need to re-flash new SD content.
Security Information: This holds the information used at power up by the IoT device to connect to your IoT WiFi network and MQTT Broker. Information submitted via the 'IoT Desktop Device Home Page' is written to this file ('secvals.txt').
Calibration Information: The information contained within this file ('calvals.txt') is used to calibrate the on-board temperature/humidity and barometric pressure sensors should it be necessary. Calibration constants can be written to the IoT device via MQTT commands from an MQTT broker or by re-flashing the SD card.
User programmable gesture detection aliases: The information contained within this file ('congesud.txt') allows the user to associate gestures with the publication of IoT payloads to a given topic.
User programmable button press aliases: The information contained within this file ('conbutin.txt') allows the user to associate system button presses with the publication of IoT payloads to a given topic.
User configurable system values: The information contained within this file ('confvals.txt'), chosen by the user, controls certain system responses, such as ambient light levels that cause the display back-light illumination to adapt to it's environment or display sensor scroll rate etc.
User configurable enumerated sensor entries: The information contained within this file ('consenin.txt'), is an association between a logical name and topics to subscribe to, to receive temperature and humidity publications for this logical device.

mDNS Server

This functionality is invoked when the IoT device has failed to connect to your WiFi network as a WiFi station and instead has become a WiFi access point something akin to a domestic WiFi router. In the case of such a router you would typically connect to it by entering the IP Address of something like 192.168.1.1 (usually printed on a label affixed to the box) directly into your browser URL bar whereupon you would receive a login page to enter the username and password to allow you to configure the device. For the ESP8266-12E in AP mode (Access Point mode) the device defaults to the IP address 192.168.4.1, however with the mDNS server running you only have to enter the human friendly name 'DSKTOPSRV.local' into the browser URL bar to see the 'IoT Desktop Device Home Page'.

MQTT Client

The MQTT client provides all the necessary functionality to; connect to your IoT network MQTT broker, subscribe to the topics of your choice and publish payloads to a given topic. In short it provisions IoT core functionality.

HTTP Web Server

As mentioned above, if the IoT device is unable to connect to the WiFi network whose SSID, P/W etc. is defined in the Security Information file held on the SD Card the device will become an Access Point. Once connected to the WiFi network provided by the Access Point, the presence of an HTTP Web Server allows you to directly connect to the device and change it's configuration via the use of an HTTP Web Browser it's purpose being to serve up the 'IoT Desktop Device Home Page' web page which is also held on the SD Card.

WiFi Station

This functionality gives the IoT device the capability to connect to a domestic WiFi network using the parameters in the Security Information file, without this your IoT device will not be able to subscribe/publish to the MQTT Broker.

WiFi Access Point

The ability to become a WiFi Access Point is a means by which the IoT device allows you to connect to it and make configuration changes via a WiFi station and a browser (such as Safari on the Apple iPad). This access point broadcasts an SSID = "DSKTPDEV" + the last 6 digits of the MAC address of the IoT device. The password for this closed network is imaginatively named 'PASSWORD'

Step 5: Software Overview

Preamble

To successfully compile this source code you will need a local copy of the code and libraries outlined below in Step 12,References Used. If you are not sure how to install an Arduino library go here.

Overview

The software makes use of the state-machine as shown in pic 1 above (full copy of source here). There are 5 main states as outlined below;

INIT
- This initialisation state is the first state entered after power up. During this state the enumerated sensor name file ('consenin.txt') is read with a call to 'readConfigurableSensorInput();'. This enumerated sensor name versus temperature/humidity topic is loaded into a linked list, a dynamic data structure for use when connected/active. See Linked list below.
NOCONFIG
- This state is entered if after power up an invalid or missing secvals.txt file is detected
PENDING NW
- This state is transitory, entered whilst there exists no WiFi network connection
PENDING MQTT
- This state is transitory, entered after a WiFi network connection has been made and whilst there exists no connection to an MQTT broker on that network.
ACTIVE
- This is the normal operational state entered once both a WiFi network connection and an MQTT Broker connection has been established. It is during this state the ambient light level, temperature, barometric pressure, heat index and humidity at the IoT Desktop Device is regularly published to the MQTT Broker.

The events controlling transitions between states are described in pic 1 above. Transitions between states is also governed by the following SecVals parameters;

1st MQTT Broker IP Address. In dotted decimal form AAA.BBB.CCC.DDD
2nd MQTT Broker Port. In Integer form.
3rd MQTT Broker connection attempts to make before switching from STA mode to AP mode. In Integer form.
4th WiFi Network SSID. In free form text.
5th WiFi Network Password. In free form text.

As mentioned above if the IoT device is unable to connect as a WiFi Station to the WiFi network who's SSID and P/W is defined in secvals.txt held on the SD Card the IoT device will become an Access Point. Once connected to this access point it will serve up the 'IoT Desktop Device Home Page' as shown above in Pic 2 (by entering either 'DSKTOPSRV.local' or 192.168.4.1 into your browsers URL address bar). This home page allows the reconfiguration of the IoT Desktop Device via an HTTP browser.

Remote Access whilst in the ACTIVE state

Once connected to the MQTT Broker it is also possible to both re-calibrate and reconfigure the device via MQTT topic publications. The file calvals.txt has R/W access and secvals.txt has write only access exposed.

User debug

During the boot sequence the IoT device blue System led 0 gives the following debug feedback

1 Short flash : No Config file located on SD Card (secvals.txt)
2 Short flashes : IoT device is attempting to connect to WiFi network
Continuous illumination : IoT device is attempting to connect to MQTT Broker
Off : Device is active.

The above visual indication is a legacy of the first implementation of IoT connectivity. In this instance as the IoT desktop device also possess an LCD there are 3 connectivity icons which are also updated and the device goes through its various states of connection (pic 3 above).

IoT Desktop Device Functionality in ACTIVE State

Once in the ACTIVE state the ESP8266 enters a continual loop calling the following functions; timer_update(), checkTemperatureAndHumidity(), checkBarometricPressure(), checkALSValue(), updateTimeDate(), readAPDS9960(boolean *bGestureAvailableFlag, uint8_t *uiGestureValue), handleGesture(), updateDisplay(), checkButton(), updateLogging(). The net result of which has been designed to both present the user with parametric environmental data (which is regularly published) and provision a means of interaction/control with other devices via MQTT publications in an IoT infrastructure.

A comprehensive list of all topic subscriptions and publications including payload values is included in the source code.

Use of linked lists
A linked list is a dynamic data structure, similar to an array, differing in so far as it is created with no elements and dimensioned whilst the programme is running. This means the size of the array does not have to be known when the source code is compiled. A linked list (or more accurately a doubly linked list in this case, 'ptrHeadOfSensorInstances') is used here to allow the addition of enumerated temperature/humidity sensor names and their associated topics without the need to recompile the source code. All the user needs to do is remove power, extract the SD card, edit the 'consenin.txt' file and re-insert. Once the IoT desktop device power supply is reconnected the software will re-read 'consenin.txt' and add the enumeration of interest to the linked list. During normal operation in the ACTIVE state, the software passively listens for temperature and humidity publications associated with these topics. When one is received the corresponding temperature/humidity value is dynamically updated in the linked list. Concurrent with this activity, the software periodically sequentially scans the linked list and updates the system LCD with the enumerated sensor name along with the stored value for temperature and humidity. As the linked list is circular the pointer used to scan the list ('tmpSensorInstancePtr') need only be incremented as incrementing beyond the end of the list will point to the front of the list (ie. the pointer wraps inherently).

Picture 4 illustrates linked lists of various lengths and picture 5 depicts how a new element is added to the end of a linked list (the action at start up).

Simple cooperative RTOS

As mentioned in the introduction, this code also contains a simple implementation of a cooperative RTOS (Non-Preemptive, see pics 6 and 7 above. To achieve this, strictly, none of the code in the loop() function is 'blocking' ie. results in an indefinite execution loop (the implementation for gesture sensing with the APDS9960 was specifically designed with this in mind, see here for further details). Here the functions; timer_update(), checkTemperatureAndHumidity(), checkBarometricPressure(), checkALSValue(), updateTimeDate(), readAPDS9960(boolean *bGestureAvailableFlag, uint8_t *uiGestureValue), handleGesture(), updateDisplay(), checkButton(), updateLogging() are iteratively executed on each successive pass through the loop() giving control back when complete (ie. non-preemptive). The outcome of each repetitive execution is dependent upon state variables or timer call back routines each of which independently changing. In this way the code is effectively handling many pseudo simultaneous tasks.

Step 6: Sensor Calibration

When the IoT device powers up, as part of the boot sequence a file named 'cavals.txt' is read from the SD Card. The contents of this file are calibration constants as indicated above in pic 1. These calibration constants are used to adjust the readings acquired from the sensor to bring them into line with a reference device. There is one further value which defines a reporting strategy for the device and is described below along with the procedure followed to calibrate the sensors.

Reporting Strategy

This parameter determines how the remote sensor reports any ambient parametric changes local to it. If a value of 0 is selected the remote sensor will publish any change it sees in the temperature, humidity, ambient light levels or barometric pressure each time the respective sensor is read (approx every 10 seconds). Any other value will delay the publication of a change by 1...60 minutes. Modifying this parameter allows for optimisation of MQTT network traffic.

Temperature calibration

To calibrate the temperature sensor I followed the same process as outlined here step 4, again using a simple y=mx+c relationship. I used IoT Temperature, Humidity Sensor #1 as the reference device. Values from the sensor are in degrees celcius.

Humidity Calibration

As I possess no means to accurately record or even control local ambient humidity, to calibrate the sensor I used a similar approach to that above here step 4, again using Sensor #1 as reference. However the above said, I have recently found an excellent article on the web describing how to calibrate humidity sensors. I may well try this approach in the future. Values from the sensor are in %age of relative humidity.

Barometric Pressure Calibration

Again as I possess no means to accurately record let alone control local barometric pressure, whilst I added parameters to calibrate zero and scale, they are not used. Values from the sensor are in millibars.

Barometric Trend

Barometric trend is calculated by taking a windowed (MAX_BAROMETRIC_SAMPLES) rolling average of the barometric value read off the BMP085 sensor. The delta between old and new barometric values is determined and scaled '((fNewBarometricTrendAverage - fOldBarometricTrendAverage)/MAX_BAROMETRIC_DELTA)' this 'smoothed' barometric vaue is used to update a 'iBarometricDeltaCount'. A +ve iBarometricDeltaCount results in an upward trend arrow, a zero iBarometricDeltaCount results in a '-' stable icon and a -ve iBarometricDeltaCount gives a downwards trend arrow. Pictures 2 ... 5 give a graphical depiction of the above.

Ambient Light Sensor

The datasheet for this device indicated sufficient accuracy to not require any calibration and therefore I didn't export any parameters in calvals.txt for this sensor. Values from the sensor are in Lux.

Step 7: Gesture Control

As mentioned above the gesture sensing functionality was an integration of the circuit given in this Instructable, pic 1 shows the circuit integrated into the IoT desktop device.

To ensure this gesture sensing worked I used Sigrok PulseView 0.5.0-git-b732b48 along with a '24MHz 8 Channel USB Logic Analyzer' to capture the sequence shown in pic 2 above.

This sequence shows how the IoT desktop device captures a right moving hand gesture;

Gesture Sensing sub-system, New gesture detected, gesture code placed on data lines D0...3
Gesture Sensing sub-system, Gesture Available Line - Active Low
ESP8266 Reads data lines D0...3 and asserts Gesture Clear - Active Low
Gesture Sensing sub-system,Data lines D0...3 cleared
Gesture Sensing sub-system,Gesture Available Line - Inactive High
ESP8266 de-asserts Gesture Clear - Inactive High

Step 8: MQTT Topic Naming Convention

As mentioned in an earlier Instructable (here) I settled on the topic naming convention outlined in pic 1 above.

Namely, 'AccessMethod/DeviceType/WhichDevice/Action/SubDevice'

It's not perfect but it does allow for useful filters to be applied to see all sensor outputs for a given parametric topic thus allowing for easy comparison as in pic 2 above with MQTTSpy.

It also supports reasonably extensible logical groupings of functionality within a given IoT device.

In implementing these topics in software I used hard coded topic strings with fixed, embedded numerical identifiers for each device as opposed to dynamically generating the topics at run time so as to save on RAM and keep performance high.

Note : If you're not sure how to use MQTTSpy see here 'Setting Up an MQTT Broker. Part 2 : IoT, Home Automation'

Step 9: OpenHAB Configuration

I modified the OpenHAB configuration given in my earlier IoT Instructables (here, here, here and here) adding in the new functionality pics 1...2, along with barometric trend, second ambient light trend, RSSI (Received Signal Strength Indication) trend in pics 3...5 above.

If you wish to use the 'stevequinnhousehold' example all you need to do is download the configuration files from here and drop them into the OpenHAB directories shown above in pic 6.

Note : If you're not sure how to use OpenHAB see here 'Setting Up and Configuring OpenHAB. Part 6 : IoT, Home Automation'

Step 10: Testing the Design

Testing was carried out using two methodologies;

The first was using MQTT Spy to exercise all of the available subscribed topics and check the published responses (depicted in pic 1 above). As this was achieved manually it can be time consuming and prone to errors, although manual execution does enable 100% coverage.

MQTTSpy was chosen because it is an excellent tool to hand format a given payload and publish it to any topic with ease. It also displays a clear log which is very useful for debugging (pic 2 above).

The second approach was adopted as the source code became more complex (>4000 lines). Increased complexity means longer manual testing cycles and more complex tests. In order to improve the reliability, determinism and quality of tests, automated testing was used via a python test executive (pic 3).

See Step #10 in this Instructable on how automated testing was introduced.

A full copy of the automated tests used in this Instructable is available here.

A video of the automated test sequence in operation is shown above. The sequence executes the following steps;

Query software version
Cycle Led 1
Cycle Led 2
Parametric check of LCD Read function
Check all values in 'consenin.txt' have been parsed and added to the internal linked list
Check temperature update via reception of publication from a subscribed topic
Check humidity update via reception of publication from a subscribed topic
Check clock increments on display
Check clock rolls over at midnight
Check date rolls over at midnight
Check logging works
Check logging period updates

Note : If you want to use the above automated testing you will need to use the LiquidCrystal_I2C_PCF8574 Arduino library. As far as I know, this is the only library which allows the user to directly read both DD and CG ram in the LCD memory over I2C.

Step 11: Conclusion

Overall I think project was a success on many parts.

The finished article is aesthetically pleasing,
The software works well and has been up and running robustly for many months,
The system integrates well with my current home automation environment,
The electronics work flawlessly,
The final packaging had no issues with relative proximities to other devices,
Smooth fading of the LCD back-light just gives the finished device a better look and feel,
Finally, I got to create a working PCB.

In general

The layout worked well, giving good access to all connectors, ESP8266 programming buttons and CR2032 battery carrier.

The circuit worked 'out of the box' due in most part to all the prototyping I had carried out along with many measurements to ensure everything fitted in. I'm particularly proud of the fit of the BMP085 module which sits between two Molex connectors and overhangs IC6 a PCF8574A I/O expander. Though I think with hindsight I should have integrated the barometric sensor directly on the board rather than use a module.

One thing to note when combining many I2C modules is, invariably the Chinese manufacturer will provision the module with 10K/4K7 pull up resistors attached. By combining many of these devices you are effectively paralleling them up and increasing the bus load. You may find your I2C stops working. If this happens, remove some of the pull-ups.

In my design I had fully prototyped everything so I knew there would be no issues. However to make sure I had 'belt and braces' I also added in pads to my level converter to allow me to remove all module based pull ups and add my own. It turned out I didn't need them in the end.

A note on gesture sensing

The gesture sensor is very precious regarding it's unobstructed view on the world. I had to 'clip off' part of the CS Nylon screws to give better clearance. Also make sure you get the same sensor I chose which only has the detector on the underside of the pcb module, this allows you to get the sensor as far into an aperture as possible. Most other sensors I found had surface mount components clustered on the same side as the detector. This is fine for prototyping but precludes embedding in any final design.

Future development

Could add some way to set the RTC manually via the system buttons where it is not possible to connect to an IoT network.

What would I have done differently?

Maybe next time I could use a graphics display and not a standard character LCD. Though the character LCD works fine, I think a graphics display would have more visual impact, despite the extra code and RAM usage. It can also give more real estate to include extra data such as home internet speed, YouTube views or tweets etc.
I ended up with a lot of cables and connectors to route I/O to and from the enclosure front panel. In future I would look to find a way to reduce the volume of cables. Perhaps use some form of data bus between the main board and panel.
I would also look to integrate my own SMPS rather than use an external module, this would obviate the need for a 2-way inline Molex connector between the power jack and the SMPS subsystem.
As this was the first time I had used Kicad I chose the standard CR2032 footprint. To be honest I was disappointed with the mismatch between the pads and the silkscreen. In future I will tweak this model to be a better fit.
Pay more attention to design for automated testing. The level of complexity of the software within a seemingly simple IoT device in reality requires thorough testing if the device is to be relied upon for consistent operation. Manual testing is not sufficient, prone to errors and time consuming.

Step 12: References Used

The following sources were used to put this Instructable together;

Source code for the IoT Desktop Device (this contains a copy of everything)

https://github.com/SteveQuinn1/IoT_Desktop_Device

PubSubClient.h

By : Nick O'Leary
Purpose : Enables the device to publish or subscribe to MQTT topics with a given Broker
From : https://github.com/knolleary/pubsubclient

BounceI2C.h

By : Steve Quinn
Purpose : Allows switch debouncing over an I2C input device. This is a GITHub fork from Bounce2.h by Thomas O Fredericks.
From : https://github.com/SteveQuinn1/BounceI2C

DHT.h

By : Adafruit
Purpose : Arduino library for DHT11DHT22, etc Temp & Humidity Sensors
From : https://github.com/adafruit/DHT-sensor-library

BH1750.h

By : claws
Purpose : An Arduino library for the digital light sensor breakout boards containing the BH1750
From : https://github.com/claws/BH1750

DS1307RTC.h

By : Paul Stoffregren
Purpose : An Arduino library for DS1307 Real Time Clock chip with the Time library
From : https://github.com/PaulStoffregen/DS1307RTC

LiquidCrystal_I2C_PCF8574.h

By : SteveQuinn
Purpose : An Arduino library for an I2C based LCD interface using the PCF8574 I/O expander
From : https://github.com/SteveQuinn1/LiquidCrystal_I2C_PCF8574

Adafruit_BMP085.h