Introduction: CyberSafe : Your Personal Cloud IoT Platform

By plus.google.com/+SatyavratWagle

Turn your old Raspberry Pi into a Private, Cloud connected IoT platform like Thingspeak or PubNub! Stream your sensor data seamlessly!

The buzzword for the past couple of years has been IoT, and major corporations have been quick to jump upon the bandwagon. For software giants like Microsoft, Google and IBM, the logical step forward was creation of Platforms as a Service or PaaS. If this term seems like unintelligible technical jargon, think Microsoft Azure, Amazon Web Services, Google Analytics and IBM BlueMix.

If Amazon and Microsoft can do it, why can't we?
So this project will be showing you how to design your own platform as a service based on your Raspberry Pi. This system will allow you to stream sensor data over WiFi and log it in a database on the Raspberry Pi. You will also have a GUI which will display current values of the sensors and also plot a graph of those values when you click a button.

PS : I've entered this project into a few competitions, if you like it, please do vote for it in the top right corner of the post.
Thanks a lot!

Step 1: Why Should I Do This Project?

You might wonder why, if there already exist platforms such as these, should you go through all the trouble of making your own platform. Well, there are certain compelling reasons why you should

  1. No data/data rate limit : Many of these platforms are built to cater to a vast number of people, and no matter how large the servers might be, there have to be certain limitations on how much data you can send per unit time, to muzzle the overhead to a manageable level. That might be one request per second up to 15 seconds as well.This project allows real time data transfer, without any rate limit, because you're the only one who can use it.
  2. Customizable : Cloud computing has certain qualities when it comes to describing a PaaS, one of them being the type of applications that can be run on them. This one, being entirely your own can be morphed to do just about any task which you might have.
  3. No Proprietary or Fixed APIs : This has always been sort of a headache to me. The provided APIs, which to a certain degree are fixed. You need to modify your entire code to accommodate these APIs and have it run properly. In this case, you will be able to design your own APIs, and access the data independently. This however, is out of the scope of this project (sorry!)
  4. It's FUN! : What's life without a challenge? This project is a great technical and mental exercise if nothing else!

Step 2: Caveats and Corollaries

That being said, I don't have billions of dollars to spend on platform development, so there are certain limitations to this project, which are;

This project is built based on a local sensor network, meaning that the Raspberry Pi and Sensor Node are on the same network. This can be overcome by a process called port forwarding. I'll go through it briefly in the tutorial.
WARNING : This process causes your data to leave your local network and is transported over the Internet unencrypted. Please make sure you don't send sensitive data via this procedure. I take no responsibility for any damage which may occur as a result (*sigh of relief*)

The size of your database depends largely on the size of the SD card mounted on the Raspberry Pi. For my experiment, I have used an 8 GB card to store sensor data over 24 hours, taken at 1 minute intervals. So, if you have a huge database, you might want to mount a larger SD card.

Finally, this platform is quite literally, a platform for you to build upon. Any analytics, machine learning algorithms or signal processing techniques you might want to use can be implemented using the vast variety of Python libraries built for those purposes. The main goal of this project is to design your ad-hoc platform and to make data available for further calculations.

Step 3: Prerequisites

Before doing this project, please make sure you have done the following

Have a MySQL server installed and running on your Raspberry Pi Connected a Raspberry Pi and an Edison to your home network

Assigned static IPs to your Raspberry Pi and Edison (not strictly necessary, but very helpful in the long run.)

Required Software
Apart from the materials mentioned in above, this is a brief description of the programming modules we will be using and why.

The programming language we will be using in this project is Python. I chose this language because it is prototyping-friendly while not being to constrained. The following Python libraries will be used.

sockets : This library allows low level networking via the TCP/IP protocol. This is the carrier protocol for our messages over WiFi.
MySQLdb : This library allows you to check and modify contents of a MySQL database from Python. The MySQL database will be where all your sensor readings will be recorded.
matplotlib : This library allows for MatLab-like functionalities on any system running Python. We will be using it to visualize our data.
Tkinter : This library is a favorite for GUI creation. We will use this library to create a GUI on the Raspberry pi to view our sensor data.
mraa : This library is not strictly necessary; however, as I am using an Intel Edison as the sensor node, I will need this library for GPIO access on the node side.Have the following libraries installed on your Raspberry Pi

Step 4: Component Descriptions

Raspberry Pi : The Raspberry Pi is used as the aggregator node in this project, and is used for data collection from the various nodes connected to it over TCP/IP. It parses the recieved messages to extract meaningful sensor data and then stores it within a local MySQL database. It is also used to provide a GUI for the PaaS, which provides functionalities for remote monitoring of data, charting graphs for the same.

Intel Edison : The Intel Edison is used as a remote sensor node which collects data from the environment. It then packs this data into a frame format which is then transported to the data aggregator node over TCP/IP

Sensor Hub : The sensor hub comprises of 3 different sensors which are used to measure environmantal parameters as a function of voltage. The sensors are Light, Temperature and Humidity sensor.

Step 5: Project Flow

In the entire project, there are three independent processes going on. They are;

  • Sensor data acquisition and transport from Edison to Raspberri Pi.
  • Parsing and logging the data in a MySQL database on the Raspberry Pi.
  • Selecting and displaying the data on the GUI as values or a graph.

The reason I say they are independent is that they are asynchronous and the procedures for any step is not dependent on any other steps. All the steps only change database values.

Following is an overview of the steps taken in each process

Sensor Data Acquisition on the Edison

  1. Connect a socket to the Raspberry Pi's IP address and port Begin communication over TCP/IP
  2. Read sensor data from sensor hub board connected to the analog out pins of the Intel Edison
  3. Frame the data with tags and appropriate identifiers
  4. Send the data to the Raspberry Pi
  5. Wait for 60 seconds
  6. Return to step 3

Parsing and Logging Data in the MySQL Database

  1. Open a socket and listen to the selected port
  2. When a packet is received, store in buffer
  3. Split the packet according to delimiters to obtain Source, Type and Value
  4. Connect to database Store the Value in appropriate log of required Source and Value
  5. Update the Value for appropriate table
  6. Commit changes
  7. Close connection to database
  8. Return to step 1

Select and Display Data on GUI

Create a 6x3 window

  1. Display preliminary text
  2. Wait for 2 seconds
  3. Connect to database
  4. Get updated values from update table
  5. Update values on GUI
  6. Return to step 3

If Plot Graph button is pressed

  1. Connect to database
  2. Get all Values for selected Type and Source
  3. Store all values in an array
  4. Plot values on a graph
  5. Display graph

MySQL Databases

The MySQL databases will be the workhorses for this project along with TCP/IP. They will be storing all the sensor data that is sent by the sensor node. We will be using two databases, one for Latest Values, and one for Continuous Logging

The latest values database will only contain the Source, Type and Value, and the continuous logging will include additional index and timestamp fields.

Communication Protocol and Framing

We will be using TCP/IP as the communication protocol between the sensor node and aggregator. in order to structure the transportation such that both parties have a standardized approach to communication, data frames are structured as follows

Source/Type/Value

For example, if the sensor node in the bedroom sensed a temperature of 30 degrees, it would send the frame

Step 6: Procedure

  1. First of all, make changes to the supplied code, add in your Raspberry Pi's IP address and port for TCP communication.
  2. Also add in your MySQL password, user name and database to be used.
  3. Also change the names of the tables in the createtables.py
  4. Find a good place to put your sensor node. It should be central location in the room, which receives most of the light and air supply isn't blocked. This is required for valid temperature readings.
  5. When the sensor node is placed, power it up. Upload the nodesens.py code to the sensor node over an FTP client like FileZilla, but don't run it.
  6. Boot up your Raspberry Pi and open up your MySQL database to the one you want to use the stored data.
  7. Run createtables.py code to create the tables and cross check in MySQL Open the terminal and use the command sudo python listen.py & to start the listener for TCP messages in the background.
  8. Run raspi_paas.py and make sure the GUI is running.
  9. SSH into your sensor node and run nodesens.py
  10. Great! Your own personal IoT platform is now up and running!
  11. You can view the updated sensor readings on the GUI or view the plot by pressing the Plot buttons.

Attachments

Step 7: Screenshots and Further Improvements

I ran this setup for 24 hours, with my sensors monitoring the temperature and ambient light. It gave me a fairly smooth curve for light intensity as expected.

This project is really a starting point or a base for many further expansions that can be done. Some improvements that may be implemented are;

  • As stated before, this project has been implemented on a local network and can just as easily be replicated over the internet. As promised, I will go over the process of port forwarding. Port forwarding is a technique used to make communication channels available to the external network. Port forwarding is something that needs to be done to your router, not the Raspberry Pi. Typically, the option is available on the router configuration page, and varies by manufacturer. You should have an option in the Gateway category to forward the ports. Simply add your Raspberry Pi's IP address and port there, and you're good to go! Now you can stream data from anywhere in the world! Your sensor node might be in Australia and you can get the data in Austria!
  • Another improvement that can be done is the addition of a number of nodes to form a wireless sensor network with the Raspberry Pi as a hub. This would require more hardware and some amount of synchronizing so that the messages don't collide. A round robin system would work.

So this has been an overview of my project, your Private IoT Platform as a Service, hosted on a Raspberry Pi!