Intel Edison Sensor Dashboard Using Freeboard/Python/Flask (minimal Programming Necessary)

First I wanted to say thank you for the opportunity to work with an Edison and for being given one by Intel/Instructables for the purpose of entering their IoT contest. I think this is a great way to do community building and encourage developers to get involved and on board. It should also get you some feedback (so for the Intel folks who might stumble on this I’ll show some of the difficulties I had).

Second I wanted to thank the many tutorials I have gleaned useful information from to make this possible. The tutorials on Intel’s site are great and have been augmented by the ones I’ve found here on this site. As you may have guessed by now I write a verbose Instructable. I will try to put up headings and divide things so you can find exactly what you are looking for in short time.

So why did I create this instructable and what is it? In short it is a way to take sensors or any numerical data you can get on the Edison (and portions of this can be readily adapted to work with the Raspberry Pi, the beaglebone, etc.) and display it on a dashboard type display on a connected device. My original proposal to Instructables to get the kit for this contest was to expose car sensors (like obd II, off the shelf sensors, and the ones in the “transportation kit” to users on commodity web devices (laptops, cell phones, etc.). In essence I thought it would be interesting to build something like the telemetry systems F1, Lemans, and other teams use, but on an affordable budget. In another Instructable I will put all of these things together to show how this can be done. For this one I will limit it to the simple goal of getting data from the Edison and putting it on a dashboard on a local network. Extrapolating this to work with the “cloud”, freeboard.io, or on the open internet should be straightforward.

Pre-requisite knowledge – a little programming, a little linux, and maybe a little python. If there is something you don’t understand I encourage you to look it up. If I didn’t provide a reference for something and you think it should be there let me know in the comments. There are other ways to do this of course. Intel in fact has strung together similar functionality and it is documented here: https://www.instructables.com/id/Connecting-Intel-...

Step 1: Hardware and Software List (with Some Justification)

Intel Edison – the contest uses this hardware and so that is what I am using. In reality you could do this with other platforms, but they each lack something. The Raspberry PI in particular doesn’t have onboard ADC and some would argue the HDMI and video processing pieces of it are not needed for a project like this.

Intel Edison Arduino or other breakout board – The Edison is not useful if you can’t easily power it or get to its pins.

A usb cable – For initial set up and power A power supply – Optional, you can power the Edison with a usb cable if necessary

A sensor – again optional, you can use whatever numerical producing piece of code you’d like. I will include examples of both. One will be a alcohol sensor and the other will be a memory service that reports free memory.

Python – specifically 2.x – I will catch some criticism for this, I know. I have some Python knowledge and I have no javascript knowledge. Run what you know, right. I had initially considered using node.js, but couldn’t easily figure out how to get it to handle restful jsonp requests. I also tried to use node.js + express, but the monogodb requirement killed that idea. Mongo, according to some searching doesn’t compile (or compile easily) for the Edison platform.

Flask – a web framework for Python. This makes it easy-ish to configure web replies to data queries in a RESTful/jsonp kind of way (You don’t need to really understand what REST or JSONP is at this point. So, don’t pause to look it up until later if my explanation isn’t detailed enough. Just know that this type of data presentation is what the dashboard needs. In fact you could skip the dashboard part of this tutorial and send the data to other things that like data in this format, many of which are IoT enablers, data loggers, etc.). As a side note: I had at one point tried to just dump data to a file, which was the wrong way to do things.

Freeboard – an open source dashboard (javascript based) page that is easy to configure for the end user (it requires no programming). Freeboard won out because it is simple to use, requires no database backend, and has very simple hosting requirements. All you have to do is point it at a jsonp data source and then select options to create a dashboard.

Psutils – a python library that lets you get host information like memory available, cpu usage, processes, network status, network connections, etc.

mraa/upm – mraa is a c++ library provided by Intel to help abstract the process of getting data from sensors plugged into the GPIO pins of the Edison (and it works with a few other boards as well, like the Raspberry PI). Mraa will let you get analog data, pwm, spi, uart, i2c, etc. Furthermore code has been written for an extra layer of abstraction on top of it for a long list of popular sensors, making it even easier to get data. (For those wondering I read somewhere that mraa doesn’t actually stand for anything.) upm - stands for useful packages and modules - Some pre-made code for common sensors, motors, actuators, etc. that works with mraa. This, for example, lets you get a temperature in Celsius vs a voltage number converted into a digital step that you have to figure out.

Step 2: Set Up the Edison

For this step I’ll defer to others’ directions as they helped me immensely in this process. It shouldn’t take you too long and when you are done you will have a running and working Edison with Yocto installed, updated, and working as an AP (which is optional, but if you want to have a tiny network with just a client device (like a cell phone) and the Edison you will have to do this.)

For setting up the Edison I used these tutorials. They are fairly recent and easy enough to comprehend. Sparkfun youtube tutorials. Embedded at the beginning of this step.

Sparkfun written tutorials: https://learn.sparkfun.com/tutorials/edison-getti...

The only issues I had were the following:

-I originally was trying to power the device with the wall plug while updating the system image using the Intel phone tool. I switched to using two usb cables connected to my computer and it worked just fine after that.

-After the update I was unable to use a serial terminal to get a login prompt from the device. It was because I didn’t RTFM (https://software.intel.com/en-us/setting-up-serial-terminal-on-system-with-windows) and went ahead and kept picking the wrong COM port. Here is how to set up the Edison as and AP if you are interested: https://learn.sparkfun.com/tutorials/edison-getti...

So, after these steps you should have a working Edison that you can ssh to and secure copy files to (prefer Filezilla if you use Windows).

Step 3: Set Up the Python Environment

Python is already installed on the Edison via the Yocto linux image. So, we are all set in that regard.

To install the dependencies for what we are going to do I like to use pip. I followed the instructions here: http://mendrugory.weebly.com/blog/intel-edison-an... to install it. (Skip the first step which adds the 1.1 repo...this is old stuff and the 1.5 repo (at time of publishing) is what is already set up).

For posterity (and in case the blog linked to disappears):

#opkg update
#opkg upgrade
#vi /etc/opkg/base-feeds.conf

(if you aren’t familiar with vi check this out: http://www.cs.colostate.edu/helpdocs/vi.html )

Hit the “i” key on your keyboard (to enter insert mode in vi) and type in the following:

src/gz all http://repo.opkg.net/edison/repo/all
src/gz edison http://repo.opkg.net/edison/repo/edison src/gz core2-32 http://repo.opkg.net/edison/repo/core2-32

This adds some unofficial package repositories to the ones your edison can download software from. I don’t know how good they are or how often they are maintained. So, use them with a grain of salt.
The press the “esc” key on your keyboard and type “wq”. This changes you from insert mode to command mode in vi and the command “wq” stands for “write quit” which writes to the file and quits vi.

Finally install PIP:

#opkg update
#opkg install python-pip

(yes it has a lot of dependencies)
The fix the problem pointed out by the original poster. I prefer to download setup tool and put in on a device using scp or filezilla. You can choose what you’d like at this point. (Pip is THE package manager for python, which begs the question: why is it not included. You may of course manually install the packages I talk about below, but with pip it is a much easier, smoother process. Plus, you know, package management is the way things ought to be done.) Install the Flask framework: Type pip install flask . Pip is a python package manager that takes care of installing all the dependencies you need to run flask.

You will also want to install the psuitls package I will use in an example here. However, you cannot do this with pip. So, head over to https://github.com/giampaolo/psutil/archive/maste... download, unzip and we are going to do essentially what is here under the “linux” session:

https://github.com/giampaolo/psutil/blob/master/IN...

-gcc is already installed

-python-dev is already installed

#wget --no-check-certificate https://github.com/giampaolo/psutil/archive/master.zip
#unzip master.zip
#cd psutil-master
#python setup.py install

and that’s it.

Tip: disable the default web server by doing this:

#systemctl disable edison_config
#systemctl stop edison_config

Then make sure your flask server is running on port 80. I like to use debug with flask because it will watch to see if you have changed anything in the file and then when you have it will try to reload everything. It also spews useful info to the browser and console. Turn it off for production type situations.

Step 4: Install Freeboard and Test Something Simple

Freeboard can be hosted from almost any web server software and can even be used locally without a webserver (go to the freeboard github, download it and open index.html on your computer to give it a try. I put everything from the freeboard in the /static folder which is flasks way of hosting files.

Got to my github page and download the zip file https://github.com/stirobot/webStatsPyJSON/archiv... .

Note: My page for this will change and so will my code. So, beware that I have linked to the latest revision. The most up to date code, with more “software sensors” and possible refinements is what you will get. It may eventually break something in this Instructable (at which point I will try to post a link to the revision that works here). Upload the files to a directory on your Edison. (The home directory is probably good for now as security isn’t part of this tutorial and not something I worried about when writing this code. Privilege escalation and other issues come into play depending on who runs the script, where it sits, permissions on the files, etc. I’m not going to cover that here. There are books on this subject for those interested. )

Run python webSensorsJson.py . This will start the webserver. Now on a computer or phone on the same network as your Edison go to http://youredisonsipaddress/memused This should give you an output like you see here (this is a very simple JSON data structure, other sensors and feeds will provide much more complicated ones and freeboard will allow you to “surf” these to get the specific data you are interested in):

{

"memused": 25

}

This is the json output of the code. Every time you hit that endpoint it will check to see how much of the system’s memory is in used (note: I'm not sure all of psutils stuff works with the edison. The memory part might not actually work perfectly). Next go to http://youredisonsipaddress/static/index.html which will take you to the freeboard dashboard configuration page.

Click to add a data source in the upper right part of the page and add the data source (see picture in the header). Don’t try to route the data through thingproxy as it won’t work. Feel free to test out the other settings. This particular example is very simple and doesn’t take any arguments. For an example that does that please check the ping example on my github page.

Next click save. Now it should appear in the list of data sources and should eventually show a timestamp instead of never to indicate that it has polled that endpoint for data.

Next add a display element and give it that data source. Click “add pane” and then the plus sign in the newly added pane to add an element. Here I am configuring a gauge to show the free memory (note: use the “+datasource” button to automatically fill in that section with the data you are interested in):

Then click save and the memory gauge widget should appear. At this point I would suggest playing with freeboard to see what other widgets there are, how to delete them, arrange them (drag and drop), how to save a dashboard, etc.

Step 5: A Code Walkthrough

So now that you see what can be done, let’s do a code walkthrough to show what you would have to do to add your own sensors. In this example I will use mraa and upm to add a simple sensor

First will write the code that gets data from the sensor and returns a value (as in the picture above).

I use mraa and upm library that gets data from the sensor automatically. You can use whatever sensor you’d like. The documentation is here: http://iotdk.intel.com/docs/master/mraa/python/ and here http://iotdk.intel.com/docs/master/mraa/python/

Step 6: A Code Walkthrough

Then I need to add some code so we can pull that data back

The I need to add that as an import in my webserver file with a directive and add a route for it.

Then I need to add some code so we can pull that data back.

@app.route(‘/alcohol’)

Tells Flask that when someone hits http://youredisonsipaddress/alcohol to run the code after it.

@support_jsonp

Makes sure that what is returned can be queried in the RESTful sense of things. What this means is that 1)if you have a complex data structure to return it will be able to be queried for specific parts of that data strutcture and 2)it will conform to the way freeboard (and other similar things) expect the data service to respond. The code for doing this is not something I wrote and can be found here: https://gist.github.com/farazdagi/1089923 and https://gist.github.com/farazdagi/1089923

alcJson = {"alcoholppm":alccontent}

Calls the analog sensor reading function from the other file (library) and puts it in a Python dictionary data structure which in the next line…

resp = jsonify(alcJson)

….is made into a json data structure

resp.status_code = 200

Not necessary, but give an http 200 return code (if anything you are probably familiar with the 404 “not found” code. HTTP has a bunch of codes, the 200 ones are for “good” responses. I leave this here because some people might want to track things using other 200 codes for various reasons.

return resp

returns the response to the GET that the browser did.

Step 7: The Ping Example

The ping example is very similar to the memory and analog examples but it takes an argument in the url like this: https:///youredisonsipaddress/ping/hostnameyouwanttoping

And it does it in code like that in the picture above.

@app.route('/ping/<string:whoToPing:>')

The stuff enclosed in “<” and “>” tells you it is an argument that is taken from the url. The whoToPing then becomes a variable of String type that can be used in the function that follows.

def pinger(whoToPing):

The url variable gets passed in here

Step 8: Expand This

So you have the basics down, now what?

I haven’t figured out how to refresh things under one second, which will be necessary if I want to get car data (or if you want to get say accelerometer data).
There is probably a better way to do max and min (a backend db is probably preferable, but slower, more code, not as simple, etc.)
One could abstract out the config of the sensors to the dashboard for even less coding
Themes could be applied to freeboard
A data logging widget could be written which logs to memory and allows for the download of data over time to the web browser.

Introduction: Intel Edison Sensor Dashboard Using Freeboard/Python/Flask (minimal Programming Necessary)