IOT ESP8266 Tutorial – Displaying the data on the Raspberry Pi with MatPlotLib – Part 3

This is the third of a multi-part posting on building and analyzing a solar powered  ESP8266.   In the first posting we are showing how to connect up an ESP8266 to the SunAirPlus Solar Power Controller/Charger/Data Collection board and to a solar panel/battery and build a REST http interface on the ESP8266.  In the second we connected up the ESP8266 REST interface to a Raspberry Pi with a MySQL database.   In Part 3 we are generating graphs and looking at the data coming into the Raspberry Pi from the ESP8266.

The purpose of this ESP8266 project is five fold:

1. Demonstrate ESP8266 on Solar Power
2. Measure EPS8266 power consumption dynamically
3. Show how to use a REST interface to send to a database on a  Raspberry Pi
4. Display the data on the Raspberry Pi on graphs using MatPlotLib
5. Adding a stepper motor to the solar panel to track the sun (next project) and measure the results versus non-tracking

Part 1  IOT ESP8266 Tutorial – Solar Power your ESP8266! 

Part 2 IOT ESP8266 Tutorial – Pi to ESP8266 via REST 

Part 3 IOT ESP8266 Tutorial – Displaying the data on the Raspberry Pi with MatPlotLib

We are using the Adafruit ESP8266 Huzzah breakout board for these postings.

The ESP8266

The ESP8266 is made by a privately held company in China called Espressif.   They are a fabless semiconductor company that just came out of nowhere and shook up the whole industry.   Now all the major players are working on inexpensive versions of an IOT chip with WiFi connectivity.  And they are all struggling to make it as inexpensive as the ESP8266

The Adafruit ESP8266 Huzzah

The Adafruit ESP8266 Huzzah board is a great  breakout for the ESP8266.  It makes it much easier to use than the really cheap modules.

Most of the low cost modules are not breadboard friendly, don’t have an onboard 3.3V regulator or level shifting for signals.  The Huzzah has all of those features.  For more on the ESP8266 Huzzah board see this posting.

The ESP8266 Software

We are using the Arduino IDE for this project.  See how to use the Arduino IDE with the ESP8266 in this posting.

The System Block Diagram

SolarPowerESP8266 is an IOT system built by SwitchDoc Labs for an upcoming article on building IOT devices.  It consists of five major pieces:

• – ESP8266 Huzzah WiFi/CPU (programmed in Arduino IDE)
• – SunAirPlus – Solar Power Controller/Charger Data Collector
• – LiPo Battery
• – 6V 3.4W Solar Panel
• – Raspberry Pi as Data Logger / Analytics / Display

Making Graphs on the Raspberry Pi

IN the IOT world, we generate a lot of data.   Some of this data is collected and put into databases.   Then comes the task of analyzing, displaying and interpreting the data.  We often have data that is better shown as a graph rather than a table.  It is much easier to pick out a pattern when you show the data visually.  This is especially true in the land of “Big Data” of which the IOT is a part.

One of the things we use  the Raspberry Pi for  for is to generate graphs.  Yes, we know we could ship the data to another server (such as IBM Bluemix) and generate the graphs there, but since we have a nice linux platform, we decided to make the graphs on site.  Then we ship the graphs both to a webpage for perusal.

MatPlotLib  is a python library for making publication quality plots using methods similar to MATLIB. You can output formats such as PDF, Postscript, SVG, and PNG.

Here’s an example of the embedded graph generated by MatPlotLib for Project Curacao:

This particular graph shows the actual installation of the box in Curacao.  Note how the solar power takes off when it is put up in the sun.  Next we show the MatPlotLib code to generate a graph, and then show how to put it on a RasPiConnect screen.

Example MatPlotLib Code

In this IOT project, the SolarPowerESP8266 is sending data about solar power generation, battery charging and power consumption on the ESP8266 board.

All of this code is on Github at https://github.com/switchdoclabs/SDL_PI_IOTDataCollector.

Here is the example matplotlib code and graphs (including the call to MySQL to get the data) for the ESP8266.  We are showing the system voltages on the Battery, the Solar Cells and the ESP8266 Output Load voltage supplied to the ESP8266 by SunAirPlus.  Note you can see where we turned off the power supply simulating the solar cells.  It has been cloudy here all week. You can see the long decline of the battery as it discharges.   We are running the battery depletion test to figure out when to shut off the ESP8266 to prevent LiPo battery damage.

# Generates graph of all voltages measured on the SolarPowerESP8266
# filename: SolarPowerESP8266VoltageGraph.py
# Version 1.0 December 2015
# SwitchDoc Labs
#
#
#


import sys
import time

import gc
import datetime

import matplotlib
# Force matplotlib to not use any Xwindows backend.
matplotlib.use('Agg')

from matplotlib import pyplot
from matplotlib import dates

import pylab

import MySQLdb as mdb



def SolarPowerESP8266VoltageGraph(source,days):


    print("SolarPowerESP8266VoltageGraph source:%s days:%s " % (source,days))
    print("SolarPowerESP8266VoltageGraph running now")


    # now we have get the data, stuff it in the graph

    try:
        print("trying database")
        db = mdb.connect('localhost', 'root', 'password', 'IOTSolarData');

        cursor = db.cursor()

        query = "SELECT TimeStamp, sample_timestamp, battery_load_voltage, solarcell_load_voltage, output_load_voltage FROM SolarPowerData where now() - interval %i hour < TimeStamp ORDER BY TimeStamp, sample_timestamp" % (days*24)
        cursor.execute(query)


        result = cursor.fetchall()
        t = []
        s = []
        u = []
        v = []

        for record in result:
            t.append(record[0])
            s.append(record[2])
            u.append(record[3])
            v.append(record[4])

        print ("count of t=",len(t))

        #dts = map(datetime.datetime.fromtimestamp, t)
        #print dts
        fds = dates.date2num(t) # converted
        # matplotlib date format object
        hfmt = dates.DateFormatter('%m/%d-%H')

        fig = pyplot.figure()
        fig.set_facecolor('white')
        ax = fig.add_subplot(111,axisbg = 'white')
        ax.vlines(fds, -200.0, 1000.0,colors='w')

        ax.xaxis.set_major_locator(dates.HourLocator(interval=6))
        ax.xaxis.set_major_formatter(hfmt)
        ax.set_ylim(bottom = -200.0)
        pyplot.xticks(rotation='vertical')
        pyplot.subplots_adjust(bottom=.3)
        pylab.plot(t, s, color='r',label="Battery Voltage",linestyle="-",marker=".")
        pylab.plot(t, u, color='b',label="SolarCell Voltage",linestyle="-",marker=".")
        pylab.plot(t, v, color='g',label="ESP8266 Voltage",linestyle="-",marker=".")
        pylab.xlabel("Hours")
        pylab.ylabel("Voltage")
        pylab.legend(loc='lower left')

        pylab.axis([min(t), max(t), 0, 7])
        pylab.figtext(.5, .05, ("SolarPowerESP8266 Voltages Last %i Days" % days),fontsize=18,ha='center')

        pylab.grid(True)

        pyplot.show()
        pyplot.savefig("static/solarvoltagegraph.png")
        pyplot.savefig("/var/www/solarvoltagegraph.png")

    except mdb.Error, e:

        print "Error %d: %s" % (e.args[0],e.args[1])

    finally:

        cursor.close()
        db.close()

        del cursor
        del db

        fig.clf()
        pyplot.close()
        pylab.close()
        del t, s, u, v
        gc.collect()
        print("SolarPowerESP8266VoltageGraph finished now")

Next, we generate the current values graph for the SolarPowerESP8266 system.

# Generates graph of all currents measured on the SolarPowerESP8266
# filename: SolarPowerESP8266CurrentGraph.py
# Version 1.0 December 2015
# SwitchDoc Labs
#
#
#


import sys
import time

import gc
import datetime

import matplotlib
# Force matplotlib to not use any Xwindows backend.
matplotlib.use('Agg')

from matplotlib import pyplot
from matplotlib import dates

import pylab

import MySQLdb as mdb



def SolarPowerESP8266CurrentGraph(source,days):


    print("SolarPowerESP8266CurrentGraph source:%s days:%s " % (source,days))
    print("SolarPowerESP8266CurrentGraph running now")


    # now we have get the data, stuff it in the graph

    try:
        print("trying database")
        db = mdb.connect('localhost', 'root', 'password', 'IOTSolarData');

        cursor = db.cursor()

        query = "SELECT TimeStamp, sample_timestamp, battery_current, solarcell_current, output_current FROM SolarPowerData where now() - interval %i hour < TimeStamp ORDER BY TimeStamp, sample_timestamp" % (days*24)
        cursor.execute(query)


        result = cursor.fetchall()
        t = []
        s = []
        u = []
        v = []

        for record in result:
            t.append(record[0])
            s.append(record[2])
            u.append(record[3])
            v.append(record[4])

        print ("count of t=",len(t))

        #dts = map(datetime.datetime.fromtimestamp, t)
        #print dts
        fds = dates.date2num(t) # converted
        # matplotlib date format object
        hfmt = dates.DateFormatter('%m/%d-%H')

        fig = pyplot.figure()
        fig.set_facecolor('white')
        ax = fig.add_subplot(111,axisbg = 'white')
        ax.vlines(fds, -200.0, 1000.0,colors='w')

        ax.xaxis.set_major_locator(dates.HourLocator(interval=6))
        ax.xaxis.set_major_formatter(hfmt)
        ax.set_ylim(bottom = -900.0)
        pyplot.xticks(rotation='vertical')
        pyplot.subplots_adjust(bottom=.3)
        pylab.plot(t, s, color='r',label="Battery Current",linestyle="-",marker=".")
        pylab.plot(t, u, color='b',label="SolarCell Current",linestyle="-",marker=".")
        pylab.plot(t, v, color='g',label="ESP8266 Current",linestyle="-",marker=".")
        pylab.xlabel("Hours")
        pylab.ylabel("Current")
        pylab.legend(loc='lower left')

        pylab.axis([min(t), max(t), min(s), max(u)])
        pylab.figtext(.5, .05, ("SolarPowerESP8266 Currents Last %i Days" % days),fontsize=18,ha='center')


        pylab.grid(True)

        pyplot.show()
        pyplot.savefig("static/solarcurrentgraph.png")
        pyplot.savefig("/var/www/solarcurrentgraph.png")

    except mdb.Error, e:

        print "Error %d: %s" % (e.args[0],e.args[1])

    finally:

        cursor.close()
        db.close()

        del cursor
        del db

        fig.clf()
        pyplot.close()
        pylab.close()
        del t, s, u, v
        gc.collect()
        print("SolarPowerESP8266CurrentGraph finished now")

Finally, a simple Python program to run all of our graphs.

updateGraphs.py

#
#
# updates all SolarPowerESP8266 Graphs
#
# SwitchDoc Labs
# December 2015
#
#
import SolarPowerESP8266VoltageGraph
import SolarPowerESP8266CurrentGraph


SolarPowerESP8266VoltageGraph.SolarPowerESP8266VoltageGraph("updateGraphs", 10)
SolarPowerESP8266CurrentGraph.SolarPowerESP8266CurrentGraph("updateGraphs", 10)

On The Web

Installing MatPlotLib

Here are the steps to get the necessary packages for MatPlotLib:

$ sudo apt-get install libblas-dev        ## 1-2 minutes
$ sudo apt-get install liblapack-dev      ## 1-2 minutes
$ sudo apt-get install python-dev        ## Optional
$ sudo apt-get install libatlas-base-dev ## Optional speed up execution
$ sudo apt-get install gfortran           ## 2-3 minutes
$ sudo apt-get install python-setuptools  ## ?
$ sudo easy_install scipy                 ## 2-3 hours

$ sudo apt-get install python-matplotlib  ## 1 hour

Thank you https://wyolum.com/numpyscipymatplotlib-on-raspberry-pi/

Conclusion

This three part tutorial showed how to build a solar powered ESP8266 and implement a REST interface.  Part 2 showed how to read the REST interface and store the data in a MySQL database.   Part 3 shows how to generate a graph with the data.  We will be adding additional systems to this software as we add new IOT devices here at SwitchDoc Labs.