Graphs on Raspberry Pi / RasPiConnect / MatPlotLib

September 8, 2014 Matplotlib Graphs, Project Curacao, Raspberry Pi, RasPiConnect, Solar Power
Raspberry Pi
Main Project Curacao RasPiConnect Window

Drawing and Displaying Graphs on Raspberry Pi and RasPiConnect using MatPlotLib

We often have data that is better shown as a graph rather than a table.  It is much easier to pick out a pattern when you show the data visually.  This is especially true when you have a  complex sensor project like Project Curacao (There are 17 sensors on the project and We are about to add 5 more).   Project Curacao is a solar powered Raspberry Pi / Arduino project located down on the island of Curacao.

One of the things we use  the Raspberry Pi for on Project Curacao for is to generate graphs.  Yes, we know we could ship the data to another server and generate the graphs there, but since we have a nice linux platform, we decided to make the graphs on site.  Then we ship the graphs both to a webpage and also over to the RasPiConnect directory for display on the RasPiConnect App.

Here is a tutorial for RasPiConnect that we published in MagPi Magazine.

MatPlotLib is a python library for making publication quality plots using methods similar to MATLIB. You can output formats such as PDF, Postscript, SVG, and PNG.

Here’s a picture of the embedded graph (in RasPiConnect):

Raspberry Pi MatPlotLib
Main Project Curacao RasPiConnect Window

This particular graph shows the actual installation of the box in Curacao.  Note how the solar power takes off when it is put up in the sun.  Next we show the MatPlotLib code to generate a graph, and then show how to put it on a RasPiConnect screen.

Example MatPlotLib Code

Here is the example matplotlib code and graph for one of the graphs (including the call to MySQL to get the data). I am using the Solar/Wind line to show which power source is selected.  Reg Wind Volt gives the voltage on the regulated side of the DC/DC 12V – 6V Buck converter (fed to the solar power charging circuitry) and Unreg Wind Volt is on the Wind Turbine side of the converter.  We choose to use just points because of the intermittent nature of the wind.  See the post on the wind storm we had a couple of weeks ago.  This was the graph generated after a couple of trips in the car with various wind turbines.

MatPlotLib
Solar Wind MatPlotLib Graph
# solar wind graph generation
# filename: solarwindgraph.py
# Version 1.4 01/12/14
#
# contains graphing routines
#
#
import sys
import time
import RPi.GPIO as GPIO
import gc
import datetime
import matplotlib
# Force matplotlib to not use any Xwindows backend.
matplotlib.use(‘Agg’)
from matplotlib import pyplot
from matplotlib import dates
import pylab
import MySQLdb as mdb
sys.path.append(‘/home/pi/ProjectCuracao/main/config’)
# if conflocal.py is not found, import default conf.py
# Check for user imports
try:
        import conflocal as conf
except ImportError:
        import conf
def  solarwindgraph(source,days,delay):
        print(“solarwindgraph source:%s days:%s delay:%i” % (source,days,delay))
        print(“sleeping :”,delay)
        time.sleep(delay)
        print(“solarwindgraph running now”)
        # blink GPIO LED when it’s run
        GPIO.setmode(GPIO.BOARD)
        GPIO.setup(22, GPIO.OUT)
        GPIO.output(22, False)
        time.sleep(0.5)
        GPIO.output(22, True)
        # now we have get the data, stuff it in the graph
        try:
                print(“trying database”)
                db = mdb.connect(‘localhost’, ‘root’, conf.databasePassword, ‘ProjectCuracao’);
                cursor = db.cursor()
                query = “SELECT TimeStamp, RegulatedWindVoltage, UnregulatedWindVoltage, SolarWind FROM batterywatchdogdata where  now() – interval %i hour < TimeStamp” % (days*24)
                cursor.execute(query)

 

                result = cursor.fetchall()
                t = []
                s = []
                u = []
                v = []
                for record in result:
                        t.append(record[0])
                        s.append(record[1])
                        u.append(record[2])
                        v.append(record[3])
                print (“count of t=”,len(t))
                # scale array for Solar =0 Wind = 1
                for i in range(len(v)):
                        v[i] = v[i] * 10
                #dts = map(datetime.datetime.fromtimestamp, t)
                #print dts
                fds = dates.date2num(t) # converted
                # matplotlib date format object
                hfmt = dates.DateFormatter(‘%m/%d-%H’)
                fig = pyplot.figure()
                fig.set_facecolor(‘white’)
                ax = fig.add_subplot(111,axisbg = ‘white’)
                ax.vlines(fds, -200.0, 1000.0,colors=’w’)
                ax.xaxis.set_major_locator(dates.HourLocator(interval=6))
                ax.xaxis.set_major_formatter(hfmt)
                ax.set_ylim(bottom = -200.0)
                pyplot.xticks(rotation=’vertical’)
                pyplot.subplots_adjust(bottom=.3)
                pylab.plot(t, s, color=’b’,label=”Reg Wind Volt”,linestyle=””,marker=”.”)
                pylab.plot(t, u, color=’r’,label=”Unreg Wind Volt”,linestyle=””,marker=”.”)
                pylab.plot(t, v, color=’g’,label=”Solar/Wind”,linestyle=”-“,marker=”.”)
                pylab.xlabel(“Hours”)
                pylab.ylabel(“Voltage”)
                pylab.legend(loc=’upper left’)
                pylab.axis([min(t), max(t), 0, 20])
                pylab.figtext(.5, .05, (“Solar / Wind System Last %i Days” % days),fontsize=18,ha=’center’)
                pylab.grid(True)
                pyplot.show()
                pyplot.savefig(“/home/pi/RasPiConnectServer/static/solarwindgraph.png”)
        except mdb.Error, e:
                print “Error %d: %s” % (e.args[0],e.args[1])
        finally:
                cursor.close()
                db.close()
                del cursor
                del db
                fig.clf()
                pyplot.close()
                pylab.close()
                del t, s, u, v
                gc.collect()
                print(“solarwindgraph finished now”)

RasPiConnect Code

The RasPiConnect Local.py code for a graph follows:

#W-11 is Pi System Status
if (objectServerID == “W-11”):
                #check for validate request
                if (validate == “YES”):
                        outgoingXMLData += Validate.buildValidateResponse(“YES”)
                        outgoingXMLData += BuildResponse.buildFooter()
                        return outgoingXMLData
            # normal response requested
imageName = “systemstatistics.png”
              responseData = “<html><head>”
                responseData += “<title></title><style>body,html,iframe{margin:0;padding:0;}</style>”
                responseData += “</head>”
                responseData += “<body><img src=””
                responseData += Config.localURL()
                responseData += “static/”
                responseData += imageName
                responseData += “” type=”jpg” width=”800″ height=”300″>”
                responseData +=“</body>”
                responseData += “</html>”
                outgoingXMLData += BuildResponse.buildResponse(responseData)
      outgoingXMLData += BuildResponse.buildFooter()
                return outgoingXMLData

Note that if you want to use a button to select different type of graphs for the same RasPiConnect control, you write the graph name to a file and the read the file to get what graph to display.  Make sure you assign the Web Control to refresh when you push the button.  Then you get an immediate response.

You can select a different graph by using a feedback control button to write that file (that contains the graph name) and cycle through graphs (see my video on this blog for an example) as below:

# FB-11 – change the graph display
if (objectServerID == “FB-11”):
                #check for validate request
# validate allows RasPiConnect to verify this object is here
                if (validate == “YES”):
                        outgoingXMLData += Validate.buildValidateResponse(“YES”)
                        outgoingXMLData += BuildResponse.buildFooter()
                        return outgoingXMLData
# not validate request, so execute
               responseData = “XXX”
if (objectName is None):
objectName = “XXX”
               lowername = objectName.lower()
                if (lowername == “display voltages”):
                        responseData = “display currents”
                        responseData = responseData.title()
                f = open(“./local/GraphSelect.txt”, “w”)
                f.write(lowername)
                f.close()
                elif (lowername == “display currents”):
                        responseData = “display solar/wind”
                        responseData = responseData.title()
                f = open(“./local/GraphSelect.txt”, “w”)
                f.write(lowername)
                f.close()
                elif (lowername == “display solar/wind”):
                        responseData = “display voltages”
                        responseData = responseData.title()
                f = open(“./local/GraphSelect.txt”, “w”)
                f.write(lowername)
                f.close()
# defaults to display currents
                else:
                        lowername = “display currents”
                f = open(“./local/GraphSelect.txt”, “w”)
                f.write(lowername)
                f.close()
                        responseData = “display voltages”
                        responseData = lowername.title()
                outgoingXMLData += BuildResponse.buildResponse(responseData)
                        outgoingXMLData += BuildResponse.buildFooter()

                return outgoingXMLData

Installing MatPlotLib

Here are the steps to get the necessary packages for MatPlotLib:

$ sudo apt-get install libblas-dev        ## 1-2 minutes
$ sudo apt-get install liblapack-dev      ## 1-2 minutes
$ sudo apt-get install python-dev        ## Optional
$ sudo apt-get install libatlas-base-dev ## Optional speed up execution
$ sudo apt-get install gfortran           ## 2-3 minutes
$ sudo apt-get install python-setuptools  ## ?
$ sudo easy_install scipy                 ## 2-3 hours

$ sudo apt-get install python-matplotlib  ## 1 hour

Thank you https://wyolum.com/numpyscipymatplotlib-on-raspberry-pi/

We run graphs on our pi system in Project Curacao based upon entries in apschedule:

    scheduler = Scheduler()
    job = scheduler.add_cron_job(powerdatacollect.datacollect5minutes, minute=”*/5″, args=[‘main’, 0])
    job = scheduler.add_cron_job(watchdogdatacollect.watchdogdatacollect, minute=”*/5″, args=[‘main’, 30])
    job = scheduler.add_cron_job(environdatacollect.environdatacollect, minute=”*/15″, args=[‘main’, 10])
    job = scheduler.add_cron_job(systemstatistics.systemstatistics15minutes, minute=”*/15″, args=[‘main’, 20])
    job = scheduler.add_cron_job(doallgraphs.doallgraphs, minute=”*/15″, args=[‘main’,10,60])
    # camera
    job = scheduler.add_cron_job(useCamera.takeSinglePicture, hour=”*”, args=[‘main’,50])
    # send daily picture
    job = scheduler.add_cron_job(sendPictureEmail.sendPictureEmail, hour=”22″,minute=”20″, args=[‘main’,0])

One good thing about running tasks like this in another thread, is that a hang in one of the threads does not stop the whole system.  Just that thread.