Tutorial: DataLogger #3 –  Measure/Graph/Log Current with the Raspberry Pi

The Software

Since current is so important in many of our designs at SwitchDoc Labs, we decided to build a Raspberry Pi based DataLogger to read how much current was going into the Arduino based transmitter and use this information to modify the system to reduce the current.  This tutorial shows how to build the DataLogger.

Part 1: DataLogger Introduction

Part 2:  DataLogger Hardware

Part 3: DataLogger Software

The DataLogger consists of a Raspberry Pi 3 running the DataLogging Python Software,  MySQL, MatPlotLib and an Apache Webserver.  This part of the tutorial shows how to install all of the software and talks about the DataLogger.py software.  Since we are currently measuring current, we are using a Grove INA3221  3 Channel current/voltage measurement device.

Installation of the Raspberry Pi Software

We will need to install the following three major software packages.   MySQL/phpMyAdmin/Apache web server, apscheduler, MatPlotLib and then the DataLogging Python software itself.

MySQL/phpMyAdmin/Apache

MySQL is used to store the data from the DataLogger software to provide a database that can be used for later analysis and graphing.   The phpMyAdmin software is used to setup and control the MySQL database server and finally Apache is used to serve phpMyAdmin webpages and the DataLogger graphs.

Installing MySQL/phpMyAdmin/Apache – Use the following excellent tutorial:  https://pimylifeup.com/raspberry-pi-mysql-phpmyadmin/

Apscheduler

apscheduler is a very good Python package that is used to schedule events from within a main Python program that run in different threads.  One good thing about running tasks like this in another thread, is that a hang in one of the threads does not stop the whole system.  Just that thread.

sudo pip install setuptools --upgrade 
sudo pip install apscheduler

MatPlotLib

MatPlotLib is a python library for making publication quality plots using methods similar to MATLIB on the Raspberry Pi. You can output formats such as PDF, Postscript, SVG, and PNG.

Install the following packages (This will take quite a while)

$ sudo apt-get install libblas-dev        ## 1-2 minutes
$ sudo apt-get install liblapack-dev      ## 1-2 minutes
$ sudo apt-get install python-dev        ## Optional
$ sudo apt-get install libatlas-base-dev ## Optional speed up execution
$ sudo apt-get install gfortran           ## 2-3 minutes
$ sudo apt-get install python-setuptools  ## ?
$ sudo easy_install scipy                 ## 2-3 hours

$ sudo apt-get install python-matplotlib  ## 1 hour

DataLogger Python Software

To install the DataLogger software do the following:

cd
git clone https://github.com/switchdoclabs/SDL_Pi_DataLogger.git

Now you have all the required software installed.

Two last things:

1. Open up phpMyAdmin, go to the SQL Tab and copy the contents of DataLogger.sql into the MySQL Box and hit “Go” to build the database.
2. Copy DataLogger.html to /var/www/html:

sudo cp DataLogger.html /var/www/html

Now you are ready to go.

Major Software Blocks in DataLogger

There are three major software blocks in the DataLogger software that bear some explanation.

The Scheduling Software

Here are the constants defining how often the scheduling code runs.

 

#How often in seconds to sample Data
SampleTime = 0.5
#How long in seconds to sample Data
LengthSample = 60000
#How often to generate graph in seconds
GraphRefresh = 10
#How many samples to Graph
GraphSampleCount = 300

The scheduling code starts four different jobs, all run in separate threads.

• readData – reads the INA3221 and stores the data in MySQL
• buildGraph – builds the MatPlotLib graph and copies it over to /var/www/html
• killLogger – kills the Logging software after a long period of time
• tick – prints out the current time

    scheduler = BackgroundScheduler()
    scheduler.add_job(readData, 'interval', seconds=SampleTime)
    scheduler.add_job(buildGraph, 'interval', seconds=GraphRefresh)
    scheduler.add_job(killLogger, 'interval', seconds=LengthSample)
    scheduler.add_job(tick, 'interval', seconds=60)
    scheduler.start()
    scheduler.print_jobs()

 

The INA3221 Sampling Software

The scheduler calls the data sampling software every “SampleTime”, 0.5 seconds in our example above.   The INA3221 is read and then the data is written to the MySQL database.

 

def readData():
    	print('readData - The time is: %s' % datetime.now())


  	print "------------------------------"
  	shuntvoltage1 = 0
  	busvoltage1   = 0
  	current_mA1   = 0
  	loadvoltage1  = 0


  	busvoltage1 = ina3221.getBusVoltage_V(LIPO_BATTERY_CHANNEL)
  	shuntvoltage1 = ina3221.getShuntVoltage_mV(LIPO_BATTERY_CHANNEL)
  	# minus is to get the "sense" right.   - means the battery is charging, + that it is discharging
  	current_mA1 = ina3221.getCurrent_mA(LIPO_BATTERY_CHANNEL)  

  	loadvoltage1 = busvoltage1 + (shuntvoltage1 / 1000)

        # set Label
	# myLabel = "LIPO_Battery"
        myLabel = ""  

  	print "(Channel 1) %s Bus Voltage 1: %3.2f V " % (myLabel, busvoltage1)
  	print "(Channel 1) %s Shunt Voltage 1: %3.2f mV " % (myLabel, shuntvoltage1)
  	print "(Channel 1) %s Load Voltage 1:  %3.2f V" % (myLabel, loadvoltage1)
  	print "(Channel 1) %s Current 1:  %3.2f mA" % (myLabel, current_mA1)
  	print

  	shuntvoltage2 = 0
  	busvoltage2 = 0
  	current_mA2 = 0
  	loadvoltage2 = 0

  	busvoltage2 = ina3221.getBusVoltage_V(SOLAR_CELL_CHANNEL)
  	shuntvoltage2 = ina3221.getShuntVoltage_mV(SOLAR_CELL_CHANNEL)
        # "-" removed for this demo program
  	current_mA2 = ina3221.getCurrent_mA(SOLAR_CELL_CHANNEL)
  	loadvoltage2 = busvoltage2 + (shuntvoltage2 / 1000)
  
        # set Label
	# myLabel = "Solar Cell"
        myLabel = ""  

  	print "(Channel 2) %s Bus Voltage 2:  %3.2f V " % (myLabel, busvoltage2)
  	print "(Channel 2) %s Shunt Voltage 2: %3.2f mV " % (myLabel, shuntvoltage2)
  	print "(Channel 2) %s Load Voltage 2:  %3.2f V" % (myLabel, loadvoltage2)
  	print "(Channel 2) %s Current 2:  %3.2f mA" % (myLabel, current_mA2)
  	print 

  	shuntvoltage3 = 0
  	busvoltage3 = 0
  	current_mA3 = 0
  	loadvoltage3 = 0

  	busvoltage3 = ina3221.getBusVoltage_V(OUTPUT_CHANNEL)
  	shuntvoltage3 = ina3221.getShuntVoltage_mV(OUTPUT_CHANNEL)
  	current_mA3 = ina3221.getCurrent_mA(OUTPUT_CHANNEL)
  	loadvoltage3 = busvoltage3 + (shuntvoltage3 / 1000)
        
	# set Label
	# myLabel = "Output"
        myLabel = ""  
  
  	print "(Channel 3) %s Bus Voltage 3:  %3.2f V " % (myLabel, busvoltage3)
  	print "(Channel 3) %s Shunt Voltage 3: %3.2f mV " % (myLabel, shuntvoltage3)
  	print "(Channel 3) %s Load Voltage 3:  %3.2f V" % (myLabel, loadvoltage3)
  	print "(Channel 3) %s Current 3:  %3.2f mA" % (myLabel, current_mA3)
  	print
	#
	# Now put the data in MySQL
	# 
        # Put record in MySQL

        print "writing SQLdata ";


        # write record
        deviceid = 0
        query = 'INSERT INTO '+tableName+('(timestamp, deviceid, channel1_load_voltage, channel1_current, channel2_load_voltage, channel2_current, channel3_load_voltage, channel3_current) VALUES(UTC_TIMESTAMP(),  %i,  %.3f, %.3f, %.3f, %.3f, %.3f, %.3f)' %( deviceid,  busvoltage1, current_mA1, busvoltage2, current_mA2, busvoltage3, current_mA3))
        print("query=%s" % query)

        cur.execute(query)	
	con.commit()
<\pre>

Do not forget the con.commit() at the end.  Otherwise, your data will mysteriously disappear because readData() is run in a separate thread from the main program.

The MatPlotLib Graphing Software

When called by the scheduler above (every 10 seconds in our example code), the graph is rebuilt from the last .   This allows the HTML page to get a new graph every interval.   The graph is generated from “GraphSampleCount”, 300 samples in our example.

def buildGraph():
    		print('buildGraph - The time is: %s' % datetime.now())

		# open database
		con1 = mdb.connect('localhost', 'root', password, 'DataLogger' )
		# now we have to get the data, stuff it in the graph 

    		cursor = con1.cursor()

		query = '(SELECT timestamp, deviceid, channel1_load_voltage, channel1_current, channel2_load_voltage, channel2_current, channel3_load_voltage, channel3_current, id FROM '+tableName+' ORDER BY id DESC LIMIT '+ str(GraphSampleCount) + ') ORDER BY id ASC' 

		print "query=", query
		try:
			cursor.execute(query)
			result = cursor.fetchall()
		except:
			e=sys.exc_info()[0]
			print "Error: %s" % e


		print result[0]
		t = []   # time
		u = []   # channel 1 - Current 
		averageCurrent = 0.0
 		currentCount = 0
		for record in result:
			t.append(record[0])
			u.append(record[7])
			averageCurrent = averageCurrent+record[7]
			currentCount=currentCount+1

		averageCurrent = averageCurrent/currentCount
		
		print ("count of t=",len(t))

		fds = dates.date2num(t) # converted
		# matplotlib date format object
		hfmt = dates.DateFormatter('%H:%M:%S')
		#hfmt = dates.DateFormatter('%m/%d-%H')

		fig = pyplot.figure()
		fig.set_facecolor('white')
		ax = fig.add_subplot(111,axisbg = 'white')
		ax.vlines(fds, -200.0, 1000.0,colors='w')



		#ax.xaxis.set_major_locator(dates.MinuteLocator(interval=1))
		ax.xaxis.set_major_formatter(hfmt)
		ax.set_ylim(bottom = -200.0)
		pyplot.xticks(rotation='45')
		pyplot.subplots_adjust(bottom=.3)
		pylab.plot(t, u, color='r',label="Arduino Current",linestyle="-",marker=".")
		pylab.xlabel("Seconds")
		pylab.ylabel("Current mA")
		pylab.legend(loc='lower center')

		pylab.axis([min(t), max(t), 0, max(u)+20])
		pylab.figtext(.5, .05, ("Average Current %6.2fmA\n%s") %(averageCurrent, datetime.now()),fontsize=18,ha='center')

		pylab.grid(True)

		pyplot.show()
		pyplot.savefig("/var/www/html/DataLoggerGraph.png", facecolor=fig.get_facecolor())	



		cursor.close()       	 
		con1.close()

		fig.clf()
		pyplot.close()
		pylab.close()
		gc.collect()
		print "------Graph finished now"

When you play with this code, make sure you look up each command at the matplotlib.org site so you understand what you are doing. it is a bit arcane to say the least.

Example Run and Results

 

pi@RPi3:~/SDL_Pi_DataLogger $ sudo python DataLogger.py

Datalogger

Sample uses 0x40 and SunAirPlus board INA3221
 Will work with the INA3221 SwitchDoc Labs Breakout Board
Program Started at:2016-06-07 00:56:41

Jobstore default:
    readData (trigger: interval[0:00:00.500000], next run at: 2016-06-07 00:56:41 UTC)
    buildGraph (trigger: interval[0:00:10], next run at: 2016-06-07 00:56:51 UTC)
    tick (trigger: interval[0:01:00], next run at: 2016-06-07 00:57:41 UTC)
    killLogger (trigger: interval[16:40:00], next run at: 2016-06-07 17:36:41 UTC)
Press Ctrl+C to exit
buildGraph - The time is: 2016-06-07 00:56:41.355321
query= (SELECT timestamp, deviceid, channel1_load_voltage, channel1_current, channel2_load_voltage, channel2_current, channel3_load_voltage, channel3_current, id FROM WeatherLink ORDER BY id DESC LIMIT 300) ORDER BY id ASC
(datetime.datetime(2016, 6, 4, 17, 23, 26), 0L, 4.168, -39.6, 0.672, 0.0, 5.024, 9.6, 162901L)
('count of t=', 300)
readData - The time is: 2016-06-07 00:56:41.893182
------------------------------
(Channel 1)  Bus Voltage 1: 4.14 V 
(Channel 1)  Shunt Voltage 1: 1.00 mV 
(Channel 1)  Load Voltage 1:  4.15 V
(Channel 1)  Current 1:  10.00 mA

(Channel 2)  Bus Voltage 2:  0.00 V 
(Channel 2)  Shunt Voltage 2: 0.00 mV 
(Channel 2)  Load Voltage 2:  0.00 V
(Channel 2)  Current 2:  0.00 mA

(Channel 3)  Bus Voltage 3:  0.00 V 
(Channel 3)  Shunt Voltage 3: 0.00 mV 
(Channel 3)  Load Voltage 3:  0.00 V
(Channel 3)  Current 3:  0.00 mA

writing SQLdata 
query=INSERT INTO WeatherLink(timestamp, deviceid, channel1_load_voltage, channel1_current, channel2_load_voltage, channel2_current, channel3_load_voltage, channel3_current) VALUES(UTC_TIMESTAMP(),  0,  4.144, 10.000, 0.000, 0.000, 0.000, 0.000)
readData - The time is: 2016-06-07 00:56:42.409886
------------------------------
(Channel 1)  Bus Voltage 1: 4.02 V 
(Channel 1)  Shunt Voltage 1: 1.00 mV 
(Channel 1)  Load Voltage 1:  4.03 V
(Channel 1)  Current 1:  10.00 mA

(Channel 2)  Bus Voltage 2:  0.00 V 
(Channel 2)  Shunt Voltage 2: 0.00 mV 
(Channel 2)  Load Voltage 2:  0.00 V
(Channel 2)  Current 2:  0.00 mA

(Channel 3)  Bus Voltage 3:  0.00 V 
(Channel 3)  Shunt Voltage 3: 0.00 mV 
(Channel 3)  Load Voltage 3:  0.00 V
(Channel 3)  Current 3:  0.00 mA

writing SQLdata 
query=INSERT INTO WeatherLink(timestamp, deviceid, channel1_load_voltage, channel1_current, channel2_load_voltage, channel2_current, channel3_load_voltage, channel3_current) VALUES(UTC_TIMESTAMP(),  0,  4.024, 10.000, 0.000, 0.000, 0.000, 0.000)
No handlers could be found for logger "apscheduler.scheduler"
------Graph finished now
readData - The time is: 2016-06-07 00:56:43.349699
------------------------------
(Channel 1)  Bus Voltage 1: 3.82 V 
(Channel 1)  Shunt Voltage 1: 1.00 mV 
(Channel 1)  Load Voltage 1:  3.83 V
(Channel 1)  Current 1:  10.00 mA

(Channel 2)  Bus Voltage 2:  0.00 V 
(Channel 2)  Shunt Voltage 2: 0.00 mV 
(Channel 2)  Load Voltage 2:  0.00 V
(Channel 2)  Current 2:  0.00 mA

(Channel 3)  Bus Voltage 3:  0.00 V 
(Channel 3)  Shunt Voltage 3: 0.00 mV 
(Channel 3)  Load Voltage 3:  0.00 V
(Channel 3)  Current 3:  0.00 mA

writing SQLdata 
query=INSERT INTO WeatherLink(timestamp, deviceid, channel1_load_voltage, channel1_current, channel2_load_voltage, channel2_current, channel3_load_voltage, channel3_current) VALUES(UTC_TIMESTAMP(),  0,  3.824, 10.000, 0.000, 0.000, 0.000, 0.000)
readData - The time is: 2016-06-07 00:56:44.349705
------------------------------

This shows the current and voltage in connected to Channel 1 in the Device Under Test (DUT).

Here is an example graph from the resulting webpage.

Conclusion

The Raspberry Pi DataLogger allows us to measure many different types of variables and use the same basic code to write them to a database for further analysis.   The use of the auto refresh HTML page allows us to see the data as it happens without a great deal of system and network overload.   And you can easily modify the HTML code as well as the Python DataLogging Code.