Guest Blog – Larry Bonnette’s Solar Powered OurWeather Station #3/3

 

Editors Note:  We were tickled to receive this guest blog by Larry Bonnette.   He did a great job on building this Solar Powered OurWeather Kit and by adapting the GroveWeatherPi solar panel kit to make the Solar Powered OurWeather Kit.   FYI, SwitchDoc Labs is releasing a Solar Power Kit tailored for OurWeather in the next couple of weeks.

This is a three part series about building this weather station for the Tri-county Barnstormers RC Club.

• Part 1 - Building the Box
• Part 2 - Theory of Operation (How It All Works!)
• Part 3 - Webpages - Transmission and Display of the Results

Part 3 – Webpages – Transmission and Display of the Results

[callout size=”col-4″ last_column=”true” title=”Contact The Author” description=”Larry Bonnette – bonnette_l@yahoo.com”  flip_left_edge=”false” flip_right_edge=”true”]

Welcome to the third and final installment of the solar powered “OurWeather” weather station discussion. In this installment, we will discuss how we move data off of the weather station and onto the website.

Use your favorite browser and browse to www.tri-countybarnstormers.com/weathsta.htm and you will see information that has

been transmitted to the barnstormer website from the weather station at the Ray Brickhouse RC flying field in New Waverly Texas (Figure 1). The web page should look like Figure 2.

 

 

 

A Cellular Hotspot

How did the data get to the web site and how did we go about displaying it in a web browser?

The first thing you need to know is that Ray Brickhouse field is located in a very small town. There is no internet service in this town. But, Interstate 45 runs real close to it. This means we have terrific cell service. The first hurdle was to establish internet service at the field (inexpensively).

After quite a bit of research we decided to buy a Unite mobile hotspot from AT&T. We also purchased a pay as you go data service. The hotspot cost was about $60 and the service cost is $25 per month for 2G of data.

I know, 2G is not a lot of data, and one visit to YouTube would use it up very quickly. But we don’t make the service available for use at the field. It is only used to transmit weather data to the website.

The hot spot gives us internet access but it introduces a different problem, a hot spot sits on two networks. A local network (where the weather station resides) and a remote network (where the internet is). A hot spot is designed so that computers on the local side of the hotspot are allowed to initiate conversations with servers on the remote side of the hot spot. However, servers on the remote side cannot initiate conversations with servers on the local side.

This is a problem because if I have a web site on the internet. How would I go about asking a weather station on the local side for data?

There are a few possible solutions for this problem including “Port forwarding” (which introduces its own issues).

Enter the Raspberry Pi

The solution that we decided to implement was to place a server on the local side of the hotspot that can initiate communications with both the weather station and the website. Figure 3 shows the setup.

Referring to Figure 3. On the left is the website. It is on the internet. On the right is the weather station. It is on the local net.

In the middle is the hotspot (in the yellow case) and to its left is a Raspberry Pi (in the blue case) The USB cable between them is used to power the hotspot from the Pi. I removed the battery from the hotspot to keep it from overcharging.

A Raspberry Pi is a small computer (3.3” x 2.2”) that runs Linux. Using this computer, we can ask the weather station (on the local net) for weather data. Then, we can initiate a conversation with the website (on the internet). We can transmit the weather data to the web site.

We use the Raspberry Pi to collect weather data at specific intervals (every 60 seconds) using the OurWeather REST interface.  The Pi sends a request to the weather station. This request asks the weather station to send all data available to the Pi. The information received from the weather station is saved on the Pi as a file (wthrdata.dat). This file is then sent to the webpage and stored in a directory accessible to web pages on the website.

On the website, there is a webpage that uses java script to extract the data from the file sent by the Pi and display it on a web page.

Let’s look at the steps one at a time.

1. 1) The Raspberry Pi uses “HTTP” (using a REST interface) to ask the weather station to send all information it has. This request looks like this: https://12.34.56.78/FullDataString (the address 12.34.56.78 is the TCP/IP address of the weather station). This request returns a string of information that the Pi saves in a file called wthrdata.dat.
2. 2) The Pi then makes a connection to the website and sends the file to the website.

How exactly is all of this done? Well, The Unix operating system (Linux is a form of UNIX and is the operating system on the Raspberry Pi) has a file transport program built into the OS called FTP (File Transfer Protocol). Using FTP we can make a connection to the website on the internet and “transfer” the “wthrdata.dat” file to the website. Once the file is on the website we can just grab it with a browser and display it.

How did we get the file from the weather station? We send a JSON request to the weather station and the weather station responds with weather data (that we put into a file called wthrdata.dat.

Parsing the JSON on the Raspberry Pi

We wrote a PERL script (wrthrdata.pl) to do this. Figure 4 is an example of that script.

1. 1) Sets up the JSON request to gather the data from the weather station. The 192.168.0.100 address is the weather station address.
2. 2) Creates a file on the Pi called “wthrdata.dat” The line “print $fh $json_all” puts everything received from the weather station to into this file. The “close $fh” saves the file on the Pi

At this point we have created a file on the Pi (wthrdata.dat) with weather data in it.

3. 3) The information in these three lines are the login credentials for your website. We will use these credentials to FTP the weather file to the website.
4. 4) Here is where we establish an FTP connection to the web site.
5. 5) We tell FTP the location of the file to send to the website.
6. 6) We send the file to the website. And, the script ends.

Each time this script is run it gets the current data from the weather station and sends it to the website.

The script is ready but how do we use it. Well you can test it by logging into the Pi and typing
“perl wthrdata.pl” the script should run and there should be a “wthrdata.dat” file on the Pi AND on the website.

The CRON Script on the Raspberry Pi

To have the Pi run this script automatically once a minute you use CRON.

CRON is a UNIX time based job scheduler and can be used to run a script at certain intervals. You use “crontab -e” to access (and edit) this list. So, to add our perl script to this list you execute crontab -e and edit the file to include something like the following:

* 7-20 * * *  /usr/local/bin/perl /home/pi/wthrdata.pl

This entry tells the Pi to run the script every 60 seconds from 7am to 8pm.

The data is now being retrieved from the weather station, placed on the Pi and then sent from the Pi to the website every 60 seconds.

The Website Side – Getting the Code and Modifying the Source

And now, The website side. We need to find a way to extract the information out of the “wthrdata.dat” file and display it on the web site.

SwitchDoc Labs provides you with an assembly manual for the OurWeather station.  On page 34 they have you go to https://www.switchdoc.com/OurWeatherPage.html. This provides you which a page that allows you to connect to your weather station and retrieve data from it.

We use this page for a different purpose. And, that purpose is to “Borrow” code from it to create our own customized web page.

[Editors Note:   What Larry is doing is great!   This is why we provide open source software]

Use a browser and browse to the page (Figure 5) then right click in the center of the page and select “view source” and grab the code. I am using Microsoft “Edge” as a browser so Figure 6 is what my screen looks like.

Copy all of this code (ctrl A, ctrl C) and paste it into your favorite text editor (mine is notepad++). Now, you can customize this page to your hearts content. You can call it something like “test.html”

Let’s look at how the page works and how to make modifications.

The OurWeather Web Page Modifications

The first thing to remember is to make modifications in small steps. Test each modification. Then save the modification and go on to the next step. So, before you begin save test.html to test.cpy. then edit test.html. If your modification works save test.html to test1.cpy and so on…..

Using test.html. Refer to Figure 7. In Block number 1 you see the very top of the file. At line 7 is the <script> tag. This indicates the beginning of the java script that handles retrieving and manipulating of the weather data. The end of the script is down around line 336 where the </script> tag is, indicating the end of the java script.

Look at block 1 lines 9,10,12,13 etc. The “var” statements. These declarations are where the data from the weather station will reside. Right now, there is nothing in these variables.

Look at block 2 line 82. This is where the real work begins. The name of this “function” is “getCurrentOurWeather()”.  This function gets the data from the weather station (line 91). Decodes the data (line 92 through 94) and then saves each data item into the variables in block 1. Line 100 (block 2) is an example of getting the “time and date” data and placing it into the variable “SampleTimeStamp”. Notice that the time is retrieved from the “16^th” element of the data file.

Block 5 is the contents of the file “wthrdata.dat”. Look at the line 1 and look to the right until you see the first comma “,” now count the commas stating at zero 0,1,2,3,4,5,6 etc. at the 16^th comma look at the data to the left of the comma and you should see the date and time.

Block 6 is an example of how you would go about getting the data from the file instead of the weather station. Remember, the data is now in a file on the web site and we need to use it instead of getting it directly form the weather station.

Decoding the data file is pretty simple. Take a look at block 3. It is saying that the “Outdoor_Temperature” is at position “0” in the data. Look at block 5 and on line 1, the very first number you come to is the temperature (in “C” not “F”). Then in block 3 line 134 the temperature is converted from “C” to “F” with a little math. Finally, in block 3 line 135 the temperature is stored in “text” format in the “Outdoor_Temperature” variable.

To display it in a web page. Look at block 4 line 353 and you can see that the temperature (in the variable Outdoor_Temperature) is referenced as an “id” and displayed on the page.

If you want to see how we made the modifications to the switchdoc OurWeather html file you can simply browse to www.tri-countybarnstormers.com/weathsta.htm right click on the page and “view source”.

Pretty simple and really cool.

Thanks again for your interest.