Tutorial for Upgrading SkyWeather2 – The Dow Hurst
Average
Editors Note: Dow has done it again! Here he shows us how to update SkyWeather2 on the Raspberry Pi. He is an active participant on forum.switchdoc.com and has been a huge help to all levels of users!
I’ve noticed in the forum that several people have had trouble figuring out how to upgrade the SkyWeather2 software that came with their system. I culled through the forums to base the following instructions on my own and others experience in successful upgrades. Plus, with all the work SwitchDoc Labs is putting into finding bugs, incorporating user code to improve the software, and adding functionality, there is no point in leaving your system with the original software. There’s lots of good stuff waiting in a newer version!
Backing up your current version
First method is a simple copy and paste from the command line on your pi. (If you don’t know how to get to a command line, we have that answered in a tutorial-for-using-ssh-to-grep-logs)
Let’s ssh to the pi and make sure you are in the right place. Also, my example gives a specific name to the copied current version so you know what your backup represents. I prefer naming the backup directory with either ver0xx or mod0xx to let me know what version of SkyWeather2 is captured there and whether I modified any of the python code. The second command lets you know what version you have in the source directory for the backup. The third command copies recursively the directory and everything inside of it to the newly named backup copy. It also preserves date stamps and permissions. I had modified certain python scripts with info from the forum, so I included _mod024 at the end of the name of the backup copy since that was the version I was backing up.
cd /home/pi
grep “config.SWVERSION” ./SDL_Pi_SkyWeather2/SkyWeather2.py
cp -rp ./SDL_Pi_SkyWeather2 ./SDL_Pi_SkyWeather2_mod024
Stop SkyWeather2 and possibly dash_app
We need to stop the current SkyWeather2 python script, and if the dash_app is running, then both. After this we will remove the SDL_Pi_SkyWeather2 directory and pull down a fresh copy of the latest software using git. First, we need to know what the PID, or process identifier number is for those numbers and then use the kill command to stop them. The command below lists all the running processes for all users, pipes the output into a search and retrieve command called grep that is looking for any line containing the string “py.” Then, that is output is also piped into another grep command that is looking for the string “SDL”, which isolates the exact lines we want. The numbers highlighted in red in the output below are the PIDs of the processes.
ps aux | grep py | grep SDL
root 21759 0.9 0.8 194480 33088 ? Ssl Jul17 11:44 /usr/bin/python3 /home/pi/SDL_Pi_SkyWeather2/SkyWeather2.py
root 21791 6.8 3.5 213736 138176 ? Ss Jul17 88:22 /usr/bin/python3 /home/pi/SDL_Pi_SkyWeather2/dash_app/index.py
Now that we know the PIDs of the SkyWeather2 and dash_app processes we can use the kill command to stop them from running. Of course, the assigned PIDs will be different every time SkyWeather2 is started, whether upon reboot or manually.
kill -15 21759 21791
If you want to double check the processes have been terminated, just rerun the previous command on your pi by pressing the up arrow, ↑, on your keyboard until the previous command appears, then press Enter to run it. The linux terminal remembers previously issued commands accessible via the up and down arrows, or the “history” command. You can quickly access previously issued commands multiple ways. I recommend you read the history manual page, just issue the command “man history” and press “q” to exit. There is a lot of useful information contained there. Also, the kill command has many signals it can send to a process. I selected the terminate signal. To see a list of all the signals, try the following:
kill -l
By the way, that is the letter L in lowercase, not the number one!
Cloning with git
Let’s address a couple of gotcha’s right now. Will your SkyWeather2 pi reach the internet to access a git repository? If not, then you will need to troubleshoot that now. I’d run the following test and if doesn’t work, post on the forum. This is a reasonable internet access test to find out if your pi will resolve a DNS name to an IP address and access the SkyWeather2 git repo:
ping github.com
PING github.com (140.82.113.4) 56(84) bytes of data.
64 bytes from lb-140-82-113-4-iad.github.com (140.82.113.4): icmp_seq=1 ttl=53 time=15.4 ms
64 bytes from lb-140-82-113-4-iad.github.com (140.82.113.4): icmp_seq=2 ttl=53 time=15.6 ms
64 bytes from lb-140-82-113-4-iad.github.com (140.82.113.4): icmp_seq=3 ttl=53 time=15.5 ms
git ls-remote -h https://github.com/switchdoclabs/SDL_Pi_SkyWeather2.git
5e433b69218e92c23d3618960d88e1ea89555557 refs/heads/master
Next, do you have the information in the JSON file available? We just made a backup copy of that if you had already configured and run SkyWeather2 previously. If you hadn’t configured SkyWeather2 during the setup process, then your fine. You will just configure the new version when finished with the upgrade.
cat ./SDL_Pi_SkyWeather2_mod024/SkyWeather2.JSON
{“key”: “value”, “ProgramName”: “SkyWeather2”, “ConfigVersion”: “001”, “SWDEBUG”: false, “enable_MySQL_Logging”: true, “MySQL_Password”: “password”, “enable_WLAN_Detection”: true, “PingableRouterAddress”: “192.168.1.1”, “mailUser”: “foobar@gmail.com”, “mailPassword”: “xyxyxyxyxyxy”, “notifyAddress”: “foobar@gmail.com”, “fromAddress”: “foobar@gmail.com”, “enableText”: true, “textnotifyAddress”: “6365553226@vtext.com”, “runLEDs”: false, “SolarMAX_Present”: false, “BMP280_Altitude_Meters”: “267.0”, “Sunlight_Gain”: “High”, “USEWEATHERSTEM”: false, “INTERVAL_CAM_PICS__SECONDS”: “120”, “STATIONKEY”: “xyxyxyxy”, “WeatherUnderground_Present”: false, “WeatherUnderground_StationID”: “FOOBAR555”, “WeatherUnderground_StationKey”: “xyxyxyxy”, “USEBLYNK”: true, “BLYNK_AUTH”: “xyxyxyxyxyxyxyxyxyxyx”, “USEWSLIGHTNING”: false, “USEWSAQI”: false, “USEWSSKYCAM”: true, “ALTWSSKYCAM”: false, “DustSensorSCL”: “20”, “DustSensorSDA”: “21”, “DustSensorPowerPin”: “5”, “GPIO_Pin_PowerDrive_Sig1”: “4”, “GPIO_Pin_PowerDrive_Sig2”: “4”, “WATCHDOGTRIGGER”: “6”, “REST_Enable”: false, “Camera_Night_Enable”: false, “Camera_Rotation”: 0, “MQTT_Enable”: false, “MQTT_Server_URL”: “localhost”, “MQTT_Port_Number”: “1883”, “MQTT_Send_Seconds”: “500”, “English_Metric”: false}
All those unique key value pairs in the file that are unique to your SkyWeather2 pi will need to be copied over. I have been able to just copy my JSON file into the new directory after issuing the git clone operation we will perform shortly. Don’t sweat this, just make sure you have a copy present of the JSON file in your backup directory if you had already configured and run SkyWeather2.py.
We want to remove the current SDL_Pi_SkyWeather2 directory at this point.
rm -rf /home/pi/SDL_Pi_SkyWeather2
For all you worriers, notice how I put the full path to the directory in the command above? That guarantees we are removing the specific directory we want. It is very possible to not pay attention to the current working directory you are in and inadvertently recursively remove directories and files we do not want to remove. Raise your hands if you accidentally deleted the wrong file sometime in your life! It is a good idea to not use wild card characters such as * or use relative paths, like “../../somedir/foobar” with the “rm” command, as this is prone to disaster.
Now, to the meat of what we are doing, running the git clone command. Make sure you are still in the pi user home directory and then run the git clone command.
cd /home/pi
git clone https://github.com/switchdoclabs/SDL_Pi_SkyWeather2.git
A new SDL_Pi_SkyWeather2 directory will be created and filled from the remote repository. This a clean unconfigured version of the latest SkyWeather2 software ready for you to work with. I love this procedure as any changes or tweaks we might have made in the old version of SkyWeather2 are no longer going to cause problems with a git upgrade of the software. And, you still have your backup of the old version if you want to play with that later.
Installing vcgencmd if needed
If you didn’t purchase the SwitchDoc Labs SD card, then you may need to install vcgencmd, a program that access the pi’s hardware. If I remember correctly, moving from version 024 to 025 required installation of this program. If you already have it, then the command output will look like this and won’t mess up anything. Do not worry if you get a notification about pip3 being an older version and don’t try to upgrade pip3 at this time. Also, using sudo will trigger a request for your pi user’s password, so enter it and press Enter when asked:
sudo pip3 install vcgencmd
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: vcgencmd in /usr/local/lib/python3.7/dist-packages (0.1.1)
If vcgencmd hasn’t been installed, it will be. You can try the command out now quickly to see why we want it:
sudo vcgencmd measure_temp
temp=45.7’C
This how the dash_app is able to acquire the information so it can show the pi’s CPU temperature on the SkyWeather Status webpage.
Configure Maria MySQL for the upgraded SkyWeather2 software
We need to update the maria mysql database to handle the new SkyWeather2 software in case there are new values. A side note for those not used to the command line, you can use filename completion by pressing the Tab key to speed up typing those long directory and filenames. After you type the first few letters of a path such as /hom, press Tab and the shell should complete the last e of the directory /home. Much longer names that don’t have duplicates are where this shell trick really helps. If you have duplicate names out to a certain point in the name, you will have to type the next character and repeat pressing Tab to get the rest of the name completed. The third command below will require a password. The default, provided you haven’t already changed it, is “password” on the SwitchDoc Labs SD card.
cd /home/pi/SDL_Pi_SkyWeather2
sudo mysql -u root -p < WeatherSenseWireless.sql
sudo mysql -u root -p WeatherSenseWireless < updateWeatherSenseWireless.sql
Update the SDR rtl433 Radio Driver
Let’s not forget about important updates to the driver for the SDR rtl433 radio that may have occurred too. Here we are moving to the correct directory, performing an update to the already installed software tree with git by using the “pull” flag instead of cloning. We aren’t deleting the subdirectory, as I haven’t read in the forums of anyone modifying the SDR software and thereby needing to merge their code with updated git code changes. Git is made for managing changes to software by developers, but that is outside of the scope of this article. Also, I am not a developer and am not familiar with those aspects of git. In the commands below, the cmake and make commands will take some time to run on the pi, just be patient. Do not worry about warnings produced in the output as they are normal. Errors will stop the compilation of the software, but warnings are usually just the compiler complaining about minor issues that don’t prevent the software from compiling properly.
cd /home/pi/SDR/rtl_433
git pull
cd ./build
cmake ../ && make
sudo make install
Woohoo! You are almost done! Just one last procedure and a reboot ahead.
Configure SkyWeather2
Two different paths here: if you had already had a running SkyWeather2 system, then you can simply copy your JSON file from the backup into the new cloned directory.
cd /home/pi/SDL_Pi_SkyWeather2
cp -p /home/pi/SDL_Pi_SkyWeather2_mod024/SkyWeather2.JSON .
Notice that last command used the shell trick of using the “.” to represent the current working directory, which is the new cloned SDL_Pi_SkyWeather2 directory. Also, the “-p” flag to the cp command preserves the permissions and date stamp on the JSON file. Why not keep that valuable info of when you last modified the contents of the file? I think it is nice to hang on to info like that.
If you hadn’t run SkyWeather2 before then follow the directions in the manual to initially configure SkyWeather2. I believe this is the SkyWeather2 Configuration and Operations manual.
Having SkyWeather2 and the dash_app startup on boot is nice and guarantees that you won’t forget to start the software manually. Also, if you don’t have your pi plugged into a decent uninterruptible power supply, or UPS, then when the power drops due to an outage or storm, SkyWeather2 and dash_app won’t be running after the power returns (I can wax poetic on UPSes and how important they are). You may have, if upgrading, already modified the /etc/rc.local file. If not, and you’d like to have this feature, you will need to edit this file. We make a backup copy first in case we mess up in our edits and need to recover.
sudo cp -p /etc/rc.local /etc/rc.local.orig
sudo vi -C /etc/rc.local
Uncomment the lines highlighted in red below by deleting the # symbol at the beginning of the line. By deleting the # symbol the lines become active commands that will start SkyWeather2 on boot. Move the cursor with the arrow keys to the # symbol on the first of the three lines I’ve highlighted below. Press the “x” key to delete that # character. Press the down arrow to move down to the next # symbol and repeat. Once you have deleted those three # symbols use the vi command “:wq” to save the file and exit vi.
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will “exit 0” on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
# Print the IP address
_IP=$(hostname -I) || true
if [ “$_IP” ]; then
printf “My IP address is %s\n” “$_IP”
fi
#cd /home/pi/SDL_Pi_SkyWeather2
#nohup sudo python3 SkyWeather2.py &
#chmod 666 nohup.out
exit 0
If you want dash_app to start as well, we need to add the following lines below the three lines we just uncommented.
sleep 5
cd /home/pi/SDL_Pi_SkyWeather2/dash_app
nohup sudo python3 index.py &
chmod 666 nohup.out
Reopen the /etc/rc.local file if you closed it.
sudo vi -C /etc/rc.local
Move with the arrow keys so your cursor is on the line in rc.local:
chmod 666 nohup.out
Press the “o” key to move into input mode and add a new line below where you were originally. Now use your mouse to copy in your web browser the four commands I’ve provided above. Switch your mouse to the pi’s terminal and right click to open the menu allowing you to paste, or just middle click to paste the copied text. Save your changes by pressing the Esc key and the use the “:wq” command to save and exit vi. Your file should look like this now:
cat /etc/rc.local
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will “exit 0” on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
# Print the IP address
_IP=$(hostname -I) || true
if [ “$_IP” ]; then
printf “My IP address is %s\n” “$_IP”
fi
cd /home/pi/SDL_Pi_SkyWeather2
nohup sudo python3 SkyWeather2.py &
chmod 666 nohup.out
sleep 5
cd /home/pi/SDL_Pi_SkyWeather2/dash_app
nohup sudo python3 index.py &
chmod 666 nohup.out
exit 0
By the way, the “sleep 5” command just allows 5 seconds to pass before starting the dash_app. Sometimes giving a program time to get started, such as SkyWeather2, before starting another that relies on output from the first program is helpful (dash_app does need SkyWeather2). If you are using wifi instead of ethernet, then it may take the pi time to get wifi configured. Adding a sleep statement of up to 30 seconds before starting SkyWeather2 itself is helpful. I would not add it without first asking about any network problem you may have on the forum.
Now you can reboot the pi.
sudo shutdown -r now
When the pi finishes rebooting, log back in and check if SkyWeather2 and the dash_app (if you elected to have it run) are up.
ps aux | grep py | grep SDL
You should see a couple of processes matching SkyWeather2 and the dash_app, like we saw previously. If so, you can check the version of SkyWeather2 by looking at the first few lines of the log file it produces.
head -n 10 /home/pi/SDL_Pi_SkyWeather2/nohup.out
./SkyWeather2.JSON File exists
##########################################################
SkyWeather2 Weather Station Version 025 – SwitchDoc Labs
Program Started at:2021-07-17 19:54:19
##########################################################
b”
———————-
We are now running version 025 of SkyWeather2! I hope you have found this to be a relatively easy process to upgrade. If you ever want to look back or compare the current version with your old version of the software, you have your backup directory available. I’ve found this method of upgrading invaluable. As you gain experience you may find using git to manage your code changes more useful than just cloning as we have done here.
System Administration Advice
I would also recommend creating a Notes.txt file in your SkyWeather2 directory where you explain what your changes are and in which file you made them. It will be a breadcrumb trail for you to follow later when you are trying to remember what you modified and where. Please post a comment or PM me in the forum if you find any errors or have questions.
Conclusion
I hope this was a helpful tutorial on upgrading SkyWeather2 to new versions. As of this publication date, the latest version of SkyWeather2 is V026.03. John has told me that a V026.04 is coming in a few weeks with minor bug fixes and then V027 with all the new support for time lapses and the new Solar SkyCam WeatherSense instrument.