Tutorial: IOT / Installing and Testing Mosquitto MQTT on the Raspberry Pi for Buster

February 5, 2018 ESP8266, IOT, MQTT, Raspberry Pi, SwitchDoc Products, Tutorials

Tutorial: IOT / Installing and Testing Mosquitto MQTT on the Raspberry Pi for Buster

As some of you may know, SwitchDoc Labs has writtena book on the IOT, “Raspberry Pi IOT Projects”.   The final project in the book is building an IOT RFIDIMG_7188 2 reader based on the ESP8266.  One of the major parts of that chapter is how to hook up the ESP8266 to the Raspberry Pi using the MQTT protocol, specifically the implementation called Mosquitto.

What is MQTT?

MQTT is a publish-subscribe based “light weight” messaging protocol for use on top of the TCP/IP protocol, such as the WiFi packets that we are using in this project. It is designed for connections with remote locations where a “small code footprint” is required or the network bandwidth is limited.   Both of these conditions are met with an ESP8266 IOT design, so it makes sense to use.  There is also an excellent library available for MQTT for the Arduino IDE  https://github.com/knolleary/pubsubclient.  The publish-subscribe messaging pattern requires a message broker. The broker is responsible for distributing messages to interested clients based on the topic of a message.

mqttorg-glowPublish–subscribe is a pattern where senders of messages, called publishers (in this case our ESP8266 is the publisher), don’t program the messages to be sent directly to  subscribers, but instead characterize message payloads into classes without the specific knowledge of which subscribers the messages are sent to.   Similarly, subscriberswill only receive messages that are of interest without specific knowledge of which publishers there are.  Mosquitto operates as the broker in this system and routes the published data to the appropriate subscribers.

You can think of MQTT as writing stories for a newspaper where you don’t know who will be subscribing to the article.

MQTT Projects

We have a number of MQTT Projects and tutorials on SwitchDoc Labs:

 

JSON Data Payload

JSON  is an open standard format that uses human-readable text to transmit data objects consisting of attribute–value pairs. It is the primary data format logo-jsonused for asynchronous browser/server communication, largely replacing XML.  XML is a “heavier” protocol that is also hierarchical in nature, but with a great deal more redundancy that JSON.  Yes, there are class wars going on for people that advocate JSON over XML, but in todays world of higher speed communication, it rarely matters.  You can make the argument that the higher data density of JSON is a better choice for IOT applications.

Here is an example of the data packet we are using in the ESP8266 Bluemix code in JSON for the LightSwarm data payload:

{"d":
      {
         "LightSwarm IOT":"LS1",
         "sampleCount":2118,
         "lightValue":383
      }
}

 

Mosquitto

There are a number of MQTT brokers available for different machines.   For this project, we have selected one of the most popular and stable brokers, MQTT-Mosquitto2“Mosquitto”.   Note the two “t”’s in Mosquitto.  The bane of spell checkers everywhere.

Mosquitto supports MQTT v3.1/3.1.1 and is easily installed on the Raspberry Pi and somewhat less easy to configure.   Next we step through installing and configuring the Mosquitto broker.

Installing the MQTT  “mosquitto”

Unfortunately, the Raspberry Pi normal “apt-get” archives do not contain the latest version of the Mosquitto software.  If you don’t install the latest version of the broker, you will get odd errors (because of version compatibility errors) and it will not work.  So, the first thing is to open a terminal window (or log in using ssh) to your Raspberry Pi and do the following:

sudo wget https://repo.mosquitto.org/debian/mosquitto-repo.gpg.key
sudo apt-key add mosquitto-repo.gpg.key
cd /etc/apt/sources.list.d/
sudo wget http://repo.mosquitto.org/debian/mosquitto-buster.list
sudo apt-get update

Next we can install the three parts of Mosquitto proper.

  • mosquitto – the MQTT broker (or in other words, a server)
  • mosquitto-clients – command line clients, very useful in debugging
  • paho-mqtt – the Python language bindings

 

If you’re going to use MQTT in a Python project, you’ll have to install paho-mqtt, which replaces the old Mosquitto Python module.

sudo apt-get install mosquitto mosquitto-clients
sudo apt-get install python-pip	
sudo pip install paho-mqtt

As is the case with most packages from Debian, the broker is immediately started.  Since we have to configure it first, stop it.

sudo /etc/init.d/mosquitto stop

 

Configuring and Starting the Mosquitto Server

Before using Mosquitto, we need to set up the configuration file. The configuration files is located at /etc/mosquitto.

Open the file as follows:

sudo nano /etc/mosquitto/mosquitto.conf

You should see the following:

# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

pid_file /var/run/mosquitto.pid

persistence true
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

include_dir /etc/mosquitto/conf.d

Change the “log_dest” line to:

log_dest topic

This puts the logging information as a “topic” so we can subscribe to it later on to see what is going on in our IOTRFID system.

Next add the next six lines:

log_type error
log_type warning
log_type notice
log_type information

connection_messages true
log_timestamp true

Now your /etc/mosquitto.conf files should look like:

# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

pid_file /var/run/mosquitto.pid

persistence true
persistence_location /var/lib/mosquitto/

log_dest topic


log_type error
log_type warning
log_type notice
log_type information

connection_messages true
log_timestamp true

include_dir /etc/mosquitto/conf.d

MQTT-Mosquitto2Starting the Mosquitto Server

Now start the mosquitto server:

sudo /etc/init.d/mosquitto start

Testing the Mosquitto server

 

Open up two more terminal windows.

In Terminal window 1 type:

mosquitto_sub -d -t hello/world

In Terminal window 2 type:

mosquitto_pub -d -t hello/world -m "Hello from Terminal window 2!"

When you have done the second statement you should see this in the Terminal 1 window.

~ $ sudo mosquitto_sub -d -t hello/world
Client mosqsub/3014-LightSwarm sending CONNECT
Client mosqsub/3014-LightSwarm received CONNACK
Client mosqsub/3014-LightSwarm sending SUBSCRIBE (Mid: 1, Topic: hello/world, QoS: 0)
Client mosqsub/3014-LightSwarm received SUBACK
Subscribed (mid: 1): 0
Client mosqsub/3014-LightSwarm received PUBLISH (d0, q0, r0, m0, 'hello/world', ... (32 bytes))
Greetings from Terminal window 2

Now you are running the Mosquitto broker successfully.