Setting Up an MQTT Broker. Part 2 : IoT, Home Automation

We will be using the Mosquitto MQTT Broker from Eclipse as its free and runs under Linux. It is possible to use a PC based MQTT Broker however I specifically wanted to use a Raspberry Pi as they are cheap, physically very small and pack a large punch where processing power is concerned, better still I have a load of them handy. I used a B+ Pi.

Preamble

I am assuming you have followed the steps in my earlier Instructable named 'IoT - Intranet of Things : Home Automation. Part 1 : IoT, Introduction' and have a Raspberry Pi already set up and ready to go.

All you need to do is change the host name from 'MYHOSTNAME' to 'MQTTSVR'. You can do this by using PuTTY to connect to the Pi and from the command line enter;

• sudo raspi-config

Select (9) Advanced Options then (A2) Hostname.

Enter 'MQTTSVR' and save the changes. Exit then choose to reboot the Pi

You can now re-connect PuTTY to this Pi by using the new hostname 'MQTTSVR.local'. Create a PuTTY saved session named 'MQTTSvr', it will come in handy later.

Installation

The installation process is as follows;

Connect to your new MQTT server via PuTTY and remembering to copy from this Instructable and paste into PuTTY to avoid having to type any commands by hand.

Ensure you are in the home directory. If you are you will see the following prompt

pi@MQTTSVR:~ $

If you don't see this enter

• cd ~

Now complete the following commands;

• wget http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key

then;

• sudo apt-key add mosquitto-repo.gpg.key

Now make this repository available to apt;

• cd /etc/apt/sources.list.d/

As we are using the Raspberry Pi Jessie Debian image enter the following;

• sudo wget http://repo.mosquitto.org/debian/mosquitto-jessie.list

Now enter the proceeding commands, which may take some time to complete. Also answer Yes (Y) to any questions;

• sudo apt-get update
• sudo apt-get upgrade

To complete the install process enter the following, answering Yes (Y) to any questions;

• sudo apt-get install mosquitto

Let the Pi do its thing and complete where you will see the command prompt.

pi@MQTTSVR:/etc/apt/sources.list.d $

.

.

That's it you're done. Mosquitto has now been installed as a service and will automatically start when you boot your Raspberry Pi.

Now we need to test it works.

.

Note : Before you remove power from your MQTT Broker enter the following command and wait for the Pi to shutdown.

• sudo shutdown now

.

Hint

If you want to quickly connect to your Raspberry Pi server using PuTTY without having to type in username and password each time. Do the following, assuming you have already created a saved session named 'MQTTSvr' as in Step 7, 'Part 1 : IoT, Automation' and your MQTTSVR username and password are unchanged from the example given.

Locate your PuTTY install and do the following;

1. Create a short cut to it.
2. Right click on the short cut
3. Select the 'Shortcut' tab
4. In the text box named 'Target:' Enter the following after the text "...\putty.exe"
   • -load MQTTSvr -l pi -pw raspberry
5. So in my case I installed to "C:\Program Files (x86)\PuTTY" the text will read;
   • "C:\Program Files (x86)\PuTTY\putty.exe" -load MQTTSvr -l pi -pw raspberry
6. Now drop this short cut on your desktop and double click each time you want to open a PuTTY session.

This is a good time to have a quick discussion regarding MQTT Topics.

What is a Topic?

A topic is a string made up of UTF-8 (Unicode Transformation Format 8-Bit) characters and is the data sent by a Publisher to a Broker to be forwarded on to any Subscribers. In short its a basic text string.

How are they constructed?

Topics are broken down into one or more topic level each with a minimum of at least 1 character. These levels are separated by a forward slash '/' and can include ' '(spaces).For example (not including the quote marks);

'this/is/a/six/level/topic example with spaces'

Though arguably not very useful topic, hopefully gets the point over. Topics are case sensitive so;

'test/topic'

is different to;

'Test/Topic'

This is a single topic

'x'

as are these;

' ' (space) and '/'

There is no special action necessary to create a topic, the act of subscription or publication does this for you.

And that's about it.

Is there a special format or best practice for constructing topics?

The short answer is 'no' and there are lots of views surrounding how you should make them up (and I will cover the topic in a later Instructable). However you would need to consider the following when creating your topics;

Keep them short. They are sent over the network and need to be parsed (read) by some device/broker/server, so the longer and more intricate you make them the harder they are to decode.

Where you can, use meaningful names. It makes debugging easier. As a human its a lot simpler to type, or read 'sensor1' than 'Wcp10X!3wdrka2'.

Make them have meaning to you. The examples I have seen tend to be like a kind of address such as;

'myhouse/first floor/bedroom one/ceiling/light'

This assumes a fixed location IoT device and for the most part this will be true, though this type of nomenclature fails when considering mobile or re-deployable IoT devices, such as a temperature humidity sensor, or switchable extension socket. Should you decide to move a switchable extension socket from the living room to the bedroom to replace a failed device, you will need to reprogram the IoT device to respond to its new location to be consistent.

For my series on IoT based home automation I will be using topics which refer to the device only such as;

'/EthernetDevice/Led1Command'

or

'/WiFiDevice/Temp1Status'

This is because I ultimately want to use a MySQL database to hold the configuration and IoT device information. Which I will hopefully cover later.

Oh, and one final thing, it is usually convention not to precede the first topic with a '/', but you can see I've ignored this. :-)

Are there any short cuts?

Yes, MQTT offers what is know as topic 'wild cards'. These are special characters a subscriber can place into a subscribing topic to reduce the need to subscribe to many items. There are two types of wildcard characters; '+' known as 'Single Level' and '#' for 'Multi Level'. The Picture above gives examples of both.

A word of caution. If you use the '#' multi level wild card and have a lot of IoT devices in your system, then be prepared for a lot of messages!

Are there any special topics?

Yes, topics beginning with '$' are reserved for internal statistics of the MQTT Broker. At the time of writing this Instructable there is no clear official standardisation of topics that must be published by a Broker. However it is common practice to prefix with '$SYS/' for this type of information. The mosquitto MQTT Broker conforms to this '$SYS/broker/#'. (see Linux mosquitto man page for full details. ie. man mosquitto)

So if you want to see all the mosquitto MQTT Broker statistics you would need to subscribe MQTTSpy to the following (see next steps for further details);

'$SYS/broker/#'

3 More Images

Preamble

To test our new MQTT Broker we will first install a client which can subscribe and publish to a given topic on the Raspberry Pi. We will then monitor this exchange via tool named MQTTSpy from the PC (just like in the first picture above).

Why, test the install? Well it could save you a lot of time in the long run. To correctly set up a robust home automation system you start to rely upon and to which in the fullness of time be adding lots of useful IoT clients to, you need to make sure that all the links in the chain are working correctly, that way if you observe an issue with your new IoT device you can quickly rule out as many systems as possible and zero in on the actual issue.

Installing Pub/Sub Clients (Raspberry Pi)

From a PuTTY window enter the following command on your MQTT Broker. Ensure you are in the home directory;

pi@MQTTSVR:~ $

if not enter;

• cd ~

followed by;

• sudo apt-get install mosquitto-clients

.

Installing MQTTSpy (PC)

In order to successfully run MQTTSpy you will need to have the Java Runtime Engine (JRE) greater than version 8 installed on your PC.

To check what version of the JRE you have installed, run up a 'dos in a box', ie a command line window (picture above).

From the Start button, enter into the search box with the grey text 'Search programs and files'

cmd

and hit enter. A dos in a box will appear into which you will need to type;

java -version

It will return 'java version "W.X.Y.ZZ"' and some other 'guff' (picture above) or an error. if you have anything other that greater than version 8 (actually 1.8.y.zz) you will need to follow these instructions;

Go to this Oracle downloads web page.

https://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html

Select the 'Accept Licence Agreement' radio button, download and install the 'Java SE Runtime Environment 8u92', jre-8u92-windows-x64.exe (as in the picture above).

Ok, now go to the MQTTSpy download page given below;

https://github.com/kamilfb/mqtt-spy/wiki/Downloads

Select and download mqtt-spy jar file (picture above). At the time of writing version 0.5.0 was the latest available.

Save your 'mqtt-spy-0.5.0-jar-with-dependancies.jar' file to any directory.

To get the software to run you can double click it. It this doesn't work you can enter the following from the command line, in the directory you saved the jar file.

java -jar mqtt-spy-0.5.0-jar-with-dependancies.jar

Alternatively you could unzip and drop the RunMQTTSpy.bat file (given below) into the same directory and double click this to run MQTTSpy each time.

.

Ok, you 've now installed all your test software, let's configure it and run a test.

.

Hint

To open a command prompt at a specific directory do the following;

1. Open windows explorer
2. Navigate to the directory of interest
3. Press and hold 'Alt' and 'Shift' keys
4. Right click on the directory of interest
5. From the dropdown menu select 'Open command window here'

Out of the box, the minimal configuration required by MQTTSpy is as follows;

1. Connecting to your MQTT Broker
2. Subscribe to a given topic
3. Optionally, publish to a given topic

.

Here the video above shows how to connect to our mosquitto MQTT Broker 'MQTTSVR' and add a wild card to view all test topics and test the MQTT Broker. The running order is as follows;

1. Open two auto login PuTTY sessions
2. Start MQTTSpy by double clicking the 'RunMQTTSpy.bat' file (I've done the video as if you have just used MQTTSpy for the first time, so you can see the initial configuration)
3. Create a standard configuration
4. Create a new MQTT Broker connection which we will connect to our Broker
5. Enter the Broker details 'MQTTSVR.local' and give it a client ID 'Win 71'
6. Then connect to the Broker 'Open connection' and save this new set up so we can use it later.
7. Select this Broker tab, which has gone 'GREEN' to show it has connected to MQTTSVR
8. Make a subscription to all test topics (with multi level wild card)
   • '/test/#'
9. From a PuTTY session window make a subscription to the test topic (no wildcards).
   • mosquitto_sub -t "/test/Topic"
10. From the other PuTTY session window publish a message to the test topic.
    • mosquitto_pub -t "/test/Topic" -m "Hello World"
11. You will now see this Hello World message arrive at both our client subscribers (PuTTY and MQTTSpy)
12. Now to prove we can publish a message from a client not located on the MQTT Broker we copy the test topic to the publish box in MQTTSpy and send the message 'SteveQuinn' to the MQTT Broker
13. You can now see this message arrive both at the PuTTY subscribed client and the MQTTSpy client (here MQTTSpy has become both subscriber and publisher).
14. Finally we close the connection to the MQTT Broker from the control panel and quit MQTTSpy.
15. Followed by closing the two PuTTY sessions. Remember to use ^C (Ctrl C) to quit the subscription before quitting.

.

Ok, so now we are accomplished MQTT Broker testers. What does this give us?

If we are having any issues with our Home Automation set up, we can now quickly run up MQTTSpy and check to see if the Broker is healthy or any published messages are 'malformed' etc. We can even publish messages, meaning we can test any IoT device we create.

Although in the examples I have given above I use PC based software to carryout my testing equally it is possible to use a smartphone app.

Pictured above is one such application named 'MQTTool' for the iOS. Although not as feature rich as MQTTSpy it is easy to use and allows you to do some basic testing.

3 More Images

From the play store I could see quite a few free apps, I tried many out but found the app pictured above to be the best of them 'MQTT Dashboard'. Though you need to put in an IP Address not the host name 'MQTTSVR.local' it does have a better method to publish to topics, allowing you to create text boxes, switches, sliders etc.

.

I included a screen grab from the Pi detailing how to get the IP Address of the Pi. To do this, connect via putty and enter;

• ifconfig

The Pi will respond with the screen shot above and I have highlighted the IP address you will need to enter into MQTT Dashboard. I have assumed you are connecting to your network via Ethernet. If you have connected via WiFi read 'wlan0' for 'eth0'

In the next three Instructables we will create three simple IoT devices for your Intranet which are based around the ESP8266-01 and the Arduino ATMega2560 with Ethernet Shield. The first of which we will revisit to improve the design and extend its use to show just how easy it is to make your own IoT devices.

I used the following references to put this Instructable together;

Official MQTT site;

• http://mqtt.org

MQTTSpy Home Page;

• http://kamilfb.github.io/mqtt-spy/

MQTTSpy Downloads, Getting Started Guide and further reading;

• https://github.com/kamilfb/mqtt-spy/wiki/Downloads
• https://github.com/kamilfb/mqtt-spy/wiki/GettingStarted
• https://github.com/kamilfb/mqtt-spy/wiki/ScriptingPublications
• https://github.com/kamilfb/mqtt-spy/wiki/Configurations
• http://www.hivemq.com/blog/mqtt-toolbox-mqtt-spy-advanced
• http://www.hivemq.com/blog/mqtt-essentials-part-3-client-broker-connection-establishment

Good place for MQTT Clients;

• http://www.hivemq.com/blog/seven-best-mqtt-client-tools

MQTT Spy advanced usage

• http://www.hivemq.com/blog/mqtt-toolbox-mqtt-spy-advanced

Mosquitto Eclipse Home Page

http://projects.eclipse.org/projects/technology.mosquitto

Mosquitto MQTT Broker details, downloads, install process etc.;

• http://mosquitto.org/2013/01/mosquitto-debian-repository/

Good background material

• http://www.hivemq.com/blog/mqtt-essentials/

Good source of best practice for Topic naming conventions

• http://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices
• https://openhardwarecoza.wordpress.com/2015/03/29/openhab-mqtt-arduino-and-esp8266-part-2-publish-subscribe-command-state-and-wtfs/

Google groups protocol discussions

• https://groups.google.com/forum/#!forum/mqtt

UTF-8 details

• https://en.wikipedia.org/wiki/UTF-8