Tutorial: Voice Time! OurWeather and Amazon Alexa – Part 1

We are pleased to announce Version 026 of the OurWeather software and the big change is that we now support the Amazon Alexa, Echo, Echo Plus, and all other Amazon Voice systems.   In this series of tutorials we will show you how to connect up the Alexa systems in your house to your OurWeather station.   We will be rolling this really cool feature set into our other major kit offerings including:

• – SkyWeather
• – Smart Garden System
• – The ThunderBoard Raspberry Pi IOT Lightning Detection Kit

It will be an exciting several months here at SwitchDoc Labs.

This is a five part instructional tutorial to get your OurWeather to Amazon Alexa Interface working.  All source code is provided!

While we are focusing on OurWeather in this series, the techniques we developed are good for many projects.

 

• - Part 1 - Voice Time!  OurWeather and Amazon Alexa - Part 1
• - Part 2 - Setting up PubNub MQTT Broker
• - Part 3 - The Alexa Skill
• - Part 4 - The AWS Lambda Serverless Function
• - Part 5 - Setting OurWeather MQTT to Alexa up and System Tests

 

What is OurWeather?

OurWeather is an ESP8266 based  connected weather station containing 7 different sensors that will teach about electronics, software and the weather.   It contains all the sensors and wires to make a fully functional, WiFi weather station.   No software programming is required, it works right out of the box. However, if you want to change the way OurWeather works, add your own software or do something entirely different, we have the Open Source software available.   And what is one of the coolest features of OurWeather?   Building the kit requires NO SOLDERING. It uses Grove Connectors.    It is all plug and play based building, with a full step-by-step photograph based instruction manual.  It is a safe and educational way to learn about weather, electronics and science topics.   You can watch our successful kickstarter video here: https://youtu.be/pw6hSNSQnsM

See the Grove Connector Tutorial Here.

 How does the OurWeather to Alexa Interface Work?

Below is a block diagram of how the system works to get OurWeather Information up to Alexa and how to respond to voice commands with real information.

 

The Alexa Voice to OurWeather Project

There are four major parts to this project.

• – PubNub – MQTT Broker between OurWeather and the AWS Lambda function
• – The Alexa Skill – Controls Alexa on Echos in your house
• – The AWS Lambda Serverless Function for brokering information from PubNub to the Alexa Skill
• – OurWeather – MQTT Publisher Weather Information to PubNub

 

What is MQTT?

Since the main transport protocol used in this project is MQTT, an explanation is in order.

MQTT is a publish-subscribe based “light weight” messaging protocol for use on top of the TCP/IP protocol, such as the WiFi packets that we are using in this project. It is designed for connections with remote locations where a “small code footprint” is required or the network bandwidth is limited.  The publish-subscribe messaging pattern requires a message broker. The broker (PubNub in this case) is responsible for distributing messages to interested clients based on the topic of a message.

Publish–subscribe is a pattern where senders of messages, called publishers (in this case our ESP8266 is the publisher), don’t program the messages to be sent directly to  subscribers, but instead characterize message payloads into classes without the specific knowledge of which subscribers the messages are sent to.   Similarly, subscribers will only receive messages that are of interest without specific knowledge of which publishers there are.  Mosquitto operates as the broker in this system and routes the published data to the appropriate subscribers.

You can think of MQTT as writing stories for a newspaper where you don’t know who will be subscribing to the article.

What is JSON?

JSON  is an open standard format that uses human-readable text to transmit data objects consisting of attribute–value pairs. It is the primary data format used for asynchronous browser/server communication, largely replacing XML.  XML is a “heavier” protocol that is also hierarchical in nature, but with a great deal more redundancy that JSON.  Yes, there are class wars going on for people that advocate JSON over XML, but in todays world of higher speed communication, it rarely matters.  You can make the argument that the higher data density of JSON is a better choice for IOT applications.

The Four Blocks of the Alexa Voice to OurWeather Project

PubNub

PubNub is an MQTT broker in the same sense as Mosquitto MQTT Broker is on a Raspberry Pi Platform.

PubNub is a global Data Stream Network (DSN) and realtime infrastructure-as-a-service (IaaS) company based in San Francisco, California. The company makes products for software and hardware developers to build realtime web, mobile, and Internet of Things(IoT) applications.

PubNub’s primary product is a realtime MQTT  publish/subscribe[2] messaging API built on their global data stream network which is made up of a replicated network of at least 14 data centers located in North America, South America, Europe, and Asia. The network currently serves over 300 million devices and streams more than 750 billion messages per month.

The foundation of PubNub is providing you the ability to integrate scalable, realtime data streams into your applications. Using the publish/subscribe paradigm, subscribers to a particular channel will receive any and all messages that are published to that channel. It doesn’t matter if there is one subscriber, 10 subscribers, 1000 subscribers or millions of subscribers, a published message will be delivered to all of those subscribers on that channel in less than ¼ second.

PubNub also supports BLOCKS micro services which can act like AWS Lambda Severless functions.  More on BLOCKS in future blog postings.

As in the block diagram above, data from OurWeather is sent via MQTT to PubNub.   PubNub stores a short history of messages (the last one will be grabbed by the AWS Lambda function below) as a topic (OWIOT1  is the channel in this case).

The basic level of PubNub (which we are using) is free.

The Alexa Skill

Alexa is Amazon’s voice service and the brain behind tens of millions of devices like the Amazon Echo, Echo Dot, and Echo Show. Alexa provides capabilities, or skills, that enable customers to create a more personalized experience. There are now tens of thousands of skills from companies like Starbucks, Uber, and Capital One as well as other innovative designers and developers.

We are building an Alexa skill that will interface to the OurWeather complete Weather Kit from Switchdoc Labs.   It provides the interface from your Amazon echo to the OurWeather system.   The Alexa skill manages the voice part of the project.   It takes verbal commands (“Alexa, ask OurWeather Status” which gives the current weather on YOUR OURWEATHER).  It is important to note that you will never publish your skill to the general market.  This is for all your Amazon Echos, etc. at your house wired to your Amazon account.   At our CTOs house, he has about 10 Alexa devices all wired into his Amazon account and wired into his various SwitchDoc projects, such as OurWeather, SkyWeather and the famous SmartGardenSystem.

An Amazon Alexa skill takes voice input from an Amazon Echo or other device, translates it to text, compares it agains what the skill is expecting (“utterances”) and then sends a JSON response to a specific application or server (we have used Apache servers in the past to receive this input, but this requires you to poke a hole in your home firewall and was really too complex for the normal user to implement).   This lead to our use of the “event driven” AWS Lambda function.  No server.  No firewall problems.  No software server on your site.  Much simpler.

The AWS Lambda Serverless Function

The Lambda function is the most difficult to understand part of this project.    Why do we need this?   An Amazon Alexa Skill is concerned with “utterances” and how to interpret them.   When you talk to Alexa, Alexa translates your speech to text, sends a request to the Lambda function.  The Lambda function interprets this Alexa JSON request, gathers the weather data via an MQTT history request  from PubNub (which came from OurWeather via MQTT), and sends a text response (JSON again) to Alexa with the required weather data embedded, and Alexa speaks the result.

AWS Lambda is an event-driven, serverless computing platform provided by Amazon as a part of the Amazon Web Services. It is a compute service that runs code in response to events and automatically manages the compute resources required by that code. It was introduced in 2014.

We are using node.js to write our Lambda Function.   Parts were very challenging (dealing with asynchronous callbacks in a environment that will terminate when you are done running code), but we got it to work perfectly.   You will find the open source code in Part 3.

The purpose of Lambda, as compared to AWS EC2 (cloud based servers), is to simplify building smaller, on-demand applications that are responsive to events and new information. AWS targets starting a Lambda instance within milliseconds of an event. Node.js, Python, Java and C# through .NET Core are all officially supported as of 2016, and other languages can be supported via call-outs. However, some runtimes, such as the Java Virtual Machine, may be slower than others to start.

Below is a JSON request from the Alexa Skill to the Lambda Function. Note the Lambda function is event driven, so it wakes up with this request, gets the weather data via MQTT from PunNub, builds the response and sends it back to Alexa (with confidential stuff redacted):

 

{
  "session": {
    "new": true,
    "sessionId": "SessionId.f02ae7fb-5045-4c86-8937-b44a74e63f6e",
    "application": {
      "applicationId": "amzn1.ask.skill.1XXXXX"
    },
    "attributes": {},
    "user": {
      "userId": "amzn1.ask.account.XXXXXXXX"
    }
  },
  "request": {
    "type": "IntentRequest",
    "requestId": "EdwRequestId.3560fc59-308d-41a4-8396-2e56d4781ce3",
    "intent": {
      "name": "temperature",
      "slots": {}
    },
    "locale": "en-US",
    "timestamp": "2017-12-30T00:45:17Z"
  },
  "context": {
    "AudioPlayer": {
      "playerActivity": "IDLE"
    },
    "System": {
      "application": {
        "applicationId": "amzn1.ask.skill.111c1582-8cc3-43af-afb9-984fb01f805a"
      },
      "user": {
        "userId": "amzn1.ask.account.XXXX"
      },
      "device": {
        "supportedInterfaces": {}
      }
    }
  },
  "version": "1.0"
}

And the JSON reply to the Alexa Skill from the Lambda function:

{
  "version": "1.0",
  "response": {
    "outputSpeech": {
      "type": "PlainText",
      "text": "Outside Temperature is 25.9 degrees F"
    },
    "shouldEndSession": true
  },
  "sessionAttributes": {}
}

 

A valid question at this point, is why the Lambda function doesn’t just subscribe (via MQTT) to the Weather Data channel (OWIOT1) on PubNub?    The reason is that we don’t want the Lambda function to be running all the time (which you need to be running to be a subscriber), we want the Lambda function to awake only on a request from Alexa.   Lambda functions are event driven from triggers.   Alexa is our trigger in this case.   On a technical note, Lambda functions wake up “stateless” every time.  The function has no execution to execution built int to the function.   You can use an external database (Amazon Elastic Beanstalk for example), but that just increases the cost.  Lucky for use, the “state” information we have (the weather information) is stored at PubNub and our Lambda function just grabs it there in the PubNub channel history.

AWS Lambda supports securely running native Linux executables via calling out from a supported runtime such as Node.js.

AWS Lambda usage below a threshold (which we will be under) is free.

OurWeather MQTT Interface

OurWeather is an single board computer ESP8266 based weather station combined with a bunch of different weather sensors.  We use the Arduino IDE to program this device and the open  source Arduino code is up on our GitHub.com repository here.  As of Version 026, the Alexa Interface is supported.    This will be up on the automatic update server for OurWeather shortly.

After you set up the PubNub keys on OurWeather, you will periodically (every 5 minutes) update PubNub via MQTT.

JSON Data Payload

Here is an example of the data packet we are using in the OurWeather code in JSON for the MQTT payload:

 {
  "FullDataString": "-3.30,67.40,23.24,103156.00,503.18,15.56,0.00,315.00,5.00,5.04,15.56,0.00,0.00,0.00,315.00,0,01/03/2018 09:45:39,SwitchDoc Labs,0,-1,0.00,0.00,0.00,0.00,0.00,0.00,3.62,36.40,4.67,14.40,0.00,27.20,V:0,WXLMG ,1"
}

The definition of the FullDataString is given in the OurWeather Advanced Usage Manual, under the downloads section of the OurWeather Product Page.

Coming Next

In Part  2, we set up and test the PubNub MQTT broker account.

In Part 3, we get a developers account on Amazon, and set up the Alexa Skill.

In Part 4, we set up the Lambda function and tie it all together.