Legacy Satellite Configuration

This tutorial explains how to setup the legacy satellite configuration.

Please refer to this documentation to configure your satellites using the dedicated service snips-satellite.

Satellite Configuration

Forewords

This tutorial explains how you can use satellites (multi-room dialog) with your base device. For instance, if you want your base device in the living room and have a satellite in your kitchen and/or one in your bedroom and so on.

  • Base: The main Snips device hosting the intent detection engine.

  • Satellite: The vocal interface located in other room, running at minimum an audio server to stream the input sound to the base device.

On the following configurations, audio is continuously streamed over the network.

Hardware requirements

Satellite main feature is to act as a vocal interface for the base device and optionally also run the Wake Word detector and actions scripts to accomplish some actions locally.

Any devices such as the Pi Zero that are at least able to run snips-audio-server, snips-hotword and snips-skill-server can be used as a satellite.

This tutorial is based on the Raspberry Pi Zero configured as a satellite and a Raspberry Pi 3 configured as a base.

Those two boards are using a ReSpeaker 2-Mic HAT to capture and emit sound.

Choose your configuration

Depending on the different components running, there are mainly two configurations:

Configuration

Components running

Advantages

Audio Server Only

snips-audio-server

snips-skill-server (Opt.)

Minimum performance requirements for the satellites

Audio Server + Wake Word

snips-audio-server

snips-hotword-server

snips-skill-server (Opt.)

Configure a different Wake Word per satellite

Regardless of the configuration you choose, you must have already flashed the latest Raspbian Stretch Lite and connected to your Raspberry Pi via SSH.

If you do not know how to flash a system image on your Raspberry Pi, please check this documentation.

If you do not know how to set up the network connection on your Raspberry Pi, please read this section.

The base is supposed to be already set up with the snips platform.

If you need actions to be executed on your satellite, make sure to install snips-skill-server beforehand.

You want to use the leds feedback on your satellite ReSpeaker PiHAT2? Install the snips-skill-respeaker action code.

Audio Server Only Setup

Step 1 - Audio Server installation on the satellite

This configuration will only require snips-audio-server to be installed

If you don't know how to install snips-audio-server on your device, check the manual installation procedure and only install the snips-audio-server once snips aptitude packages repository is added to your system.

sudo apt install snips-audio-server

Check also the audio configuration procedure if you don't know how to ensure that your audio server uses the sound card on your device.

If you are using a ReSpeaker 2-Mic HAT and you have sam installed on your computer, just run sam setup audio to set up your device sound card.

Step 2 - Connect your base and satellite(s)

Let's configure both devices to make them communicate together accordingly to our use case.

This configuration takes place in /etc/snips.toml for each device.

The configuration changes will ensure that:

  • The satellite and the base are connected to the same MQTT server (hosted on the base in this tutorial).

  • Each device's audio-server have a specific siteId to differentiate them from each other.

  • The Wake Word detector hosted on the base listen to the satellite audio stream.

Satellite(s) configuration

Edit /etc/snips.toml file:

$ sudo nano /etc/snips.toml

Set the MQTT server address

Under [snips-common] section, uncomment mqtt = "localhost:1883" and change it to mqtt = "<BASE IP_ADDRESS or HOST_NAME>:1883" to indicate the MQTT server you want your satellite to be connected to.

For instance, if the hostname of your base is living-room and your base also host the MQTT server, the value should be mqtt = "living-room.local:1883".

Set the satellite siteId

Under [snips-audio-server] section, uncomment bind = "default@mqtt" and change it to bind = "<YOUR_SITE_ID>@mqtt".

Let's say your satellite is in the kitchen, you'll call it kitchen and change this value for bind = "kitchen@mqtt".

If you have multiple satellites, set a specific siteId for each device.

If you want to dive into the usage of "siteId", refer to the Hermes protocol.

After edit, our configuration looks like:

[snips-common]
mqtt = "living-room.local:1883"
[snips-audio-server]
bind = "kitchen@mqtt"

Restart the audio server to have the configuration changes taken into account.

$ sudo systemctl restart snips-audio-server

Base configuration

Edit /etc/snips.toml file:

$ sudo nano /etc/snips.toml

Set the base siteId

Under [snips-audio-server] section, uncomment bind = "default@mqtt" and change it for bind = "<YOUR_SITE_ID>@mqtt".

If your base is in the living room, you'll call it living-room and change this value for bind = "living-room@mqtt".

Make the Base Wake Word detector listen to the satellite

The Wake Word detector can be configured to listen to a list of siteIds. In our configuration we want it to listen to both the base and the satellite.

Under [snips-hotword] section, uncomment audio = [+@mqtt] and change it for audio = ["YOUR_BASE_SITE_ID@mqtt", "YOUR_SAT_SITE_ID@mqtt"].

If your satellite siteId is kitchen and your base siteId is living-room change the audio value for audio = ["living-room@mqtt", "kitchen@mqtt"].

Now our configuration looks like:

[snips-hotword]
audio = ["living-room@mqtt", "kitchen@mqtt"]
[snips-audio-server]
bind = "living-room@mqtt"

If you have more than one satellite, simply add its siteIds to the list.

Restart the audio server and Wake Word detector to have the configuration changes taken into account.

$ sudo systemctl restart snips-audio-server snips-hotword

Audio Server and Wake Word Detector Setup

Step 1 - Audio Server and Wake Word Detector Installation on the satellite

This configuration will only require snips-audio-server and snips-hotword to be installed.

This part is almost identical to the Audio Server Only setup excepted that you also need to install snips-hotword

sudo apt install snips-hotword

At this step, a Wake Word model must be installed on the satellite.

You can either,

  • Install an assistant on your satellite to use its Wake Word model (the one located in the custom_hotword subfolder)

Make sure beforehand that your assistant does not contain actions that will impede your satellite to react as you expect (or remove them afterwards with rm -rf /var/lib/snips/skills/*)

sam connect <your satellite>
sam install assistant <your assistant>

(Please, don't sam init, it will install the whole platform)

  • Or, provided that your base is already configured with an assistant, copy it to your satellite using scp.

$ ssh pi@<your-base>.local
$ scp -r /usr/share/snips/assistant pi@<your-satellite>:/usr/share/snips/assistant

Step 2 - Connect your Base and Satellite(s)

Let's configure both devices to make them communicate together accordingly to our use case.

This configuration takes place in /etc/snips.toml in each device.

The configuration changes will ensure that:

  • The satellite and the base are connected to the same MQTT server (hosted on the base in this tutorial).

  • Each device's audio-server have specific siteIds to differentiate them from each other.

  • The Wake Word detector hosted on devices only listen to their own audio servers.

Satellite(s) configuration

Edit /etc/snips.toml file:

$ sudo nano /etc/snips.toml

Set the MQTT server address

Under [snips-common] section, uncomment mqtt = "localhost:1883" and change it to mqtt = "<BASE IP_ADDRESS or HOST_NAME>:1883" to indicate the MQTT server you want your satellite to be connected to.

For instance, if the hostname of your base is living-room and your base also host the MQTT server, the value should be mqtt = "living-room.local:1883".

Set the satellite siteId

Under [snips-audio-server] section, uncomment bind = "default@mqtt" and change it to bind = "<YOUR_SITE_ID>@mqtt".

Let's say your satellite is in the kitchen, you'll call it kitchen and change this value for bind = "kitchen@mqtt".

If you have multiple satellites, set a specific siteId for each device.

If you want to dive into the usage of "siteId", refer to the Hermes protocol.

Make the Satellite Wake Word detector to listen to itself only

The Wake Word detector can be configured to listen to a list of siteIds. In our configuration we want it to listen to itself.

Under [snips-hotword] section, uncomment audio = [+@mqtt] and change it for audio = ["YOUR_SAT_SITE_ID@mqtt"].

After edit, our configuration looks like:

[snips-common]
mqtt = "living-room.local:1883"
[snips-audio-server]
bind = "kitchen@mqtt"
[snips-hotword]
audio = ["kitchen@mqtt"]

Restart the audio server and wake word detector to have the configuration changes taken into account.

$ sudo systemctl restart snips-audio-server snips-hotword

If you would like to use a personal wake word, simply modify the model address to your own.

If you want to learn how to make your own wake word, read this section.

Base configuration

Edit /etc/snips.toml file:

$ sudo nano /etc/snips.toml

Set the base siteId

Under [snips-audio-server] section, uncomment bind = "default@mqtt" and change it for bind = "<YOUR_SITE_ID>@mqtt".

If your base is in the living room, you'll call it living-room and change this value for bind = "living-room@mqtt".

Make the Base Wake Word detector to listen to itself only

If you decided to keep the hotword detector on the base you need to tell it to listen only to its own audio server.

Under [snips-hotword] section, uncomment audio = [+@mqtt] and change it for audio = ["YOUR_BASE_SITE_ID@mqtt].

Now our configuration looks like:

[snips-hotword]
audio = ["livin-groom@mqtt", "kitchen@mqtt"]
[snips-audio-server]
bind = "living-room@mqtt"

Restart the audio server and Wake Word detector to have the configuration changes taken into account.

$ sudo systemctl restart snips-audio-server snips-hotword

Test and Troubleshoot your setup

At this point, provided that an assistant is installed on your base platform, your configuration should be fully operational.

Prequisites

  • Make sure you have an assistant and action code installed on your base device and satellite if needed. For this test, we are using the makers kit demo application and related action code.

  • Start snips-watch -vvv on a terminal on your base device (sudo apt install snips-watch if it is not installed).

Test your setup

  • Speaking only to your satellite or your base, utter some sentences to your assistant.

You should observe that:

  • Your requests had been correctly answered to.

  • You can observe in the terminal where the wake word was detected and on which device sounds is played to.

In this example we are asking the satellite (siteId="kitchen") to turn the lights on. Our satellite is configured with its own Wake Word Detector. The dialogue sound feedbacks has been disabled.

From snips-watch -vvv, we obtain this output:

[13:25:13] [Hotword] detected on site kitchen, for model hey_snips
[13:25:13] [Asr] was asked to stop listening on site kitchen
[13:25:13] [Hotword] was asked to toggle itself 'off' on site kitchen
[13:25:13] [Dialogue] session with id 'c4dc2e67-33a3-407a-b577-37739100aa90' was started on site kitchen
[13:25:13] [Asr] was asked to listen on site kitchen
[13:25:16] [Asr] captured text "turn on the light" in 3.0s
[13:25:16] [Asr] was asked to stop listening on site kitchen
[13:25:16] [Nlu] was asked to parse input "turn on the light"
[13:25:16] [Nlu] detected intent relayTurnOn with confidence score 1.000 for input "turn on the light"
Slots ->
device -> light (confidence: 1.000)
[13:25:16] [Dialogue] New intent detected relayTurnOn with confidence 1.000
Slots ->
device -> light (confidence: 1.000)
[13:25:16] [Dialogue] was ask to end session with id c4dc2e67-33a3-407a-b577-37739100aa90 by saying 'Relay has been turned on.'
[13:25:16] [Tts] was asked to say "Relay has been turned on."
[13:25:16] [AudioServer] was asked to play a wav of 60.3 kB with id '8e0b1358-d288-4ddc-aa89-a8b19026b0dd' on site kitchen
[13:25:18] [AudioServer] finished playing wav with id '8e0b1358-d288-4ddc-aa89-a8b19026b0dd'
[13:25:18] [Tts] finished speaking with id 'aef035c4-487d-4c3d-8113-0cf01f5a0bab'
[13:25:18] [Dialogue] session with id 'c4dc2e67-33a3-407a-b577-37739100aa90' was ended on site kitchen. The session ended as expected
[13:25:18] [Asr] was asked to stop listening on site kitchen
[13:25:18] [Hotword] was asked to toggle itself 'on' on site kitchen

Let's explain what's going on

[13:25:13] [Hotword] detected on site kitchen, for model hey_snips
[13:25:13] [Asr] was asked to stop listening on site kitchen
[13:25:13] [Hotword] was asked to toggle itself 'off' on site kitchen
[13:25:13] [Dialogue] session with id 'c4dc2e67-33a3-407a-b577-37739100aa90' was started on site kitchen
[13:25:13] [Asr] was asked to listen on site kitchen

This whole block shows that:

  • The wake word detector correctly picked up "Hey Snips" on the site 'kitchen' (our satellite) and is then toggled off to not pick up the wake word anymore.

  • The Dialog session starts for the site kitchen

  • The ASR on the base is asked to listen to the site kitchen (Don't mind the stop listening when the hotword is detected, it is a force reset of the ASR)

[13:25:16] [Asr] captured text "turn on the light" in 3.0s
[13:25:16] [Asr] was asked to stop listening on site kitchen
[13:25:16] [Nlu] was asked to parse input "turn on the light"
[13:25:16] [Nlu] detected intent relayTurnOn with confidence score 1.000 for input "turn on the light"
Slots ->
device -> light (confidence: 1.000)
[13:25:16] [Dialogue] New intent detected relayTurnOn with confidence 1.000
Slots ->
device -> light (confidence: 1.000)

Snips magic happens,

  • The ASR captured the text correctly.

  • The NLU could detect the intent

  • The Dialogue advertised that a new intent is detected

At this point, the intent must be answered to. If not, the session will be terminated by the dialogue service and this log will be displayed in the logs: The session was ended because one of the component didn't respond in a timely manner

[13:25:16] [Dialogue] was ask to end session with id c4dc2e67-33a3-407a-b577-37739100aa90 by saying 'Relay has been turned on.'
[13:25:16] [Tts] was asked to say "Relay has been turned on."
[13:25:16] [AudioServer] was asked to play a wav of 60.3 kB with id '8e0b1358-d288-4ddc-aa89-a8b19026b0dd' on site kitchen
[13:25:18] [AudioServer] finished playing wav with id '8e0b1358-d288-4ddc-aa89-a8b19026b0dd'
[13:25:18] [Tts] finished speaking with id 'aef035c4-487d-4c3d-8113-0cf01f5a0bab'

The action code reacted to the intent relayTurnOn and answered by asking the dialogue service to:

  • End the ongoing dialogue session

  • Play a TTS message on the siteID kitchen

[13:25:18] [Dialogue] session with id 'c4dc2e67-33a3-407a-b577-37739100aa90' was ended on site kitchen. The session ended as expected
[13:25:18] [Asr] was asked to stop listening on site kitchen
[13:25:18] [Hotword] was asked to toggle itself 'on' on site kitchen

The dialog session is closing, the platform is ready to react to a new request.

Test with mosquitto clients

Provided mosquitto clients are installed on your device (sudo apt install mosquitto-clients)

You can test your setup by trying to utter a TTS message on your base and satellite using mosquitto_pub.

Given:

  • Your satellite and base are connected to the same MQTT server (e.g. snips-base.local)

  • The satellite audio server is bind to the site Id kitchen

  • The base audio server is bind to the site Id living-room

When executing the following commands on the base device:

mosquitto_pub -t 'hermes/tts/say' -m '{"siteId": "living-room", "text": "Hello from the base!", "lang": "en-EN", "sessionId": "dummyid"}

The base says Hello from the base!

mosquitto_pub -t 'hermes/tts/say' -m '{"siteId": "kitchen", "text": "Hello from the satellite!", "lang": "en-EN", "sessionId": "dummyid"}

The satellite says Hello from the satellite!

{ hint style="warning" } If you don't obtain the expected results, check thoroughly your configuration files as it's really easy to make a mistake when updating /etc/snips.toml files. { endhint }