Satellite Configuration

Forewords

This tutorial explains how you can use satellites (multi-room dialog) with your base device. For instance, if you want your base device in the living room and have a satellite in your kitchen and/or one in your bedroom and so on.

  • Base: The main Snips device hosting the intents detection engine and the actions to be triggered.

  • Satellite: The vocal interface located in an other room, running snips-satellite service.

Audio on demand

snips-satellite reunites snips-audio-server and snips-hotword in one single service to ease the satellite configuration.

Once a Wake Word is detected on the satellite, the ASR service located on the base requests an audio stream from the satellite, thus avoiding unnecessary data transmissions over the local network.

Hardware requirements

Satellite main feature is to act as a vocal interface and Wake Word detector, optionally running actions scripts to accomplish some actions locally.

Any devices such as the Pi Zero at least able to run snips-satellite and snips-skill-server can be used as a satellite.

This tutorial is based on the Raspberry Pi Zero configured as a satellite and a Raspberry Pi 3 configured as a base.

Those two boards are using a ReSpeaker 2-Mic HAT to capture and emit sound.

Configuration Steps

You must have already flashed the latest Raspbian Stretch Lite and connected to your Raspberry Pi via SSH.

If you do not know how to flash a system image on your Raspberry Pi, please check this documentation.

If you do not know how to set up the network connection on your Raspberry Pi, please read this section.

The base is supposed to be already set up with the snips platform.

If you need actions to be executed on your satellite, make sure to install snips-skill-server beforehand.

You want to use the leds feedback on your satellite ReSpeaker PiHAT2? Install the snips-skill-respeaker action code.

Step 1 - Installation of the Snips satellite package

This configuration will only require snips-satellite to be installed

If you don't know how to install snips-satllite on your device, check the manual installation procedure and only install the snips-satellite once snips aptitude packages repository is added to your system.

sudo apt install snips-hotword-model-heysnipsv4
sudo apt install snips-satellite

Check also the audio configuration procedure if you don't know how to ensure that your audio server uses the sound card on your device.

If you are using a ReSpeaker 2-Mic HAT and you have sam installed on your computer, just run sam setup audio to set up your device sound card.

At this step, a default Hey Snips Wake Word model is installed on the satellite.

If you are using a different Wake Word, you must provide it to your satellite.

The simplest way to proceed is to copy the assistant located on your base to your satellite.

ssh pi@<your-base-hostname>.local
scp -r /usr/share/snips/assistant pi@<your-satellite-hostname>:/usr/share/snips/assistant

Step 2 - Connect your Base and Satellite(s)

Let's configure both devices to make them communicate together accordingly to our use case.

This configuration takes place in /etc/snips.toml in each device.

The configuration changes will ensure that:

  • The satellite and the base are connected to the same MQTT server (hosted on the base in this tutorial).

  • Each device's audio-server have specific siteIds to differentiate them from each other.

  • The Wake Word detector hosted on the base device only listen to its own audio server.

Satellite(s) configuration

Edit /etc/snips.toml file:

sudo nano /etc/snips.toml

Set the MQTT server address

Under [snips-common] section, uncomment mqtt = "localhost:1883" and change it to mqtt = "<your-base-hostname>:1883" to indicate the MQTT server you want your satellite to be connected to.

For instance, if the hostname of your base is living-room and your base also host the MQTT server, the value should be mqtt = "living-room.local:1883".

Set the satellite siteId

Under [snips-audio-server] section, uncomment bind = "default@mqtt" and change it to bind = "<YOUR_SITE_ID>@mqtt".

Let's say your satellite is in the kitchen, you'll call it kitchen and change this value for bind = "kitchen@mqtt".

If you have multiple satellites, set a specific siteId for each device.

If you want to dive into the usage of "siteId", refer to the Hermes protocol documentation.

After edit, our configuration looks like:

[snips-common]
mqtt = "living-room.local:1883"
[snips-audio-server]
bind = "kitchen@mqtt"

Restart the audio server and wake word detector to have the configuration changes taken into account.

$ sudo systemctl restart snips-satellite

If you would like to use a personal wake word, simply modify the model address to your own.

If you want to learn how to make your own wake word, read this section.

Base configuration

Edit /etc/snips.toml file:

sudo nano /etc/snips.toml

Set the base siteId

Under [snips-audio-server] section, uncomment bind = "default@mqtt" and change it for bind = "<YOUR_SITE_ID>@mqtt".

If your base is in the living room, you'll call it living-room and change this value for bind = "living-room@mqtt".

Make the Base Wake Word detector to listen to itself only

If you decided to keep the hotword detector on the base you need to tell it to listen only to its own audio server.

Under [snips-hotword] section, uncomment audio = [+@mqtt] and change it for audio = ["YOUR_BASE_SITE_ID@mqtt].

Now our configuration looks like:

[snips-hotword]
audio = ["living-room@mqtt"]
[snips-audio-server]
bind = "living-room@mqtt"

Restart the audio server and Wake Word detector to have the configuration changes taken into account.

sudo systemctl restart snips-audio-server snips-hotword

If for some reason this configuration does not suit your needs.

The legacy satellite configuration documentation can be found here.

Enable site groups

A common issue with satellites is to have multiple sites answering to your vocal commands at the same time.

For instance, this issue occurs if your satellite and your base are close enough to you to capture both the wakeword you uttered.

A solution to this problem is to use site groups allowing you to group your satellite and base vocal interfaces to prevent multiple dialogue sessions to be triggered.

This feature can be configured in the [snips-dialogue] section of the configuration file using the following parameters.

  • sites_in_same_group_default: All sites belong to the same group if set to true (default value is false)

  • site_groups: Specify manually the groups you want your satellites to belong to.

The dialogue component will then ignore wake words happening in a group if one already happened shortly before.

By default all sites are in different groups. You can change this behaviour in the snips.toml configuration file if you want to have all your vocal interfaces grouped together.

Most users will be satisfied by grouping all their sites within the same group. This configuration is achieved by setting sites_in_same_group_default to true.

[snips-dialogue]
sites_in_same_group_default = true

A more granular control can be achieved by specifying the groups manually.

e.g. Let say a house is equipped with four vocal interfaces, two in the living room and two in the kitchen (living-room-1, living-room-2, kitchen-1, kitchen-2).

We want to group them by room to avoid multiple dialog sessions to be started at the same time when asking something to the snips platform.

[snips-dialogue]
site_groups = ["living-room-1,living-room-2", "kitchen-1,kitchen-2"]

Site ids not listed in the site_groups configuration are grouped as specified by the sites_in_same_group_default policy.

  • If true all remaining site ids are grouped together.

  • If false all remaining site ids are assigned to a dedicated group by site id).

This feature does not guarantee that the closest interface will be used during the dialog session.

If two interfaces are grouped in a same room, it is possible that the most remote voice interface will detect the wakeword first and be used during the dialog session.

Test and Troubleshoot your setup

At this point, provided that an assistant is installed on your base platform, your configuration should be fully operational.

Prequisites

  • Make sure you have an assistant and action code installed on your base device and satellite if needed. For this test, we are using the makers kit demo application and related action code.

  • Start snips-watch -vvv on a terminal on your base device (sudo apt install snips-watch if it is not installed).

Test your setup

  • Speaking only to your satellite or your base, utter some sentences to your assistant.

You should observe that:

  • Your requests had been correctly answered to.

  • You can observe in the terminal where the wake word was detected and on which device sounds is played to.

In this example we are asking the satellite (siteId="kitchen") to turn the lights on. Our satellite is configured with its own Wake Word Detector. The dialogue sound feedbacks has been disabled.

From snips-watch -vvv, we obtain this output:

[13:25:13] [Hotword] detected on site kitchen, for model hey_snips
[13:25:13] [Asr] was asked to stop listening on site kitchen
[13:25:13] [Hotword] was asked to toggle itself 'off' on site kitchen
[13:25:13] [Dialogue] session with id 'c4dc2e67-33a3-407a-b577-37739100aa90' was started on site kitchen
[13:25:13] [Asr] was asked to listen on site kitchen
[13:25:16] [Asr] captured text "turn on the light" in 3.0s
[13:25:16] [Asr] was asked to stop listening on site kitchen
[13:25:16] [Nlu] was asked to parse input "turn on the light"
[13:25:16] [Nlu] detected intent relayTurnOn with confidence score 1.000 for input "turn on the light"
Slots ->
device -> light (confidence: 1.000)
[13:25:16] [Dialogue] New intent detected relayTurnOn with confidence 1.000
Slots ->
device -> light (confidence: 1.000)
[13:25:16] [Dialogue] was ask to end session with id c4dc2e67-33a3-407a-b577-37739100aa90 by saying 'Relay has been turned on.'
[13:25:16] [Tts] was asked to say "Relay has been turned on."
[13:25:16] [AudioServer] was asked to play a wav of 60.3 kB with id '8e0b1358-d288-4ddc-aa89-a8b19026b0dd' on site kitchen
[13:25:18] [AudioServer] finished playing wav with id '8e0b1358-d288-4ddc-aa89-a8b19026b0dd'
[13:25:18] [Tts] finished speaking with id 'aef035c4-487d-4c3d-8113-0cf01f5a0bab'
[13:25:18] [Dialogue] session with id 'c4dc2e67-33a3-407a-b577-37739100aa90' was ended on site kitchen. The session ended as expected
[13:25:18] [Asr] was asked to stop listening on site kitchen
[13:25:18] [Hotword] was asked to toggle itself 'on' on site kitchen

Let's explain what's going on

[13:25:13] [Hotword] detected on site kitchen, for model hey_snips
[13:25:13] [Asr] was asked to stop listening on site kitchen
[13:25:13] [Hotword] was asked to toggle itself 'off' on site kitchen
[13:25:13] [Dialogue] session with id 'c4dc2e67-33a3-407a-b577-37739100aa90' was started on site kitchen
[13:25:13] [Asr] was asked to listen on site kitchen

This whole block shows that:

  • The wake word detector correctly picked up "Hey Snips" on the site 'kitchen' (our satellite) and is then toggled off to not pick up the wake word anymore.

  • The Dialog session starts for the site kitchen

  • The ASR on the base is asked to listen to the site kitchen (Don't mind the stop listening when the hotword is detected, it is a force reset of the ASR)

[13:25:16] [Asr] captured text "turn on the light" in 3.0s
[13:25:16] [Asr] was asked to stop listening on site kitchen
[13:25:16] [Nlu] was asked to parse input "turn on the light"
[13:25:16] [Nlu] detected intent relayTurnOn with confidence score 1.000 for input "turn on the light"
Slots ->
device -> light (confidence: 1.000)
[13:25:16] [Dialogue] New intent detected relayTurnOn with confidence 1.000
Slots ->
device -> light (confidence: 1.000)

Snips magic happens,

  • The ASR captured the text correctly.

  • The NLU could detect the intent

  • The Dialogue advertised that a new intent is detected

At this point, the intent must be answered to. If not, the session will be terminated by the dialogue service and this log will be displayed in the logs: The session was ended because one of the component didn't respond in a timely manner

[13:25:16] [Dialogue] was ask to end session with id c4dc2e67-33a3-407a-b577-37739100aa90 by saying 'Relay has been turned on.'
[13:25:16] [Tts] was asked to say "Relay has been turned on."
[13:25:16] [AudioServer] was asked to play a wav of 60.3 kB with id '8e0b1358-d288-4ddc-aa89-a8b19026b0dd' on site kitchen
[13:25:18] [AudioServer] finished playing wav with id '8e0b1358-d288-4ddc-aa89-a8b19026b0dd'
[13:25:18] [Tts] finished speaking with id 'aef035c4-487d-4c3d-8113-0cf01f5a0bab'

The action code reacted to the intent relayTurnOn and answered by asking the dialogue service to:

  • End the ongoing dialogue session

  • Play a TTS message on the siteID kitchen

[13:25:18] [Dialogue] session with id 'c4dc2e67-33a3-407a-b577-37739100aa90' was ended on site kitchen. The session ended as expected
[13:25:18] [Asr] was asked to stop listening on site kitchen
[13:25:18] [Hotword] was asked to toggle itself 'on' on site kitchen

The dialog session is closing, the platform is ready to react to a new request.

Test with mosquitto clients

Provided mosquitto clients are installed on your device (sudo apt install mosquitto-clients)

You can test your setup by trying to utter a TTS message on your base and satellite using mosquitto_pub.

Given:

  • Your satellite and base are connected to the same MQTT server (e.g. snips-base.local)

  • The satellite audio server is bind to the site Id kitchen

  • The base audio server is bind to the site Id living-room

When executing the following commands on the base device:

mosquitto_pub -t 'hermes/tts/say' -m '{"siteId": "living-room", "text": "Hello from the base!", "lang": "en-EN", "sessionId": "dummyid"}'

The base says Hello from the base!

mosquitto_pub -t 'hermes/tts/say' -m '{"siteId": "kitchen", "text": "Hello from the satellite!", "lang": "en-EN", "sessionId": "dummyid"}'

The satellite says Hello from the satellite!

If you don't obtain the expected results, check thoroughly your configuration files as it's really easy to make a mistake when updating /etc/snips.toml files.