Voice Interaction Development Kit

The place to start making technology disappear ⛳️

Introduction

This guide will get you started building with the Voice Interaction Development Kit from 3 entry points. If you are an end-user of our technology, you may focus on the General Software Setup and Satellite Setup section only. If you’d like to join the Snips community as a developer, we suggest carefully going through all the mentioned references within the Develop your Snips App section.

Voice Interaction Base Kit and Satellite Kit

🛒 ➜ Buy the Maker Kit

The Voice Interaction Base Kit and Voice Interaction Satellite Kit both come with included Raspberry Pis, all the necessary software, and the latest Raspbian installed. That means that you can start to interact with your kit once it has been plugged into a power source.

The Voice Interaction Satellite Kit comes with **a Raspberry Pi Zero W** but still requires a Raspberry Pi 3B+/Raspberry Pi 3B, since it needs to work with a base device, and requires LAN (local) WiFi connection.

Start Building

Components List
Final Apparence
Step 1
Step 2
Step 3
Step 4

We've also made a quick video to help you here.

Using The Demo Assistant

The default assistant is in English with two demo apps pre-installed: indoor environment and relay switch. The indoor environment app shares the air temperature and humidity in your current environment, based on the Grove SHT31 module. The relay switch app lets you control anything that is connected to the Grove relay module.

To query the indoor environment:

"Hey Snips, what's the temperature in the room?"

Or

"Hey Snips, what's the humidity in the room?"

The temperature unit is configurable between Celsius and Fahrenheit when installed using sam.

To control the relay switch:

"Hey Snips, please switch on the relay!"

Or

"Hey Snips, please turn off the tv"

The relay intents are designed with two slots, device and house_room. This demo action code will use none of them, it will only control the relay connected to its own on/off.

General Software Setup

This section addresses the key considerations when developing with the Snips platform. Of course, the kit can be assembled to work without any software setup.

Default Parameters List

Parameters

Voice Interaction Base Kit

Voice Interaction Satellite Kit

hostname

snips-base

snips-sat

user

pi

pi

password

raspberry

raspberry

siteId

default

default-sat

To change parameters for the Raspberry Pi , run (On Raspberry):

sudo raspi-config

To change the Snips siteId from the file, run (On Raspberry Pi):

sudo nano /etc/snips.toml

Connect Raspberry Pi via WiFi Network

To move forward, we will need to log in to the Raspberry Pi system. This can be achieved using an external HDMI screen and keyboard. However, we would recommend using ssh commands through the network, which doesn’t require anything other than your personal laptop.

The ssh service is enabled by default. So the only requirement is to give a correct wifi ssid and password to the Raspberry Pi. To achieve this, power off the Raspberry Pi and then remove the SD card. Connect this card to your personal laptop by using an embedded/ external SD card reader.

Once it's plugged in, you should be able to see a boot partition (whether using Linux or Windows). Simply create a file and name it with wpa_supplicant.conf. The following content should be filled into this file.

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=<your_country_id>
network={
ssid="<your_ssid>"
psk="<your_password>"
}

You need to change <your_country_id> to your country code (list), <your_ssid> to your wifi SSID, and <your_password> to your wifi password.

Beware: When saving this file, some of the text editors will automatically add .txt as the extension name, which is not accepted. Please manually rename it to wpa_supplicant.conf if this is the case.

Once completed, re-insert the SD card into the Raspberry Pi. Plug it in and wait about 1 minute, then you should be able to log in to this Raspberry Pi by its hostname through ssh. Depending on your kit configuration, this hostname can be snips-base.local or snips-sat.local.

Using Sam

Sam (Snips Assistant Manager) is a Command Line Interface (CLI) tool we created in order to easily setup and maintain Snips software, your assistant, and your action code on Raspberry Pi and similar devices right from your computer, without having to create an SSH session and typing complex, low-level commands.

We will only partly cover sam, but there’s a specific reference written for sam here.

Install Sam

Open a terminal window, and enter:

sudo npm install -g snips-sam

Connect Sam with Raspberry Pi

Run the following command, sam will list all the available devices found in the local network:

sam devices

The result should look like this:

Scanning Raspberry Pi devices on the network...
Found 2 device:
- snips-base (192.168.x.x)
- snips-sat (192.168.x.x)
...

Once you know the hostname or ip address of the Raspberry Pi, run:

sam connect <hostname/ip_address>

Checking Snips Status

Run the following command to check the current status of Snips components:

sam status

The result should report basic information about the device and running status of Snips software.

Update Snips Platform

Run the following command to update the Snips platform on your device:

sam update

Update Default Assistant

Since the 1.1.0 version has some breaking changes, this section will direct you to re-install the default assistant and its action code.

Use the username pi and password raspberry, login to Raspberry Pi via ssh:

ssh pi@snips-base.local

Then remove all the old Snips action code completely:

sudo rm -r /var/lib/snips/skills/*

From here, the old action code copies are all removed. We can logout Raspberry Pi:

exit

Next, please refer to these steps (1 to 5) , install the latest demo assistant and its action code.

Finally, we can re-install the animation skill for ReSpeaker hat:

sam install actions -g https://github.com/snipsco/snips-skill-respeaker.git

Install Another Assistant

To try more assistants, you will need to have a Snips account for the Snips Console where you can customize or create your own assistant by adding different Snips apps. The design of the Snips console is very straight forward, which means that you should be able to get up and running in no time. However, you may want to follow some instructions if you get confused with a concept from time to time. For this case, we do have a user guide for the Snips Console, accessible here.

For the following steps, we assume that you have already played with the Console and built your own assistant.

To deploy an assistant, run the following command:

sam install assistant

Sam will list all the available assistants under your account. Choose the one you wish to try and press [Enter]. Once the target assistant is successfully installed, you may be asked to fill in some config parameters, like the API key for a connected application. If you don't know the parameters, you can just leave this empty. This value can be changed in the config.ini file in the app folder: `/var/lib/snips/skills/.

Watching Snips log in real-time

Run the following command:

sam watch

This will give you a real-time feedback of information parsing.

Personal Wakeword Configuration

Our full step-by-step guide can be found here.

Satellite Setup

Since different Snips devices can be connected to the same MQTT bus but serve different roles, this section will walk you through how to set up multiple satellites. To set up a single satellite, refer to the detailed guide here.

Setting Unique siteId

The Satellite Kit comes with a default siteId, which is called default-sat. Normally if the hostname of the base device is not changed (by default it is snips-base), the satellite should work once connected to the network. However, this also means that if there are several Satellite Kits connected to the same Base Kit, they all have the same siteId.

To change this, log into the base device through ssh and then enter:

sudo nano /etc/snips.toml

Under the [snips-audio-server] section, change bind = "default-sat@mqtt" to bind = "<You_defiend_site_id>@mqtt" so that you can give a new siteId referring to the location of this device. For example: living_room, bedroom etc.

Also under the [snips-hotword] section, change audio = ["<You_defiend_site_id>@mqtt"] where you should enter the siteId you've just defined.

Beware: if you change the hostname of the base device, you also need to change the [snips-common] section. Change mqtt = "snips-base.local:1883" to mqtt = "<new_hostnmae>.local:1883".

Running Action Code on a Satellite

We can use sam to install an action onto a Snips device, as previously mentioned. But the fact is that most of the apps are configured to run on a base device by default.

In order to install apps on the satellite, fill the correct config option when asked by sam. For example, a typical difference is that on the base device, we usually use localhost as the MQTT hostname. Change this value to the hostname of the base device that can be identified in the local network for the satellite.

Develop your own Snips App

If you are more than just curious about how to use Snips as a new technology at home, and you have a passion for building something using this technology, Snips wants you!

Though developing for voice apps is a new experience compared to developing for traditional mobile/desktop apps, if you have the basics of programming down and know at least one of the most popular programming languages (Python, JavaScript, etc), voice apps will be a fun new challenge for you!

Introducing the Snips App

A standard Snips app should be composed of two parts: the voice bundle and the action code. Voice bundles can be produced and tested in the Snips Console, and are used to analyze the captured voice content, outputting a structured dataset. The action code is actually a program script which can be written in different programming languages, such as Python, JavaScript, or C, and is responsible for executing different actions depending on the result dataset (e.g. turning on a light).

Where to Start

Additional Resources

For a more detailed description about the demo assistant action code, please consult this repository.

For more detailed documentation/information, please reach out our on Developer Center.

For a discussion about your idea/suggestion, please reach out our on forum.