Karaoke App

Last updated 2 months ago

Introduction

The Snips Karaoke App for Raspberry Pi turns your device into a karaoke station controlled by voice. It plays locally stored videos and updates its speech recognition by itself, in order to understand their song titles and artist names.

Usage

Instructions should start with "Hey, Snips!", a feedback sound will be played at recognition. Songs and artists can be played by asking the assistant to do so, in English, from the main menu or while another song is playing. The playback and the volume can also be controlled by voice, only during lecture. At any moment, the Karaoke can play the previous and next song in alphabetical order, or redirect the user to the main menu / songs list.

Karaoke App - User Journey

Requirements

The following packages are necessary for your Karaoke App to be functional:

  • NodeJs

  • Fim

  • OMXPlayer

sudo apt-get install nodejs
sudo apt-get install fim
sudo apt-get install omxplayer

The _snips-skills user should have the access to the video users group:

sudo usermod -a -G video _snips-skills

The Snips Injection module is needed aside of the standard Snips platform:

sudo apt-get install -y snips-injection

Video files

The following is required from the video files in order to insure that the app is fully functional:

  • File names should respect the format <name of the artist> — <name of the song>.mp4, in order to be displayed correctly.

  • The cumulated size of the video files should not exceed the total memory size of your SD card, minus 2 Gb.

  • Not more than 18 videos can appear in the songs list, the remaining ones are still reachable through voice instructions.

Video files should be located on the Raspberry Pi in the directory /var/lib/snips/skills/karaoke/videos.

In order to transfer a video file through the Scp program, the command to launch is:

scp <local path to your file> <username>@<hostname>.local:/var/lib/snips/skills/karaoke/videos

Where <hostname> is the Raspberry Pi’s hostname, and <username> is the Pi's default user. <hostname>.local can be replaced by the device's IP address.The user's password will be be asked for.

Slot values injection

The Karaoke assistant needs to be able to identify any added song / artist name as new slot values. It can adapt its model and train locally its ASR and NLU, thanks to the Snips Injection program. The user needs to manually install Snips Injection beforehand as it is not included by default in the Snips platform. It can be done with:

sudo apt-get install -y snips-injection

This program runs along the rest of the platform and waits for an MQTT message on the topic hermes/injection/perform. As explained in its documentation, Snips Injection expects a payload in this format:

{
"operations": [
[
"add",
{
"Song_name" : [
"Billie Jean"
],
"Artist_name": [
"Michael Jackson"
]
}
]
}

The local modifications won’t be applied to the remote assistant from the Snips console. Therefore, they won't be found back when the assistant is re-installed on the same device, or newly installed on another device. To illustrate how it is used, here is the NodeJs code from the Karaoke App that deals with slot value injection:

Finder.prototype.injectTitles = function () {
let videos = this.videos,
payload = { "operations": [["add"]] },
addition = { song_name: [], artist_name: [] };
videos.forEach((element) => {
addition.song_name.push(element.title);
addition.artist_name.push(element.artist);
});
payload.operations[0].push(addition);
payload = JSON.stringify(payload);
console.log(payload);
this.client.publish('hermes/injection/perform', payload);
};

Troubleshooting

  • You don't have anything that appears when booting your Raspberry Pi. If you've transferred the videos in the right path, make sure that you gave to the _snips-skills user an access to the video group:

sudo usermod -a -G video _snips-skills
  • Your menu appears with your song titles but you can't read any videos. Make sure that you've updated your Snips platform to its last release. From your computer, run sam update. If this doesn't solve your problem after rebooting your Pi, access to your Pi's command line and run this command to get the logs of your app:

journalctl -u snips-skill-server -f