Quick Start nodeJS / Electron

Megazord is a very good fit to embed an assistant in any app: a game or an email app written in Electron, or even a console app written in NodeJS.
Because the Megazord library can run the entire platform, it means that the users of your app do not need to have the Snips platform preinstalled on their device to use your app. It also means that multiple apps running different assistants can all work at the same time.
This tutorial is going to show you how to use Megazord from the NodeJS bindings, in a simple console app, and in a graphical Electron app.
Prerequisites
The prerequisites for this project are:
​macOS El Capitan or later, or a Debian Linux distribution
​Node.js and NPM or yarn​
​Homebrew on macOS
​Electron​
Step 1: Install the Snips Megazord library
The Snips Megazord library embeds the whole Snips platform as a dynamic library which can be shipped with any app
First, add the Snips tap to Homebrew. In a terminal window, enter the following command:
1
brew tap snipsco/homebrew-snips
Copied!
Then, install the Snips Megazord library:
1
brew install libsnips_megazord
Copied!
Step 2: NodeJS assistant
You can get the code for this step at quickstart/vanilla​
Installing dependencies
The first step is to install the Megazord bindings:
1
yarn add megazord-javascript
Copied!
We need to stream audio to the Megazord library. When running Electron, we can use the browser microphone, but when writing a console app, we need to use a library. A simple way to do this from node is to use the mic NPM package which uses arecord or sox to stream the audio to node:
1
yarn add mic
Copied!
Using Megazord-javascript
The Megazord bindings provide a low-level API to the Megazord library, and a high-level SnipsPlatform wrapper to create assistants easily. This is what we will use:
main.js
1
const {SnipsPlatform} = require('megazord-javascript');
2
const mic = require('mic');
3
​
4
const assistant = new SnipsPlatform('assistant', {
5
    enableSnipsWatch: true,
6
    enableLogs: false,
7
});
8
​
9
var _currentSessionId = null;
10
assistant
11
    .onIntentDetected((msg, user_data) => {
12
        console.log('-- intent detected: ' + msg.intent.intent_name);
13
        console.log(' - input: ' + msg.input);
14
        _currentSessionId && assistant.endSession(_currentSessionId);
15
    })
16
    .onIntentNotRecognized((msg, user_data) => {
17
        console.log('-- intent not recognized: ' + msg.input);
18
        _currentSessionId && assistant.endSession(_currentSessionId);
19
    })
20
    .onSessionStarted((msg, user_data) => {
21
        _currentSessionId = msg.session_id;
22
    })
23
    .onSessionEnded((msg, user_data) => {
24
        _currentSessionId = null;
25
    })
26
    .onListeningStateChanged((isListening, user_data) => {
27
        console.log('-- is_listening: ' + isListening);
28
    })
29
    .onSnipsWatch((msg, user_data) => {
30
        console.log('-- snips_watch', msg);
31
    });
32
​
33
assistant.start();
34
​
35
// stream audio
36
​
37
var micInstance = mic({rate: '16000', channels: '1'});
38
var micInputStream = micInstance.getAudioStream();
39
​
40
micInputStream.on('data', function(data) {
41
    const buf = Buffer.from(data);
42
    assistant.appendBuffer(buf);
43
});
44
​
45
micInputStream.on('error', function(err) {
46
    console.error("Error: " + err);
47
});
48
​
49
micInstance.start();
Copied!
Running the code
That's it! A few tens of lines of code to add a fully-functional assistant to any node app! You can unzip an assistant in the current  path and run:
1
node main.js
Copied!
You might need to symlink the libsnips_megazord library in the current path or to add it to your LD_LIBRARY_PATH or DYLD_LIBRARY_PATH to make sure that it will be found.
On osX, the library can be found in /usr/local/Cellar/libsnips_megazord
The code from quickstart/vanilla allows recording the streamed audio as .wav files which is useful to ensure that the audio streaming works correctly. You need to set recordQueryAudio to true in main.js.
Step 3: An Electron app
You can get the code for this step at quickstart/electron​
The Megazord javascript bindings make it really easy to add an assistant to any desktop app you might want to write: games, email clients, or even your own version of Siri!
Installing dependencies with the correct Node version
The version of the FFI lib used by the Megazord bindings and by Electron must be compatible. If there are errors mentioning the Node version, you might want to use electron-rebuild, or run the following commands (see this page) each time you install a new native package.
If there are still errors, you might want to remove ~/.node-gyp and node_modules, and build again
1
# remove previous `node_modules` if any
2
rm -rf node_modules
3
​
4
# using electron-build
5
yarn add --dev electron-rebuild
6
./node_modules/.bin/electron-rebuild
7
​
8
# or building manually
9
HOME=~/.electron-gyp npm install \
10
    --runtime=electron \
11
    --target=4.2.9 \
12
    --disturl=https://electronjs.org/headers \
13
    --build-from-source \
14
    --arch=x64 \
15
    --target-arch=x64
Copied!
If there are still errors, you might want to remove ~/.node-gyp and the node_modules, and build again
Using Megazord-javascript from an Electron app
We can use Electron to get the microphone input and send it to the assistant.
Running the code
You might need to symlink the libsnips_megazord library in the current path or to add it to your LD_LIBRARY_PATH or DYLD_LIBRARY_PATH to make sure that it will be found.
On osX, the library can be found in /usr/local/Cellar/libsnips_megazord
The code is very similar to a regular NodeJS assistant, but we will be used a Recorder class to get the microphone input from the Electron browser. We will also add a way for the user to start using the assistant by either saying the wakeword or pressing a button.
The code at quickstart-megazord-javascript/electron is using React, but you can use any framework:
1
    componentDidMount() {
2
        createAudioRecorder(
3
            (data) => {
4
                this.appendBuffer(Buffer.from(data))
5
            },
6
            (recorder) => { recorder.start() }
7
        );
8
    }
Copied!
You can then run yarn start to start the electron app and a small webserver which will serve the React code.
Step 4: Using entity injection
One of the powerful features of Snips is the ability to add custom entities to your assistant in real-time. A user of your app could add contact names, music artists, songs or any other entity, and then start using them!
You can look at the code at quickstart/injection​
The idea of injection is to add new values to an entity at runtime, either from "vanilla" (as though the assistant you are adding values to was freshly downloaded from the console), or incrementally, which keeps previous entities which you might have added before.
To use the entity injection, you need to install lexical resources
brew install snips-injection
To allow the assistant to use injection, you should use enableInjection, and provide two paths: the path g2pResourcesPath to the lexical resources (on osX, it will be /usr/local/Cellar/snips-injection/0.63.3/share/snips/g2p-models), and a path userPath where the injection data will be cached
1
const {SnipsPlatform, SnipsConstants} = require('megazord-javascript');
2
​
3
const assistant = new SnipsPlatform('assistant', {
4
    enableInjection: true,
5
    userPath: 'user_injection',
6
    g2pResourcesPath: 'snips-g2p-resources',
7
    enableLogs: true,
8
});
9
​
10
assistant.start();
11
​
12
// entity injection
13
const lexicon = {}; // word: [pronunciations]
14
const operations = [{
15
    kind: SnipsConstants.SnipsInjectionKindEnum.ADD_FROM_VANILLA,
16
    values: {
17
        'contactName': [
18
            'Mom',
19
            'Steve',
20
            'Elon',
21
            'Bill'
22
        ]
23
    }
24
}];
25
​
26
assistant.requestInjection(operations, lexicon);
Copied!
When running the injection request, the platform will update the ASR in the background and replace it when it is complete. Depending on the size of the assistant and the number of entities, it can take a few milliseconds to a few seconds. During this time, it is still possible to use the platform.
If the platform is stopping after doing the injection, you might want to use enableLogs to see where the error is coming from.  This could happen if you chose a slot name which does not exist in the assistant.
​
Quick Start Jetson TX2
Articles
Last modified 2yr ago
Copy link
Contents
Prerequisites
Step 1: Install the Snips Megazord library
Step 2: NodeJS assistant
Installing dependencies
Using Megazord-javascript
Running the code
Step 3: An Electron app
Installing dependencies with the correct Node version
Using Megazord-javascript from an Electron app
Running the code
Step 4: Using entity injection