Prerequisites

The prerequisites for this project are:

  • Android Studio version 3.0 or higher

  • JDK version 7.0 or higher

  • Android API Level 21 or higher (Android 5.0 and above)

The code for this guide can be found on Github: snipsco-samples/quickstart-android.

Step 1: Create a new Android Studio project

Open Android Studio, create a new Android project. Let's name it "HelloSnips". Save the project in a location of your choice. Select "Phone and Tablet" as target devices, and choose and API level of at least 21. Select "Basic Activity" for the activity type, and name it "Main Activity". Enter "Hello Snips" for the activity title.

Step 2: Add the Snips library

In the project navigator, under Gradle Scripts, locate and open the module-level build.gradle (Module: app) file. Do the following:

  • Add the Snips Nexus repository to the list of repositories:

repositories {
maven {
url "https://nexus-repository.snips.ai/repository/snips-maven-releases/"
}
}
  • Add the Snips library to the list of dependencies:

dependencies {
// ...
implementation('ai.snips:snips-platform-android:0.60.2@aar') {
transitive = true
}
}

Step 3: Import the assistant model from the Console

If you haven't done so already, head over to console.snips.ai and create your assistant. If this is the first time you are creating an assistant, make sure to follow the Quick Start Console guide:

Once ready, download the assistant. Create a folder named assets at the same location as your AndroidManifest.xml file (usually <PROJECT_ROOT>/app/src/main), and copy the assistant.zip file to that direction. To be clear, the assistant.zip should now be located at:

<PROJECT_ROOT>/app/src/main/assets/assistant.zip

Step 4: Grant microphone access

We need to grant the app microphone access. From the project navigator, open AndroidManifest.xml, and add the following line within the <manifest> entry:

<uses-permission android:name="android.permission.RECORD_AUDIO"/>

Step 5: Setup MainActivity

From the project navigator, open MainActivity.java.

Extracting the assistant

We start by unzipping the assistant from the assets folder to a folder which is accessible by the Snips Platform. We do that using the following two helper method, that we place within the MainActivity class:

private void extractAssistantIfNeeded(File assistantLocation) {
File versionFile = new File(assistantLocation,
"android_version_" + BuildConfig.VERSION_NAME);
if (versionFile.exists()) {
return;
}
try {
assistantLocation.delete();
MainActivity.unzip(getBaseContext().getAssets().open("assistant.zip"),
assistantLocation);
versionFile.createNewFile();
} catch (IOException e) {
return;
}
}
private static void unzip(InputStream zipFile, File targetDirectory)
throws IOException {
ZipInputStream zis = new ZipInputStream(new BufferedInputStream(zipFile));
try {
ZipEntry ze;
int count;
byte[] buffer = new byte[8192];
while ((ze = zis.getNextEntry()) != null) {
File file = new File(targetDirectory, ze.getName());
File dir = ze.isDirectory() ? file : file.getParentFile();
if (!dir.isDirectory() && !dir.mkdirs())
throw new FileNotFoundException("Failed to ensure directory: " +
dir.getAbsolutePath());
if (ze.isDirectory())
continue;
FileOutputStream fout = new FileOutputStream(file);
try {
while ((count = zis.read(buffer)) != -1)
fout.write(buffer, 0, count);
} finally {
fout.close();
}
}
} finally {
zis.close();
}
}

We launch the unzip operation from the activity's onCreate method:

private File assistantLocation;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
assistantLocation = new File(getFilesDir(), "snips");
extractAssistantIfNeeded(assistantLocation);
}

Requesting record permission from the user

Next, we need to ensure that appropriate permissions have been granted by the user. In the following, we create a method ensurePermissions() that checks for permission to record, and if not granted, prompts the user to grant it. A second callback method onRequestPermissionsResult is called when the user grants (or denies) this permission. In the former case,

private boolean ensurePermissions() {
int status = ActivityCompat.checkSelfPermission(this,
Manifest.permission.RECORD_AUDIO);
if (status != PackageManager.PERMISSION_GRANTED) {
ActivityCompat.requestPermissions(this, new String[]{
Manifest.permission.RECORD_AUDIO,
Manifest.permission.READ_EXTERNAL_STORAGE
}, 0);
return false;
}
return true;
}
@Override
public void onRequestPermissionsResult(int requestCode,
@NonNull String[] permissions,
@NonNull int[] grantResults) {
super.onRequestPermissionsResult(requestCode, permissions, grantResults);
if (requestCode == 0 && grantResults.length > 0 &&
grantResults[0] == PackageManager.PERMISSION_GRANTED) {
startSnips(assistantLocation);
}
}

Creating the Snips client

Next, we create the Snips client, and binds callbacks so that we can react to various states of the platform:

private static final String TAG = "MainActivity";
private SnipsPlatformClient createClient(File assistantLocation) {
File assistantDir = new File(assistantLocation, "assistant");
final SnipsPlatformClient client = new SnipsPlatformClient.Builder(assistantDir)
.enableDialogue(true)
.enableHotword(true)
.enableSnipsWatchHtml(false)
.enableLogs(true)
.withHotwordSensitivity(0.5f)
.enableStreaming(false)
.enableInjection(false)
.build();
client.setOnPlatformReady(new Function0<Unit>() {
@Override
public Unit invoke() {
Log.d(TAG, "Snips is ready. Say the wake word!");
return null;
}
});
client.setOnPlatformError(
new Function1<SnipsPlatformClient.SnipsPlatformError, Unit>() {
@Override
public Unit invoke(final SnipsPlatformClient.SnipsPlatformError
snipsPlatformError) {
// Handle error
Log.d(TAG, "Error: " + snipsPlatformError.getMessage());
return null;
}
});
client.setOnHotwordDetectedListener(new Function0<Unit>() {
@Override
public Unit invoke() {
// Wake word detected, start a dialog session
Log.d(TAG, "Wake word detected!");
client.startSession(null, new ArrayList<String>(),
false, null);
return null;
}
});
client.setOnIntentDetectedListener(new Function1<IntentMessage, Unit>() {
@Override
public Unit invoke(final IntentMessage intentMessage) {
// Intent detected, so the dialog session ends here
client.endSession(intentMessage.getSessionId(), null);
Log.d(TAG, "Intent detected: " +
intentMessage.getIntent().getIntentName());
return null;
}
});
client.setOnSnipsWatchListener(new Function1<String, Unit>() {
public Unit invoke(final String s) {
Log.d(TAG, "Log: " + s);
return null;
}
});
return client;
}

Starting Snips

We are ready to start Snips, which we do within the onCreate method:

@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
assistantLocation = new File(getFilesDir(), "snips");
extractAssistantIfNeeded(assistantLocation);
if (ensurePermissions()) {
startSnips(assistantLocation);
}
}
private void startSnips(File snipsDir) {
SnipsPlatformClient client = createClient(snipsDir);
client.connect(this.getApplicationContext());
}

Step 6: Summing up

To sum up, the MainActivity.java file looks as follows:

package ai.snips.hellosnips;
import android.Manifest;
import android.content.pm.PackageManager;
import android.os.Bundle;
import android.support.annotation.NonNull;
import android.support.v4.app.ActivityCompat;
import android.support.v7.app.AppCompatActivity;
import android.util.Log;
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import ai.snips.hermes.IntentMessage;
import ai.snips.platform.SnipsPlatformClient;
import kotlin.Unit;
import kotlin.jvm.functions.Function0;
import kotlin.jvm.functions.Function1;
public class MainActivity extends AppCompatActivity {
private static final String TAG = "MainActivity";
private File assistantLocation;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
assistantLocation = new File(getFilesDir(), "snips");
extractAssistantIfNeeded(assistantLocation);
if (ensurePermissions()) {
startSnips(assistantLocation);
}
}
private boolean ensurePermissions() {
int status = ActivityCompat.checkSelfPermission(this,
Manifest.permission.RECORD_AUDIO);
if (status != PackageManager.PERMISSION_GRANTED) {
ActivityCompat.requestPermissions(this, new String[]{
Manifest.permission.RECORD_AUDIO,
Manifest.permission.READ_EXTERNAL_STORAGE
}, 0);
return false;
}
return true;
}
@Override
public void onRequestPermissionsResult(int requestCode,
@NonNull String[] permissions,
@NonNull int[] grantResults) {
super.onRequestPermissionsResult(requestCode, permissions, grantResults);
if (requestCode == 0 && grantResults.length > 0 &&
grantResults[0] == PackageManager.PERMISSION_GRANTED) {
startSnips(assistantLocation);
}
}
private void startSnips(File snipsDir) {
SnipsPlatformClient client = createClient(snipsDir);
client.connect(this.getApplicationContext());
}
private void extractAssistantIfNeeded(File assistantLocation) {
File versionFile = new File(assistantLocation,
"android_version_" + BuildConfig.VERSION_NAME);
if (versionFile.exists()) {
return;
}
try {
assistantLocation.delete();
MainActivity.unzip(getBaseContext().getAssets().open("assistant.zip"),
assistantLocation);
versionFile.createNewFile();
} catch (IOException e) {
return;
}
}
private SnipsPlatformClient createClient(File assistantLocation) {
File assistantDir = new File(assistantLocation, "assistant");
final SnipsPlatformClient client =
new SnipsPlatformClient.Builder(assistantDir)
.enableDialogue(true)
.enableHotword(true)
.enableSnipsWatchHtml(false)
.enableLogs(true)
.withHotwordSensitivity(0.5f)
.enableStreaming(false)
.enableInjection(false)
.build();
client.setOnPlatformReady(new Function0<Unit>() {
@Override
public Unit invoke() {
Log.d(TAG, "Snips is ready. Say the wake word!");
return null;
}
});
client.setOnPlatformError(
new Function1<SnipsPlatformClient.SnipsPlatformError, Unit>() {
@Override
public Unit invoke(final SnipsPlatformClient.SnipsPlatformError
snipsPlatformError) {
// Handle error
Log.d(TAG, "Error: " + snipsPlatformError.getMessage());
return null;
}
});
client.setOnHotwordDetectedListener(new Function0<Unit>() {
@Override
public Unit invoke() {
// Wake word detected, start a dialog session
Log.d(TAG, "Wake word detected!");
client.startSession(null, new ArrayList<String>(),
false, null);
return null;
}
});
client.setOnIntentDetectedListener(new Function1<IntentMessage, Unit>() {
@Override
public Unit invoke(final IntentMessage intentMessage) {
// Intent detected, so the dialog session ends here
client.endSession(intentMessage.getSessionId(), null);
Log.d(TAG, "Intent detected: " +
intentMessage.getIntent().getIntentName());
return null;
}
});
client.setOnSnipsWatchListener(new Function1<String, Unit>() {
public Unit invoke(final String s) {
Log.d(TAG, "Log: " + s);
return null;
}
});
return client;
}
private static void unzip(InputStream zipFile, File targetDirectory)
throws IOException {
ZipInputStream zis = new ZipInputStream(new BufferedInputStream(zipFile));
try {
ZipEntry ze;
int count;
byte[] buffer = new byte[8192];
while ((ze = zis.getNextEntry()) != null) {
File file = new File(targetDirectory, ze.getName());
File dir = ze.isDirectory() ? file : file.getParentFile();
if (!dir.isDirectory() && !dir.mkdirs())
throw new FileNotFoundException("Failed to make directory: " +
dir.getAbsolutePath());
if (ze.isDirectory())
continue;
FileOutputStream fout = new FileOutputStream(file);
try {
while ((count = zis.read(buffer)) != -1)
fout.write(buffer, 0, count);
} finally {
fout.close();
}
}
} finally {
zis.close();
}
}
}

Step 7: Run the app

On the first run, you will be prompted for access to the microphone, which you should grant. The app is now listening, waiting for the wake word (for instance, "Hey Snips", or whatever wake word you defined when you created your assistant on the Snips Console). Say the wake word, followed by a query. You should see the logs appearing in your log window!

Next steps

Congratulations on creating your first entirely private voice app using Snips! It's now time to take it a step further. Make sure to check our other Android Guides and Android Samples Apps for more complex examples!