Control HoloLens with voice commands using Unity3D and the mixed reality toolkit (MRTK)

Can you control your HoloLens app with voice commands? Cortana listens to global commands but what are your options to execute your code by speaking? In this tutorial you will learn how to use the Mixed Reality Toolkit to recognize phrases and how to respond to them.

toolkit

Create a new project

Follow the instructions to create a new HoloLens app in Unity

Import Mixed Reality Toolkit

Use toolkit

toolkit

toolkit

toolkit

toolkit

toolkit

toolkit

toolkit

Create a script to respond to voice commands

The Speech Input Handler is able to trigger functions but in this example we are going to use the ISpeechHandler interface.

Here is the code for the script:

using HoloToolkit.Unity.InputModule;
using UnityEngine;
using UnityEngine.UI;

public class SpeechController : MonoBehaviour, ISpeechHandler
{
    public Text Text;

    public void OnSpeechKeywordRecognized(SpeechEventData eventData)
    {
        Text.text = eventData.RecognizedText;
    }
}

toolkit

toolkit

Before deploying to HoloLens

If you want to deploy this to the HoloLens, you have to enable the microphone capability.

toolkit

Now you can build and deploy to the HoloLens