Control HoloLens with voice commands using Unity3D and the mixed reality toolkit (MRTK).

Create a new project

Follow the instructions to create a new HoloLens app in Unity

Import Mixed Reality Toolkit

• Download or create a Mixed Reality Toolkit package.
• Assets > Import Package > Custom Package
• Open the package you downloaded or created

Use toolkit

• Remove Main Camera from the Hierarchy
• Drag the MixedRealityCamera to the Hierarchy

• From the toolkit, drag UITextPrefab to the Hierarchy

• Set the Render Mode to Screen Space - Overlay

• Expand UITextPrefab and select Text
• Set Text to Say 'create' or 'new'
• Set Color to Red

• Create an empty Object GameObject
• From the toolkit, drag SpeechInputSource and SpeechInputHandler to GameObject

• Add keywords ‘new’ and ‘create’ to the Speech Input Source keywords collection.
• In Speech Input Handler check Is Global Listener

• Drag the toolkit InputManager to the Hierarchy

Create a script to respond to voice commands

The Speech Input Handler is able to trigger functions but in this example we are going to use the ISpeechHandler interface.

• Add a new script to the GameObject and call it SpeechController
• Open in Visual Studio.

Here is the code for the script:

using HoloToolkit.Unity.InputModule;
using UnityEngine;
using UnityEngine.UI;

public class SpeechController : MonoBehaviour, ISpeechHandler
{
public Text Text;

public void OnSpeechKeywordRecognized(SpeechEventData eventData)
{
Text.text = eventData.RecognizedText;
}
}

• Save and go back to Unity
• Drag Text to the property in the Speech Controller script:

• Click Play and if your microphone is on, the app should already respond to the two commands:

Before deploying to HoloLens

If you want to deploy this to the HoloLens, you have to enable the microphone capability.

• Open the Player Settings
• Go to the Publish Settings
• Check Microphone capability

Now you can build and deploy to the HoloLens