How to Build Mixed Reality AI Characters in Unity with Convai (Step-By-Step Guide)

Summary:
Mixed reality (MR) devices like Meta Quest and Apple Vision Pro are making it possible to interact with AI-driven virtual characters that blend seamlessly into our everyday world. In this guide, you’ll learn exactly how to bring Convai-powered AI characters into your Unity MR projects, for the MetaQuest device—using both automatic and manual setup methods—so you can create immersive, intelligent experiences for users.

Why Create AI Characters for Mixed Reality?

As MR headsets gain mainstream adoption, users are looking for interactive, lifelike digital companions—brand avatars, assistants, or guides—that feel truly present in their real environment. With Convai, your AI-driven characters can:

• Walk, talk, and interact naturally within the user’s real space
  
  
• Serve as brand ambassadors, 3D assistants, or even virtual AI companions
  
  
• Perform contextual actions: Move, point out objects, demonstrate features, or guide users step-by-step
  
  
• Deliver hands-free, intuitive experiences on Meta Quest, and more.
  
  

Convai’s Unity integration makes it simple to design these next-generation MR experiences—no heavy coding required.

For an industry perspective, see How AI NPCs Are Changing Virtual Worlds.

How to Get Started: Adding Convai AI to Your Unity Mixed Reality Project

Bringing AI-powered MR characters to life in Unity is straightforward with Convai’s tools. Here’s how to set up your project:

1. Open or Create Your Unity Project

Start with a new or existing Unity MR project.

2. Install the Convai SDK

i. Download and import the Convai SDK for Unity.

ii. Follow the Unity setup tutorial.

3. Add Mixed Reality Support (Automatic or Manual)

i. Click on "Convai" > "Custom Package Installer" > "Install MR Package."

            a. Automatic Setup: Installs all necessary packages and configures project settings (ideal for new projects).

             b. Manual Setup: Installs only the ConvaiXR package—your project settings remain unchanged (best for existing MR projects).

4. Restart the Editor

If prompted, restart Unity to complete installation.

5. Run the Mixed Reality Demo Scene

i. Go to Assets > Convai > ConvaiXR > Scenes
ii. Open ConvaiDemoMR.unity
iii. Import TMP Essentials when prompted.

6. Build & Test on Your Device

i. Go to File > Build Settings
ii. Check the MR demo scene
iii. Select your device (Meta Quest) and hit Build and Run

7. Manual Setup Notes:

i. Add the Convai Essentials XR file to your project hierarchy.

ii. For Meta SDK users, set up Ray interaction for UI panels.

iii. For XR Interaction Toolkit, use the Track Device Graphic Raycaster.

8. Import Your Convai AI Character

i. Add Convai characters following these steps.

ii. Spawn your character in your custom scene by attaching it to a "spawn object" (or use Meta SDK’s building blocks).

💡Tip: For troubleshooting, see the Mixed Reality Integration Guide or get help on the Convai Forum.

Key Use Cases & Real-World Demos

AI Characters That Move, Talk, and Act in Your Room

Demo:

• Conversational AI: The character responds naturally—"Hey, how’s it going?"
• Physical interaction: Users can request, "Can you walk towards the screen/table/door?" The AI character smoothly moves to the correct spot, enhancing presence and immersion.
• Contextual behaviors: AI can dance, jump, or highlight objects in your real room, blending digital and physical seamlessly.
  

For an example project, see Convai’s MR Character Demo.

2. Brand Avatars and 3D Assistants

• Place AI-driven brand mascots or sales assistants in a retail MR space.
• Use Convai’s narrative design to create guided experiences or tutorials.
• Deliver interactive product demos, onboarding, or training directly in the user’s environment.

The possibilities are endless! Reach out to us with some of the amazing projects you create using Convai by tagging us on LinkedIn or X.

Benefits of Convai for Mixed Reality Projects

• Lifelike AI presence: Characters walk, gesture, and converse as if they're in the real room.
• Flexible Unity integration: Automatic or manual workflows for any project stage.
• Cross-device compatibility: Supports Meta Quest, Apple Vision Pro, and other MR/XR hardware.
• No heavy coding required: Build advanced AI character experiences with visual scripting and easy setup.
• Community and support: Extensive documentation, blog tutorials, forum, and Discord.
  

For real-world inspiration, check out our community showcase- Made with Convai.

FAQs About Convai for Mixed Reality in Unity

Q: What MR devices are supported by Convai?
A: Currently the Meta Quest device is supported using Unity’s XR platform. 

Q: Can I keep my project settings if I already have an MR setup?
A: Yes, use the manual installation method to avoid overwriting your Unity configurations. Manual install steps.

Q: How do I spawn AI characters in my custom MR scenes?
A: Attach your Convai NPC as a “spawn object” in your scene, or use the Meta SDK’s spawn positions. 

Q: Where can I get help or see more examples?
A:

• Convai Documentation
• Convai Blog
• Convai Forum
• Convai YouTube Channel

Q: Is there a quickstart video?
A: Yes, check out the video tutorial and our step-by-step MR integration blog.

‍