Unity xr

Unity 6 XR Development Guide

Source: Unity 6.3 LTS Documentation (6000.3)

XR Architecture Overview

Unity defines XR as encompassing three application types:

• Virtual Reality (VR) -- The application simulates a self-contained environment around the user
• Mixed Reality (MR) -- Combines virtual environments with real-world settings for interaction
• Augmented Reality (AR) -- The application layers content over a view of the real world

The XR stack in Unity 6 consists of:

1. XR Plug-in Management -- Discovers and loads platform XR SDKs
2. OpenXR Plugin -- Open, royalty-free standard by Khronos for cross-platform XR
3. XR Interaction Toolkit (XRI) -- High-level component-based interaction system
4. AR Foundation -- Cross-platform AR framework with provider plug-in architecture
5. Input System -- Action-based input required by OpenXR and XRI

XR Plug-in Management

Configure via Edit > Project Settings > XR Plug-in Management.

• Enable OpenXR and desired Feature Groups per platform
• Select Interaction Profiles (controller mappings) in the OpenXR > Features tab
• Run Project Validation for build-time compatibility checks
• Verify the active runtime matches target hardware

Supported Runtimes and Platforms

Runtime	Target	Graphics
Windows Mixed Reality	Win64	DX11
Oculus PC + Link	Win64	DX11
Meta Quest	Android arm64	Vulkan
Magic Leap 2	Android x64	Vulkan
SteamVR	Win64	DX11

OpenXR Interaction Profiles

• Eye Gaze Interaction
• HTC Vive Controller
• Khronos Simple Controller
• Microsoft Motion Controller
• Oculus Touch Controller
• Valve Index Controller

OpenXR Scripting API

// Iterate all OpenXR features
var features = OpenXRSettings.Instance.GetFeatures();
foreach (var feature in features)
{
    Debug.Log($"Feature: {feature.name}, Enabled: {feature.enabled}");
}

// Get a specific feature by type
var feature = OpenXRSettings.Instance.GetFeature<MockRuntime>();
if (feature != null)
{
    Debug.Log($"MockRuntime Enabled: {feature.enabled}");
}

// Iterate feature groups (Editor only)
var featureSets = OpenXRFeatureSetManager.FeatureSetsForBuildTarget(
    BuildTargetGroup.Standalone
);
foreach (var featureSet in featureSets)
{
    Debug.Log($"Feature Set ID: {featureSet.featureSetId}");
}

XR Interaction Toolkit

Package: com.unity.xr.interaction.toolkit (v3.3.1 for Unity 6000.3)

A high-level, component-based interaction system for VR and AR. Three foundational elements:

1. Interactors -- Handle user input and initiate interactions
2. Interactables -- Define objects users can interact with
3. Interaction Manager -- Coordinates between interactors and interactables

Interaction States

State	Description
Hover	Interactable is a valid target; indicates intention
Select	User input (button/trigger) triggers active interaction (grab)
Focus	Persists after selection until another interactable is selected
Activate	Secondary contextual action mapped to additional controls

Update Loop

1. Request valid target lists from Interactors
2. Validate existing hover/focus/selection states
3. Exit invalid states (OnSelectExiting, OnHoverExiting, OnFocusExiting)
4. Enter new states (OnSelectEntering, OnHoverEntering, OnFocusEntering)

Interactors are always notified before Interactables for both processing and state changes.

Interactor Types

Component	Purpose
XR Direct Interactor	Close-range (touch/grab) interaction
XR Ray Interactor	Ray-based distance interaction
XR Poke Interactor	Poking/touching interaction
Near-Far Interactor	Hybrid proximity/distance
XR Gaze Interactor	Eye-gaze based interaction
XR Socket Interactor	Snap-point interaction
Climb Teleport Interactor	Climbing locomotion

Interactable Types

Component	Purpose
XR Grab Interactable	Object grabbing and manipulation
XR Simple Interactable	Basic interaction events
Climb Interactable	Climbable surfaces
Teleportation Anchor	Fixed teleport destination
Teleportation Area	Area-based teleport target
Teleportation Multi-Anchor Volume	Volume with multiple anchors

XR Grab Interactable Movement Types

• Instantaneous -- Sets Transform position/rotation every frame; lowest latency but can pass through colliders
• Kinematic -- Updates kinematic Rigidbody during FixedUpdate; syncs physics with visuals
• Velocity Tracking -- Sets Rigidbody velocity/angular velocity; prevents collision penetration but may lag

Interaction Strength

Interactors and Interactables report a normalized [0.0, 1.0] analog selection strength, processed after state changes.

Interaction Groups

A Group contains multiple member Interactors sorted by priority and only allows one Interactor in the Group to interact at a time. Supports nested Groups.

See skills/unity-xr/references/xr-toolkit.md for full API details.

Locomotion System

Locomotion Providers

Provider	Description
Teleportation Provider	Point-and-teleport movement
Snap Turn Provider	Rotates user by fixed angles
Continuous Turn Provider	Smooth rotation over time
Continuous Move Provider	Smooth movement over time
Grab Move Provider	Moves user counter to controller movement
Two-Handed Grab Move Provider	Dual-controller movement, rotation, scaling
Climb Provider	Movement while selecting Climb Interactables
Gravity Provider	Gravitational effects with grounded state detection

Architecture

• Locomotion Mediator -- Manages transformation requests; states: Idle, Preparing, Moving, Ended
• XR Body Transformer -- Queues transformations applied each Update based on priority
• XR Movable Body -- Wraps XR Origin with body-relative transformation

Transformations are applied sequentially based on ascending priority; same-priority transformations apply in queue order.

AR Foundation

Package: com.unity.xr.arfoundation (v6.3.3 for Unity 6000.3)

AR Foundation uses a two-package architecture:

• Base Package -- Provides AR feature interfaces (non-functional alone)
• Provider Plug-ins -- Platform-specific implementations

Supported Platforms

Platform	Provider Plug-in
Android	Google ARCore XR Plug-in
iOS	Apple ARKit XR Plug-in
visionOS	Apple visionOS XR Plug-in
HoloLens 2	OpenXR Plug-in
Meta Quest	Unity OpenXR: Meta
Android XR	Unity OpenXR: Android XR

Feature Subsystems

Feature	Manager Component
Session Management	ARSession
Device Tracking	ARTrackedPoseDriver
Camera	ARCameraManager
Plane Detection	ARPlaneManager
Image Tracking	ARTrackedImageManager
Face Tracking	ARFaceManager
Body Tracking	ARHumanBodyManager
Object Tracking	ARTrackedObjectManager
Point Clouds	ARPointCloudManager
Ray Casting	ARRaycastManager
Anchors	ARAnchorManager
Meshing	ARMeshManager
Environment Probes	AREnvironmentProbeManager
Occlusion	AROcclusionManager
Bounding Box Detection	ARBoundingBoxManager
Participants	ARParticipantManager

AR Gesture Interaction (XRI)

• TouchscreenGestureInputController -- Maps screen touches to gesture events
• ARTransformer -- Handles place, select, translate, rotate, and scale gestures
• Screen Space input components feed into XRRayInteractor

Hand Tracking

Hand-tracking devices expose the HandTracking characteristic and CommonUsages.HandData feature:

Hand handData;
if (device.TryGetFeatureValue(CommonUsages.handData, out handData))
{
    // Get wrist bone
    Bone rootBone;
    if (handData.TryGetRootBone(out rootBone))
    {
        Vector3 wristPosition;
        rootBone.TryGetPosition(out wristPosition);
    }

    // Get finger bones (up to 21 bones per hand)
    List<Bone> fingerBones = new List<Bone>();
    if (handData.TryGetFingerBones(HandFinger.Index, fingerBones))
    {
        foreach (Bone bone in fingerBones)
        {
            Vector3 bonePosition;
            bone.TryGetPosition(out bonePosition);
        }
    }
}

Each Bone contains: position, orientation, parent/child references.

Controllers and Haptics

Input Device Access

using UnityEngine.XR;

// Get devices by role
var devices = new List<InputDevice>();
InputDevices.GetDevicesWithRole(InputDeviceRole.RightHanded, devices);

if (devices.Count > 0)
{
    var device = devices[0];

    // Read trigger button
    bool triggerPressed;
    if (device.TryGetFeatureValue(CommonUsages.triggerButton,
        out triggerPressed) && triggerPressed)
    {
        // Handle trigger press
    }

    // Read joystick axis
    Vector2 axis;
    if (device.TryGetFeatureValue(CommonUsages.primary2DAxis, out axis))
    {
        // Use axis.x, axis.y
    }
}

Common Input Usages

Usage	Type	Description
primary2DAxis	Vector2	Thumbstick/touchpad
trigger	float	Trigger axis (0-1)
grip	float	Grip axis (0-1)
triggerButton	bool	Trigger pressed
gripButton	bool	Grip pressed
primaryButton	bool	A/X button
secondaryButton	bool	B/Y button
menuButton	bool	Menu button
userPresence	bool	User wearing headset

Haptics

HapticCapabilities capabilities;
if (device.TryGetHapticCapabilities(out capabilities))
{
    if (capabilities.supportsImpulse)
    {
        uint channel = 0;
        float amplitude = 0.5f;  // 0-1
        float duration = 1.0f;   // seconds
        device.SendHapticImpulse(channel, amplitude, duration);
    }
}

Eye Tracking

Eyes eyeData;
if (device.TryGetFeatureValue(CommonUsages.eyesData, out eyeData))
{
    // Access left/right eye positions, gaze direction, blink amounts
}

Device Lifecycle

// Subscribe to connection events
InputDevices.deviceConnected += OnDeviceConnected;
InputDevices.deviceDisconnected += OnDeviceDisconnected;

// Always validate before use
if (device.isValid)
{
    // Safe to read features
}

Tracking Origin

var subsystems = new List<XRInputSubsystem>();
SubsystemManager.GetInstances<XRInputSubsystem>(subsystems);
if (subsystems.Count > 0)
{
    subsystems[0].TrySetTrackingOriginMode(TrackingOriginModeFlags.Floor);

    // Get play area boundary
    List<Vector3> boundaryPoints = new List<Vector3>();
    if (subsystems[0].TryGetBoundaryPoints(boundaryPoints))
    {
        // Points are floor-level, clockwise-ordered
    }
}

Common Patterns

Basic XR Rig Setup

1. Add XR Origin to scene (provides camera offset and body tracking)
2. Add XR Interaction Manager (single instance coordinates all interactions)
3. Add Tracked Pose Driver to camera and controllers
4. Add Interactors to controller GameObjects
5. Add Interactables to interactive objects

Grab Object Pattern

// On the grabbable object:
// 1. Add Rigidbody
// 2. Add Collider
// 3. Add XR Grab Interactable component
// Configure Movement Type based on needs:
//   - Instantaneous: lowest latency, no physics
//   - Kinematic: physics-synced, moderate latency
//   - Velocity Tracking: full physics, potential lag

Recommended Grab Hierarchy

Grab Interactable (Rigidbody, XRGrabInteractable)
  ├── Visuals (MeshFilter, MeshRenderer)
  ├── Collider (BoxCollider, MeshCollider)
  └── Visual Feedback (Affordance providers)

XR Interaction Simulator

Use the XR Interaction Simulator component to test interactions in the Editor without physical hardware.

Anti-Patterns

• Single Interaction Manager missing -- All Interactors and Interactables require a shared Interaction Manager. Forgetting it causes silent failures.
• Using legacy Input Manager with OpenXR -- OpenXR requires the new Input System package. Legacy Input.GetAxis() will not work.
• Skipping device validation -- Always check InputDevice.isValid before reading features; devices can disconnect at any time.
• Ignoring Attach Ease In Time on sockets -- Socket interactor visual skips occur; set Attach Ease In Time to at least 0.15 to mitigate.
• Mouse/World-space UI with XR providers -- Enabling XR Plug-in Providers breaks standard mouse and world-space UI. Use XR UI Input Module instead.
• Not checking compute shader support -- On platforms without compute shaders, GPU-based features may fail silently. Check SystemInfo.supportsComputeShaders.
• Teleportation without vignette -- Instant teleportation without a Tunneling Vignette Controller causes motion sickness.
• Poke Interactor with UI Toolkit -- Known limitation; use Canvas-based UI with Tracked Device Graphic Raycaster instead.

Key API Quick Reference

Class	Namespace	Purpose
XROrigin	Unity.XR.CoreUtils	Camera rig and tracking space
XRInteractionManager	UnityEngine.XR.Interaction.Toolkit	Coordinates interactions
XRBaseInteractor	UnityEngine.XR.Interaction.Toolkit	Base interactor class
XRBaseInteractable	UnityEngine.XR.Interaction.Toolkit	Base interactable class
XRGrabInteractable	UnityEngine.XR.Interaction.Toolkit	Grabbable object
InputDevice	UnityEngine.XR	Physical XR device
InputDevices	UnityEngine.XR	Device discovery
CommonUsages	UnityEngine.XR	Standard input feature names
OpenXRSettings	UnityEngine.XR.OpenXR	OpenXR configuration
ARSession	UnityEngine.XR.ARFoundation	AR session management
ARPlaneManager	UnityEngine.XR.ARFoundation	Plane detection
ARAnchorManager	UnityEngine.XR.ARFoundation	Spatial anchors

Related Skills

• unity-input -- Input System package, action maps, bindings
• unity-graphics -- Rendering pipelines, shader considerations for XR

Additional Resources

• XR Manual
• XR Interaction Toolkit
• XR Input
• OpenXR Plugin
• AR Foundation
• XR Interaction Toolkit Package
• AR Foundation Package