AR Voyage

AR Voyage: A Lightship Experience is a hands-on look at the immersive AR experiences you can create using the Niantic Lightship AR Developer Kit (ARDK). The app is currently available on Android and iOS, and also provided as a Unity package for you learn more about ARDK features and how to implement them. You can download the AR Voyage Unity project on the Lightship Developer Portal.

Opening Voyage

  1. Download the ARDK and Generate an API key if you have not done so.

  2. Download the AR Voyage project from the Lightship Developer Portal.

  3. Unzip the tar file to create an ARVoyage folder and place that folder where you want ARVoyage to reside.

  4. Load the project in Unity Hub.

    1. In the Projects section, next to the Open button, click the drop-down and select Add project from disk.

    2. Select the ARVoyage folder.

    3. Set the Editor Version to 2020.3.33f1.

  5. Open the project in the Unity Editor by selecting ARVoyage and click Open.

    1. If you are prompted to enter Safe Mode or Ignore, choose Ignore.

    2. Add the ARDK package to the project.

    3. Open the Build Settings from the File menu and choose iOS or Android as your platform. (For Android, you may need to upgrade Gradle, see Building for Android API level 31 with OpenCL).

    4. Restart Unity.

Scenes

There are 8 scenes in the app:

  • Splash : Displays splash images

  • Loading : Handles loading ARDK features and requesting device permissions

  • Homeland : a 3D main menu for choosing which of 4 demo levels to play

  • SnowballToss (read more): ARDK demo level that demonstrates meshing and occlusion

  • Walkabout (read more): ARDK demo level that demonstrates a navigable Gameboard (an experimental feature)

  • SnowballFight (read more): ARDK demo level that demonstrates a multiplayer session with up to 4 co-located players

  • BuildAShip (read more): ARDK demo level that demonstrates classifying pixels from the camera feed into categories, aka semantic segmentation

  • VPS (read more): Demonstrates placing and interacting with virtual content in real-world locations using ARDK’s Visual Positioning System (VPS).

Scene Management and Organization

SceneStatesAndUI

The flow of a demo’s scene is factored into a collection of States, such as StateInstructions, StateScanEnvironment, StatePlayGame, StateGameOver. At runtime, only one State gameObject is active at a time.

Because the UI elements for a State are tightly coupled to state flow, States and their UIs are factored together as children of the root gameObject SceneStatesAndUI, whose upper hierarchy begins with the gameObjects SafeAreaFitter and LayoutScale that position all of the states’ UIs within the app’s supported aspect ratios.

State

Inspector-set member variables within each State include:

  • Pointers to this State’s UI gameObjects

  • Pointer(s) to the next State(s)

Methods within each State include:

  • Awake - typically sets its gameObject to inactive. (The only exception is whichever State is intended to be first in the demo, typically StateInstructions.) This ensures that only one State is active at a time.

  • OnEnable - runs when a State is set active. It subscribes to events, activates and fades in the State’s UI gameObjects, and makes any needed method calls to the SceneManager, ARDK components and scene helpers, such as turning on/off AR features like mesh scanning, Gameboard creation, etc.

  • Update - often polls for exit conditions for the State, e.g. has scanning completed, has the game time countdown reached 0 - and calls Exit if so.

  • Exit - fades out the State’s UIs, makes method calls to the SceneManager, ARDK components and scene helpers to turn off AR features as needed, sets the gameObject active for the next State in the demo, and sets its own State’s gameObject inactive.

  • OnDisable - runs when a State is set inactive after being active. It unsubscribes to all events.

SceneManager

In each scene, the SceneManager is named <SceneName>Manager. Contains constants, gameObjects, demo state, and scene-specific helper methods used by various States in the demo. These can include:

  • Objects and NPCs in the demo

  • ARDK features, such as Meshing, Depth, Gameboard, SemanticSegmentation

  • Interaction mechanics such as a Placement Button, Camera Reticle

SceneEvents

Container for UI events, and any other scene events.

ARDKManagers

ARDK functionality used in the demo’s scene, documented here, including:

  • ARSceneManager

  • ARMeshManager

  • ARPlaneManager

  • ARDepthManager

  • ARSemanticSegmentationManager

  • ARNetworkingSceneManager

  • ICoverageClient

  • WayspotAnchorService

Helpers

The demos make use of a variety of helper classes, including:

  • SceneDebugManager - In each demo scene, named <SceneName>DebugManager customized for that demo’s debug menu

  • ARMeshFloorFallback - places a large “fallback” floor plane at the height of the lowest known point in the ARDK mesh

  • ARPlaneFloorFallback - places a large “fallback” floor plane at the height of the lowest known horizontal ARPlane

  • ARPlaneHelper - allows for accessing and utilizing the current collection of ARPlanes in the session

  • ARFeaturePointHelper - Animates particles to visualize AR feature points

  • RenderPipelineHelper - Helper for various rendering functionality, including setting render scale

  • LightEstimationHelper - Helper class to change scene lighting based on ARDK light estimation.

SceneSupport

The scenes make use of additional support classes, including:

  • LevelSwitcher - for transitioning between scenes/demos (levels)

  • Fader - for fading in/out/up/down UIs and 3D objects and doing full-screen fades

  • AudioManager - for music and sound effects

  • ErrorManager - for displaying error UIs and banners

EditorOnly

When developing ARDK applications in editor, to avoid always requiring building to a device, it is convenient to employ mock versions of AR assets, such as mock meshes of environments, mock floor planes, and mock players. Each scene’s mock objects are attached to the scene’s MockScene GameObject. This object has a MockSceneConfiguration component and is tagged EditorOnly so it will destroy itself if outside of Unity Editor.

Render Pipeline

AR Voyage uses Unity’s Universal Render Pipeline (URP). Rendering settings are configured in the ARVoyageUrpAssetRenderer.

Homeland Scene

The homeland scene manages navigation of the application. It features Captain Doty, the Niantic Yeti mascot, who is venturing out on an AR Voyage. Each level is accessible by tapping its corresponding waypoint. Tapping a waypoint will trigger a UI to display as Captain Doty walks along the path.

When a level is completed, the player will be presented with a badge on the map screen. After earning the BuildAShip badge, the Niantic airship will become ready on the launch platform. When the player has earned all badges, Captain Doty will board the airship and fly away.

The World Map button in the top left of the viewport allows players to transition to the VPS experience.

The Info button in the top right of the viewport allows players to view Niantic policy links, reset their progress, and see their App ID which can be used for personal data management.

Unity Editor Toolbars

The application contains two custom Unity Editor toolbar entries for convenience.

The Scenes toolbar is managed by the EditorSceneMenu script, and allows for easily loading any of the scenes in the project.

The SaveUtil toolbar is managed by the EditorSaveUtil script, and allows for configuring player save data for testing purposes.