VPS

Demonstrates placing and interacting with virtual content in real-world locations using ARDK’s Visual Positioning System (VPS).

Two types of example VPS content are outlined below: bespoke “Tourist” experiences, and “Frost Flower” experiences. An overview of the StreetMap is also provided.

ARDK Features in use

  • ARMeshManager - Manages the detection of an environmental mesh with accompanying colliders. Used for flower placement in Frost Flower and dynamic shadows in Bespoke.

  • ARDepthManager - Manages dynamic depth occlusion of virtual content with real-world objects.

  • ARSemanticSegmentationManager - Used to detect parts of the image containing “sky” and “ground”, then exclude them from depth occlusion.

VPS Managers and Helpers

  • VpsSceneManager - Handles shared logic across all VPS states and experiences.

  • VpsIntroManager - Manages the animated airship intro sequence.

  • VpsDebugManager - Manages debug menu options across VPS states.

  • PhotoManager - Manages the compositing and saving of the instant photo images.

  • VpsWayspotManager - Wraps WayspotAnchorService and handles tasks related to localization and placing/resolving anchors.

  • VpsCoverageManager - Wraps ICoverageClient and handles querying and storing results from the VPS Coverage API.

Additional Managers and Helpers

  • AudioManger - Utility for playing world-space, non-spatial and object relative audio.

  • ErrorManager - Utility for displaying fullscreen error messages.

Universal States

A state machine controls the flow of the overall experience. The app can be in one state at a time; each state is authored to dynamically choose the appropriate next state to transition to.

StateVpsIntro

Displays an animated intro sequence featuring Doty’s airship with introductory UI panels.

Following this state, the VpsStreetMap state is sequenced next; see the StreetMap States section.

StateVpsARWarning

Displays a GUI with warning text about using AR the first time the user enters AR in a session.

StateVpsLocalization

The camera and the AR session are activated along with ARDK meshing. The user is prompted to try to localize by framing a the VPS Wayspot’s “hint” image in the viewport. During the localization process, a feature point visualization is shown to give the user feedback that the process is running, along with hints to help guide the user on how to localize.

StateVpsLocalization uses the VpsWayspotManager to start its ARDK WayspotAnchorService and monitors for it to update its LocalizationState.

If the localization attempt fails, the user is informed of the localization failure and the system automatically retries. If the localization succeeds, the the state uses the VpsWayspotManager to restore any WayspotAnchorPayloads associated with the current VpsDataEntry in its PersistentWayspotAnchorPayloadLookup. The VpsWayspotManager uses this lookup to maintain a serialized json file of anchor payload data per localization target identifier. When all anchors restore successfully, the application advances to either StateVpsBespoke or StateVpsFrostFlower, depending on the nature of the VPS wayspot the user selected on the street map. If wayspot anchors fail to restore properly, this is treated as a localization failure that is reported to the user and the system automatically restarts the localization process.

StateVpsLocalizationDestabilized

Runs when when the VpsWayspotManager detects that its WayspotAnchorService’s LocalizationState has changed from Localized. This can happen if the app loses camera tracking or when the app loses focus. All states that display content monitor for the VpsWayspotManager's LocalizationDestabilized event and transition to this state. When localization becomes destabilized, all localized instantiated content is destroyed.

StateVpsTakePhoto

During a particular AR experience, with Doty or Frost Flowers, this state performs the experience (as needed) while displaying a viewfinder and photo button for taking snapshot photos of the experience.

StateVpsViewPhoto

Displays an interactive stack of photos of Doty or Frost Flowers.

StreetMap

Displays an interactive street map showing nearby VPS locations or clusters of VPS locations, built on top of the Mapbox Unity SDK.

This interactive street map shows the user’s current position and marker icons for nearby locations / location clusters. Data for the markers is pulled from the VPS Coverage API by the VpsCoverageManager. If no results are returned by the query, the user is given the option to visit a location with known Wayspots.

Coverage API Queries

Each search query throughVpsCoverageManager happens in two parts. First, a call to RequestCoverageAreasAsync returns a list of CoverageAreas, each containing one or more LocalizationTargetIdentifiers. That list of identifiers is then used for a call to RequestLocalizationTargetsAsync, which returns more detailed information about individual locations (name, coordinates, etc.). Each new location is then wrapped in a VpsDataEntry, cached and used to draw marker icons on the map.

Calls to RequestCoverageAreasAsync for locations that are already cached (from a previous search or as part of an internal list of Frost Flower and Bespoke locations) will be returned directly without the second call to RequestLocalizationTargetsAsync.

Individual Locations

When sufficiently zoomed-in on the map, marker icons are shown for individual locations. The player can tap an individual location to see information about it, including the location’s hint image which is supplied via the url returned from the VPS Coverage API. If the player is close enough to localize at the location, they can begin the process of localizing in AR, followed by playing an AR experience at that location. The current version of AR Voyage requires that a user be within 15 meters to localize at a location. However, as GPS is sometimes inaccurate, if the user is within 30 meters of a location for 30 seconds, the app offers an option to “bypass GPS” and advance to localization. It is best practice for a user to be as close to a localization target as possible to increase the chance of a high quality localization. ARDK requires that the user is within 50 meters of a localization target for localization to be possible.

Location Clusters

When sufficiently zoomed-out on the map, clusters of locations (within a specific search radius) are shown as marker icons, indicating the type and count of locations they contain.

Clustering is performed based on Geohash, a public domain geocode system.

StreetMap Managers and Helpers

  • StreetMapManager- Manages functionality related to navigating in the map view.

  • StreetMapMarkerManager - Manages displaying the user avatar marker, and markers for locations and location clusters.

StreetMap States

StateVpsStreetMap

Activates and directs the interactive StreetMap via method calls to the StreetMapManager.

Following this state, StateVpsLocalization state is typically sequenced next; see the Universal States section.

Bespoke “Tourist” Experiences

Demonstrates custom location-specific content authored for a particular VPS location.

Certain VPS locations in the San Francisco Bay Area have been authored as bespoke “tourist” locations that allow players to interact with and photograph Doty performing location-specific actions.

Data for these locations is stored in scriptable objects that are registered with the VpsCoverageManager and their custom content is instantiated as part of the localization process by using WayspotAnchorPayloads that are included in the application’s source.

The authored “tourist” locations are:

  • Chinatown Gate - Doty poses alongside a lion statue.

  • Cupid’s Bow - Doty gets in a little archery practice.

  • Gandhi Monument - Doty arrives in their airship, and makes a flashy entrance.

  • Pier 14 - Doty casts a fishing line at Pier 14 and makes an unexpected catch.

  • Willie Mays - Doty swings for the fences in front of the Willie Mays statue.

Bespoke States

StateVpsBespoke

Instantiates the appropriate custom content based the selected Wayspot.

StateVpsBespokeGuideToDoty

Displays an animated marker with accompanying arrow to help the player position themselves to view the experience.

StateVpsBespokeOfferCamera

Displays a looping animated sequence in which Doty offers the player an instant camera while dynamically following their position.

Following this state, the universal states TakePhoto and ViewPhoto are sequenced, where Doty performs their location-specific custom behaviors.

StateVpsBespokeBadgeUnlocked

Displays the “Tourist” badge unlock message.

StateBespokePlantFlag

Doty plants a flag at this location and then departs.

Frost Flower Experiences

Demonstrates saving and restoring content placed dynamically by the user at a generic VPS location.

For generic VPS locations (i.e., the bulk of locations that don’t have “Bespoke” tourist content), players are given the opportunity to plant seeds that grow into “Frost Flowers”. Flowers grown during the first visit will soon blossom into larger “Superblooms”, viewable on a subsequent visit to the same location.

Planting

After localizing, a player can plant seeds by targeting a reticle on an invisible AR Mesh object (provided by ARMeshManager) and pressing a launch button. On impact, the VpsWayspotManager is called to place a WayspotAnchor using the transform for an instantiated “hero”. Once the anchor is created, the hero flower’s transform is automatically childed to the anchor so that it will receive any anchor pose updates that come from ARDK. A secondary process then begins raycasting toward the AR Mesh to find growth points for additional plants and grass.

Saving

Upon successfully planting a third seed, the WayspotAnchorPayload for each flower is serialized and stored locally on the device.

Loading

On subsequent visits to a given location, existing WayspotAnchors are deserialized and resolved as part of the localization process described above. New “hero” flowers and secondary foliage are then regrown at their positions.

FrostFlower Managers

FrostFlowerManager handles most aspects of targeting and planting seeds and all associated animation and visual effects. It is also responsible for handling the save/load of frost flower data between user visits to a given location.

FrostFlower States

StateVpsFrostFlower

Displays an interface for launching seeds and growing Frost Flowers when initially visiting a location. On subsequent visits the planted seeds are shown with larger blooms and more secondary foliage.

Following this state, if this a subsequent visit to this location, the universal states TakePhoto and ViewPhoto are sequenced next so the user can take a photo of the now “harvested” Frost Flower “Superblooms”.

StateVpsFrostFlowerBadgeUnlocked

Displays the “Superbloom” badge unlocked message.

StateVpsFrostFlowerSuccess

Displays a final success message.