ARCore SDK for Android v1.37.0

Known issues

• #1573: On certain devices, enabling 60FPS mode causes the tracking to become unsynchronized with the camera feed. The 60FPS mode has been temporarily disabled on these devices and will not appear in supported camera configurations.

New APIs and capabilities

Scene Semantics

Scene Semantics provides richer details about the surrounding scene, making it easier to understand the world around the user. Scene Semantics runs an ML model on the camera image feed and provides a semantic image with each pixel corresponding to one of 11 labels of outdoor concepts.

See Introduction to Scene Semantics and the semantics_java sample app for more information.

Kotlin/Java:

• Frame.acquireSemanticImage()
• Frame.getSemanticLabelFraction()
• Frame.acquireSemanticConfidenceImage()
• SemanticLabel

C:

• ArFrame_acquireSemanticImage()
• ArFrame_getSemanticLabelFraction()
• ArFrame_acquireSemanticConfidenceImage()
• ArSemanticLabel

Streetscape Geometry

Streetscape Geometry is a new ARCore Geospatial API that provides the geometry of buildings and terrain in an area around the user when the Geospatial API is enabled.

See Streetscape Geometry Developer Guide (Kotlin/Java) and Streetscape Geometry Developer Guide (C) for more information. Additionally, the geospatial_java sample app has been updated to include Streetscape Geometry.

Kotlin/Java:

• Config.StreetscapeGeometryMode
• StreetscapeGeometry
• StreetscapeGeometry.Quality
• StreetscapeGeometry.Type
• Mesh

C:

• ArStreetscapeGeometryMode
• ArStreetscapeGeometry
• ArStreetscapeGeometryQuality
• ArStreetscapeGeometryType
• ArMesh

Rooftop anchors

Rooftop anchors are a new Geospatial anchor type that helps you anchor content to a rooftop.

See Geospatial anchors (Java) and Geospatial anchors (C) for more information. Additionally, the geospatial_java sample app has been updated to include Rooftop anchors.

Kotlin/Java:

• ResolveAnchorOnRooftopFuture
• Earth.resolveAnchorOnRooftopAsync()
• Anchor.RooftopAnchorState

C:

• ArEarth_resolveAnchorOnRooftopAsync()
• ArResolveAnchorOnRooftopFuture
• ArRooftopAnchorState

Geospatial Depth

When both the Depth API and Streetscape Geometry modes are enabled in an ARCore session in areas with VPS coverage, the generated depth images are improved using Streetscape Geometry. Streetscape Geometry is integrated into each generated depth image out to a range of 65.535 meters. No changes are needed when calling the Depth API to see this benefit.

See Geospatial Depth (Java) and Geospatial Depth (C) for more information.

Vulkan rendering support

ARCore now provides Vulkan rendering support by exposing the camera image through an AHardwareBuffer. You can use this Hardware Buffer by binding it to a VkImage.

See Rendering your ARCore application using Vulkan and the hello_ar_vulkan_c sample app for more information.

Java:

• Config.TextureUpdateMode
• Frame.getHardwareBuffer()

C:

• ArTextureUpdateMode
• ArFrame_getHardwareBuffer()

Electronic Image Stabilization (EIS)

You can configure ARCore to use Electronic Image Stabilization, which smooths camera frames for a more fluid user experience.

See Enabling Electronic Image Stabilization and the hello_eis_kotlin sample app for more information.

Kotlin/Java:

• Config.ImageStabilizationMode
• Frame.transformCoordinates3d()
• Session.isImageStabilizationModeSupported()
• There is a known issue where calling Frame.transformCoordinates3d() with 2D coordinates other than OPENGL_NORMALIZED_DEVICE_COORDINATES will cause a crash instead of throwing IllegalArgumentException. This will be resolved in the following release.

C:

• ArImageStabilizationMode
• ArFrame_transformCoordinates3d()
• ArSession_isImageStabilizationModeSupported()
• There is a known issue where calling ArFrame_transformCoordinates3d() with 2D coordinates other than AR_COORDINATES_2D_OPENGL_NORMALIZED_DEVICE_COORDINATES will cause your app to abort instead of returning AR_ERROR_INVALID_ARGUMENT. This will be resolved in the following release.

ARCore async APIs

This release introduces new ARCore async APIs to improve ergonomics when working with asynchronous operations, following the Future paradigm.

Java:

• Resolving Terrain anchors: Use Earth.resolveAnchorOnTerrainAsync() to obtain a ResolveAnchorOnTerrainFuture. The old symbol Earth.resolveAnchorOnTerrain() has been deprecated and may be removed in a later version of ARCore.
• Resolving Cloud anchors: Use Session.resolveCloudAnchorAsync() to obtain a ResolveCloudAnchorFuture. The old symbol Session.resolveCloudAnchor() has been deprecated and may be removed in a later version of ARCore.
• Hosting Cloud anchors: Use Session.hostCloudAnchorAsync() to obtain a HostCloudAnchorFuture. The old symbols Session.hostCloudAnchor() and Session.hostCloudAnchorWithTtl() have been deprecated and may be removed in a later version of ARCore.
• Check APK availability: Use ArCoreApk.checkAvailabilityAsync() to register a callback. The old symbol ArCoreApk.checkAvailability() has not been deprecated.

C:

• Resolving Terrain anchors: Use ArEarth_resolveAnchorOnTerrainAsync() to obtain a ArResolveAnchorOnTerrainFuture or use ArResolveAnchorOnTerrainCallback. The old symbol ArEarth_resolveAndAcquireNewAnchorOnTerrain() has been deprecated and may be removed in a later version of ARCore.
• Resolving Cloud anchors: Use ArSession_resolveCloudAnchorAsync() to obtain a ArResolveCloudAnchorFuture or use ArResolveCloudAnchorCallback. The old symbol ArSession_resolveAndAcquireNewCloudAnchor() has been deprecated and may be removed in a later version of ARCore.
• Hosting Cloud anchors: Use ArSession_hostCloudAnchorAsync() to obtain a ArHostCloudAnchorFuture or use ArHostCloudAnchorCallback. The old symbols ArSession_hostAndAcquireNewCloudAnchor() and ArSession_hostAndAcquireNewCloudAnchorWithTtl() have been deprecated and may be removed in a later version of ARCore.
• Check APK availability: Use ArCoreApk_checkAvailabilityAsync() to register a callback. The old symbol ArCoreApk_checkAvailability() has not been deprecated.

Deprecations

C:

• ArVpsAvailabilityFuture_getState() has been deprecated. Use ArFuture_getState() instead.
• ArVpsAvailabilityFuture_cancel() has been deprecated. Use ArFuture_cancel() instead.
• ArVpsAvailabilityFuture_release() has been deprecated. Use ArFuture_release() instead.
• ArCheckVpsAvailabilityCallback has been deprecated. Use ArVpsAvailabilityCallback instead.

Other changes

• There are now base types for all async operations:
  • In Kotlin/Java, an interface Future, with the methods Future.getState() and Future.cancel(). The class VpsAvailabilityFuture now implements Future.
  • In C, a base struct type ArFuture, with functions ArFuture_getState(), ArFuture_cancel() and ArFuture_release(). All future structs in C can be cast to ArFuture. In C++, you can use ArAsFuture() to do the cast.

Bug fixes

• arcore-unity-extensions#90: Fixed a crash when destroying an ARSession while a Cloud Anchor is being resolved.