Add support for consuming sceneform sdk as a dynamic module (#450)

v1.23.0

Release 1.23.0

Controlling Renderable culling + Fixed destroying skybox + ARCore 1.31.0 + Filament 1.21.1

Features

• Added a possibility to control Renderable culling bdc7ea9

Fixes

• Fixed destroying the skybox twice 92b8c8b
• Fixed URL in the sample 5b09ab7

Updates

• ARCore 1.31.0
• Filament 1.21.1
• Updated other dependencies b0422a4

Filament 1.17.0 + Fix wrong color rendered + Fix texture frame stream init + Update dependencies

• Filament 1.15.1 -> 1.17.0

• Fixes:

  • Set transparent background color for transparent SceneView (#309)
  • Restored deprecated tone mapping (#311):
    .toneMapper(new ToneMapper.Filmic()) -> toneMapping(ColorGrading.ToneMapping.FILMIC)
    Solved Issues:
    • Blue light turns violett in Camera View (#295)
    • The red color in ViewRenderable display as orange (#252)
    • Out-of-gamut colors produced by inverse tone mapping cause hue shifts (google/filament#3909)
    • Incorrect colors in ViewRenderable (#60)
  • Fix wrong setShadowReceiver()
  • Avoid using previously bound texture if view texture isn't ready (#317)
    Solved Issue:
    • Some first frames were rendering the previous used stream while initializing the Texture. In case of AR the previously used stream might be the Camera in most cases. So that's why you should previously see the camera inside the ViewRenderable for 1 or 2 frames at start.
    • The texture status is now passed to the ViewRenderable material and the transparent color is used instead of the texture if it isn't ready.
    • Revert the "Initialize ViewRenderable with transparent texture and set it in material right away" (#273) pull request that didn't solve the issue.

• Updated several libraries, outsourced version numbers into the root gradle file

• Updated libraries:

  • compileSdkVersion 30 -> 31
  • targetSdkVersion 30 -> 31
  • appcompat 1.3.1 -> 1.4.1
  • fragment-ktx 1.3.6 -> 1.4.1
  • fragment 1.3.6 -> 1.4.1
  • kotlin_version 1.5.30 -> 1.6.10
  • core-ktx 1.6.0 -> 1.7.0
  • material 1.4.0 -> 1.5.0
  • constraintlayout 2.1.1 -> 2.1.3

• Recompiled all materials and tested all samples

• Documentation:

  • Add some Augmented Face model how to" and links to the templates

Filament 1.15.1 + ARCore 1.29.0 + New VideoNode features + Fixes + Documentation

• Filament 1.12.8 -> 1.15.1
• ARCore 1.26.0 -> 1.29.0
• VideoNode:
  • Fixed wrong positioning (#217)
  • The VideoNode is now centered and above a ParentNode.
  • Added a new flag to enable a look to camera feature.
  • With the function call setRotateAlwaysToCamera(boolean rotateAlwaysToCamera), the VideoNode will always look to the Camera
    (the user). The best effect can be achieved with a Video shown on a Plane.
• Add Proguard ARCore keep rule
• Fixes:
  • Call setDisplayGeometry after Session is resumed (#267): Fix wrong behaviors when resume is called too late
  • Fix ShapeFactory cube : Reversed front/back vertices (#272)
  • Initialize ViewRenderable with transparent texture and set it in material right away (#273): Fix default texture first frame showing the camera stream
• Documentation

Fixed VideoNode position + Fixed AugmentedFaceNode crash + Reusable cameraTextureId

• Fixed wrong positioning of a VideoNode (#217)
  • The VideoNode is now centered and above a ParentNode
  • Added a new flag to enable a look to camera feature
  • With the function call setRotateAlwaysToCamera(boolean rotateAlwaysToCamera), the VideoNode will always look to the Camera (the user).
  • The best effect can be achieved with a Video shown on a Plane. It is not very practical with ChromaKey Videos.
• Fixed AugmentedFaceNode crash
  • The augmented face node can now also being used with a model without skeleton/bones for positioning parts
• Made arSceneView.cameraTextureId public
  • You can now reuse it on another SceneView CameraNode or directly with a SurfaceView
• Added a lot of docs around LightEstimation which explains different usages
  • Please read it 😉

Load Renderable with a coroutine scope + Documentation + Fixes

• Load Renderable with a coroutine scope
• Obfuscating: Added consumerProguardFiles for Filament
• Fixes:
  • Kotlin dependencies split
  • #207 - LightEstimationConfig init
• Samples rename
• Documentation and new repo links

Filament 1.12.8 + Documentation and glTF Sample in Kotlin + Fixes Augmented Face + Optimizations

• Filament 1.12.0 -> 1.12.8
• Documentation and samples
  • Home samples moved to Kotlin
  • glTF Sample moved to kotlin (gtf-java for old java version)
  • Add "Remove an AnchorNode from the Scene"
  • Website visual changes
• Fixes:
  • Disable Light estimation when the front face camera is used because ARCore estimation only works with the back camera
• Performance
  • AugmentedFaces optimisation and generalization
  • Plain videos can now be shown again without glitches
  • Updated the Depth Material to work again with the latest Filament Version
  • Made small improvements for setting texture name in ArSceneView
• Cleanup
  • Use the current branch for all samples if build in debug mode
  • Cleanup build.gradle files

Environment Lights + Augmented Faces + Kotlin Begins + Green Flickering Fix + Performances

Big update here!!!

I have to say that I'm very enthusiastic about this release and here is why:

Entry point to Kotlin and Coroutines full migration

Every new features will also be written in Kotlin. This will bring a lot to Sceneform Maintained which can actually be pretty hard to understand since everything is fragmented in not very clear classes.
Moreover, a lot of leaks were happening not only after destroying the SceneView but also inside the frame callback which generated a memory consumption increase.
Thanks to Android Lifecycle, we will now insure that every pending call is canceled and every used resource is destroyed when the Activity, the Fragment and the View is destroyed.
Beyond all that, the old LightProbe was the last SFB consumer so this release is opening the gate to a big cleanup of unused classes and most important, we will be able to remove the SFB, JNI and lull parts.

Environment Lights

Everything we can see, every beauty comes from lights. They are the main factor of every visual things and so our physical rendering SDK.
From spectacular to realistic results, everything is about lighting.

Sceneform had a big lack on this part and was archived just after the ARCore HDR lighting release.
As an example, the directional light retrieved from ARCore was implemented as a SunNode which could have been great considering it should correspond to an anchor in the real world. But the fact is that it was enormously frame consuming for just handling a direction and intensity (you will be convinced when testing this release).

• You can now load your environment from a single HDR or KTX file locally or from an http/https url.
• IBLPrefilter.specularfilter is integrated and used for HDR files.

Have a look here #156 if you want to learn more about it.

ARCore Light Estimate provides directional light, ambient spherical harmonics, and reflection cubemap estimation

Light bounces off of surfaces differently depending on whether the surface has specular (highly reflective) or diffuse (not reflective) properties. For example, a metallic ball will be highly specular and reflect its environment, while another ball painted a dull matte gray will be diffuse. Most real-world objects have a combination of these properties.
Think of a scuffed-up bowling ball or a well-used credit card.

• Choose depending on your needs if you want REALISTIC or SPECTACULAR result.
• Go further and refine light estimation by defining your custom LightEstimationConfig

A big thank to the ARCore Team (@devbridie) and to the Filament Team (@romainguy) for helping us making the two "worlds" live together with the correct estimated to physical values.

Augmented faces

This part was missing since the SFB to glTF move but thanks to the enormous work of @fvito and @grassydragon (fragment shader genius),
after 3 months of reflections, discussions and developments (#93), the result is just amazing!!!

We are not speaking here of a simple filter that add decorations around the detected face but of a complete real time face "substituter" (bones positioning/orientation, skin/texture modeling, face occlusion)

• The visual effects can consist of up to two components: The face mesh and the face regions renderable.
• The face mesh is used for visualizing 2d images mapped to the surface of someone's face. The face regions renderable is used for visualizing 3d objects (e.g. fox nose and ears) that will move with the regions of the face that are tracked by ARCore. When creating a face regions Renderable, the regions are mapped to bones in the model.

Fix green flickering and laggy camera stream

Thanks to @RGregat the camera stream is now very responsive and the 1.19.6 green flickering is now solved.

• Swapped call order of doUpdate and doRender
• Fixed a poorly implemented replacement of a deprecated part in the ExternalTexture class

We need sponsoring to continue...

After one year of days and nights working on Scenform Maintained, we didn't get even $1 sponsoring.

Sceneform Maintained is not only a quick library for helping developers but a complete AR SDK.

After a quick look at the Play Store, we can see that they are a lot of paying users (billions for some) apps that are using this SDK.
I'm not looking forward to earn a living with Sceneform Maintained but every work that make you earn money should also be remunerated/redistributed to this repository contributors.

So my decision for now is to keep the new features (Instant placement, depth points, objects detection, environment skybox,..) private and spend my time on personal remunerative apps until I get a descent sponsoring.

Note that this is only my personal choice but other contributors can still make PR.

ARCore 1.26.0 + Filament 1.12.0 + Performances + Refactoring

• ARCore 1.25.0 -> 1.26.0
• Filament 1.10.7 -> 1.12.0
• Other depencies :
  • appcompat 1.2.0 -> 1.3.1
  • compileSdkVersion 30 -> 31
  • targetSdkVersion 30 -> 31
  • sourceCompatibility VERSION_1_8 -> VERSION_11
  • targetCompatibility VERSION_1_8 -> VERSION_11
  • classpath 'com.android.tools.build:gradle:7.0.0'
• New functions:
  • Add setters for ArFragment/ArSceneview Session and Session Config
  • Add a public destroy() function in SceneView
• Performance and cleanup
  • The Filament renderer begingFrame function call is now before anything ArCore related is updated.
  • Swapped order of the doUpdate and doRender function call in SceneView.
  • Added Choreographer.getInstance().removeFrameCallback(this); to the destroy function call in SceneView.
  • Added to BaseArFragment a new function to close the session and set the variable to null
  • Fixed unnecessary creation of a camere Texture
  • Replaced deprecated code
  • Instead of throwing an AssertionError if the SwapChain is null, just skip the render call
  • Add a call to Session.destroy() on ArSceneView.onDestroy() and ensure that everything is well called
  • Improve session.getConfig() calls
  • Review the SceneView/ArSceneView resume/pause order
  • Remove the ensure updateMode == Config.UpdateMode.LATEST_CAMERA_IMAGE
  • Set the TonMapper for the Filament-Renderer to FILMIC
  • Removed the manual setted TonMapper from every sample app
  • Replaced requestPermissions and onRequestPermissionResult with the new way by creating a ActivityResultLauncher object. The result of the request is handled via a callback function.

Filament 1.10.7 + New ArFragment Listeners + Selectable Nodes + Memory optimizations + Automatic Plane and AugmentedImages InstructionsController + Big clean on Trackables + ...

• Filament 1.9.25 1.10.7
• Added arFragment.setOnAugmentedImageUpdateListener(OnAugmentedImageUpdateListener listener)

    @Override
    public void onAugmentedImageUpdate(AugmentedImage augmentedImage) {    
        if (augmentedImage.getTrackingState() == TrackingState.TRACKING
                && augmentedImage.getTrackingMethod() == AugmentedImage.TrackingMethod.FULL_TRACKING) {

            // Setting anchor to the center of Augmented Image
            AnchorNode anchorNode = new AnchorNode(augmentedImage.createAnchor(augmentedImage.getCenterPose()));
            ...​
        }
    }

• Added arFragment.setOnArUnavailableListener(OnArUnavailableListener listener)

    @Override
    protected void onArUnavailableException(UnavailableException sessionException) {
        String message;
        if (sessionException instanceof UnavailableArcoreNotInstalledException) {
            message = getString(R.string.sceneform_unavailable_arcore_not_installed);
        } else if (sessionException instanceof UnavailableApkTooOldException) {
            message = getString(R.string.sceneform_unavailable_apk_too_old);
        } else if (sessionException instanceof UnavailableSdkTooOldException) {
            message = getString(R.string.sceneform_unavailable_sdk_too_old);
        } else if (sessionException instanceof UnavailableDeviceNotCompatibleException) {
            message = getString(R.string.sceneform_unavailable_device_not_compatible);
        } else {
            message = getString(R.string.sceneform_failed_to_create_ar_session);
        }
        ...
    }

• Internationalized default session initialization error strings
• Added node.setSelectable(boolean selectable) for controlling what nodes can be hit by the hitTest.

    /**
     * Defines if the node can be selected within the
     * {@Link com.google.ar.sceneform.collision.CollisionSystem} when a touch event happened.
     * @param selectable true if the node can be selected
     */
    public void setSelectable(boolean selectable) {
        this.selectable = selectable;
    }

• New automatic Plane and AugmentedImages InstructionsController visibility detection.
• Fix AugmentedImage instructions visibility if AugmentedImageDatabase has no images to recognize.
• Memory optimizations on the ARCore calls part.
• Removed the abstract qualifier for some BaseArFragment functions to lightweight the override
• Cleanup/simplified the Session creation and configuration
• Big clean on Trackables
  • Centralized Session.getAllTrackbles() and Frame.getUpdatedTrackables(). For performance and consuming reasons.
  • PlaneRenderer takes an updatedPlanes parameter instead of also getting it from the frame
  • Global Trackables manager for filtering
  • PlaneRenderer and InstructionsController automatically disabled depending on the session.config
  • Augmented Image detection for InstructionsController is made from Trackables instead of AumgentedImagesDatabase