Video .webm textures not clearing alpha, causing artifacts

[Haxxer]

Hello,

After Issue 3526 was closed, it appeared as if this was fixed for a time, but it has since returned. Perhaps an update to the way Chromium handles playing .webm files causes this, but in any case this issue still remains.

As per this discussion, various people have tried to raise the issue after it was marked as stale in Issue 4089, and after it was missed in a related issue on the Pixi GL Core Repo.

From my experimentations, it appears that the current implementation simply does not clear the alpha after rendering frames, causing these rendering artifacts:

• Playground examples: - white background added to demonstrate alpha difference

  • With fix: https://www.pixiplayground.com/#/edit/BSOldwbSTNfJ7e3WRSltM
  • Without fix: https://www.pixiplayground.com/#/edit/wgmg-EQcyLdFX8EKLWpwV
  • Credit to JB2A https://jb2a.com/ for allowing me to use their asset as an example
  • Note: I do not expect my makeshift patch to be thought of actual implementation, it is only for demonstration purposes.

• pixi.js version: 6.0.4

• Tested Environments

  • Browsers & Versions:
    • Google Chrome Version 94.0.4606.61 (Official Build) (64-bit)
    • Google Chrome Version 93.0.4577.82 (Official Build) (64-bit)
    • Edge Version 94.0.992.31 (Official build) (64-bit)
  • OS & Versions:
    • Windows 10 Pro (64-bit) : Version 20H2, Build 19042.1237
    • Windows 10 Home (64-bit) : Version 10.0.19042, Build 19042
    • Windows 10 Enterprise (64-bit) : Version 19041.1237

[Haxxer]

My apologies, this is not due to Pixi at all, but rather the encoding of the .webm asset.

https://www.pixiplayground.com/#/edit/xiB6wlEPquEwU8FFoWQjb

As you can see in this playground example, I'm toggling between:

baseTexture.alphaMode = PIXI.ALPHA_MODES.NO_PREMULTIPLIED_ALPHA;

and

baseTexture.alphaMode = PIXI.ALPHA_MODES.PREMULTIPLY_ALPHA;

The difference is staggering.

[colinbouvry]

Hello,
I make a filter (glsl) under MP4 H264 video with baseTexture.alphaMode = PIXI.ALPHA_MODES.NO_PREMULTIPLIED_ALPHA
but it stay in premultiplied alpha.
How to do with filter ?
Any idea ?

[ivanpopelyshev]

in filter everything is premultiplied anyway, the video is re-renderedd in temp framebuffer before it goes to filter.

There's a trick to get non-premultipleid info there, but are you sure you need it? You can use mesh-shader instead of filter and it'll get non-premultiplied info from texture , though you still have to output premultiplied in it

[colinbouvry]

Normaly video is rendered in FBO.
I need that work fine when I do this in fragment shader gl_FragColor.a = value
Is it possible to enable alpha on FBO ?
On openGl with C++ is possible !

[ivanpopelyshev]

Normaly video is rendered in FBO.
If you use filter, pixi takes an FBO, and renders sprite with a video there . Video not-pma, but blendmode is multiplying it by alpha, resulting FBO is pma. If you want it not be pma - you can try to lie: baseTexture.alphaMode = PIXI.ALPHA_MODES.PMA ( 2 ) , it will think that everything is done already and wont do any multiplications.

that way you'll get NPM in FBO. Even then, if you change gl_FragColor, to tell filter that your result is non-premultiplied but it has to be multiplied to get mixed with background - filter.blendMode = PIXI.BLEND_MODES.NORMAL_NPM.

Welcome to webgl and pma/non-pma. Only pixi can deal with that, other renderers wont even give you those options.

=========

alternatively, use mesh-shader.

[ivanpopelyshev]

PMA is main mode in Web & 2D . You cant use linear texture filtering on non-pma it will look bad. also non-pma means all your blending modes based on porter-duff wont work. Thus, most of renderers are built around PMA. to add any support of NPM is difficult.

[colinbouvry]

Too many artefact with filter.blendMode = PIXI.BLEND_MODES.NORMAL_NPM
It's like with premultiplied alpha !
Do you have examples of mesh-shader ?

[ivanpopelyshev]

https://pixijs.io/examples/#/mesh-and-shaders/shadertoy-mesh.js

[colinbouvry]

It's the same with mesh shader always artefact !
Any idea ?

[ivanpopelyshev]

hm
alphaMode = PMA , but blendMode is just NORMAL ?

We had a case like a bit before (FBO + shader non-pma), so we had to try all modes and then think why its working :)
Maybe we need better NPM support everywhere , its just i dont want it hinder our main PMA mode

====

alternatively: just use PMA everywhere. divide by alpha when npm is needed, make sure the result is pma

[colinbouvry]

quad.blendMode = PIXI.BLEND_MODES.NORMAL_NPM;
PIXI.BLEND_MODES.NORMAL does not work !
NPM ?
/* Enable WebGL alpha blending */
gl.enable(gl.BLEND)
gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA)
?

[ivanpopelyshev]

gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA) - that's NPM blendmode, it takes npm from texture and produces pma for blending.

At this point , I have to ask you to post whole demo in a separate issue, because this thing is not easy. I told you what we did with other project and its possible, i cant proceeed without a demo i can just hack into

[colinbouvry]

With filter.blendMode = PIXI.BLEND_MODES.NORMAL_NPM, it works fine !
Thanks a lot

[ivanpopelyshev]

Suddenly, it works :) Congratulations!