Render three.js inside pixi.js?

[marcofugaro]

Hey! I am trying to render a three.js scene inside pixi.js to apply it some filters in a postprocessing way, what is the way to do it more efficiently?

I have a working example, here is basically what I do, I init three.js on an offscreen canvas, and then:

const threeTexture = PIXI.Texture.from(threeApp.renderer.domElement)
const threeSprite = new PIXI.Sprite(threeTexture)
// divide by pixelratio to make it the correct size, on retinas it's 2
threeSprite.scale.x /= window.devicePixelRatio
threeSprite.scale.y /= window.devicePixelRatio
pixiApp.stage.addChild(threeSprite)

and then in the render loop I have

threeApp.draw()
threeSprite.texture.update() // tell pixi that threejs has changed
pixiApp.render()

anything else I'm missing? some nicer optimization?

Anyway, the real issue is when I resize the screen, I have this code running in three.js:

renderer.setSize(window.innerWidth, window.innerHeight)

which basically changes the dimensions of the rendered area of three.js, and of course pixi.js breaks.

How do I tell pixi.js to update the texture dimensions?

Environment

• pixi.js version: 5.0.0-rc
• three.js version: 0.101.1
• Browser & Version: Chrome 71
• OS & Version: OSX
• Running Example: https://codepen.io/marco_fugaro/pen/QYgJQY?editors=0010

[ivanpopelyshev]

Its possible, threejs examples were posted several times in Issues for v4, but i didnt make it for v5 yet.
Basically, you have to reset both pixi and threejs state every time you change the libraries.

I cant provide old demos right now, but you can search for them here and in http://www.html5gamedevs.com/forum/15-pixijs/ . Texture sharing was also discussed somewhere.

All you have to do is to search and then port to v5, good luck! I can help with both things if you fail somewhere.

I even made a demo where 2d and 3d objects are SORTED through each other, but that one i didnt share in public.

[ivanpopelyshev]

Does this solve your issues? #5430 Fix is already in 5.0.0-rc2

[marcofugaro]

@ivanpopelyshev thanks for the response! I think you are referring to this discussion and this snippet of code more precisely:

    gl.disable(gl.DEPTH_TEST)
    gl.enable(gl.BLEND)
    gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA)
    gl.disable(gl.STENCIL_TEST)
    gl.disable(gl.CULL_FACE)

    gl.frontFace( gl.CCW )    
    gl.cullFace( gl.FRONT )
    gl.colorMask(true, true, true, true)
    gl.activeTexture(gl.TEXTURE0)

    var shaderManager = session.shaderManager

    shaderManager._currentId = undefined
    for (var i=0; i<shaderManager.attribState.length; i++)
        shaderManager.attribState[i] = false
    shaderManager.setShader(shaderManager.currentShader)

    var blend = session.blendModeManager.currentBlendMode
    if (blend in PIXI.blendModesWebGL) {
        session.blendModeManager.currentBlendMode = undefined
        session.blendModeManager.setBlendMode(blend)
    }

however i don't understand where to insert this code, and what is the session variable, i imagine gl is the glcontext of the three canvas. Where do I put it in this example?

https://codepen.io/marco_fugaro/pen/QYgJQY?editors=0010

(The issues start to happen on the resize of the window)

I have the 5.0.0-rc.2 in the example, should I change something?

[ivanpopelyshev]

I don't know this old example, but all those things you mentioned are included in renderer.reset()

const threeTexture = PIXI.Texture.from(threeApp.renderer.domElement)

this is a bad idea.

Did you look at #5430 ?

[bigtimebuddy]

@marcofugaro Any other follow up on this?

[marcofugaro]

@ivanpopelyshev @bigtimebuddy sorry to keep you waiting, I finally took another stab at this.

@ivanpopelyshev Thanks for the old three in pixi demo you posted in the PR, it was really useful, I tried to implement the method 2 in the next example (also care to explain what was the bad idea with PIXI.Texture.from(threeApp.renderer.domElement)? Sorry I don't know a lot of advanced webgl)

Here is the example converted to pixi v5

https://codepen.io/marco_fugaro/pen/wOGpwv?editors=0010

Some issues I found:

1. PIXI.Application does not accept anymore a context parameter, in the v4 example there is:

var renderer = new PIXI.WebGLRenderer(canvas.width, canvas.height, {
  context: context, // shared context with threejs

I am doing this in v5

 return new PIXI.Application({
    context: context, // shared context with threejs

but obviously it gets ignored, I don't know how the demo is working

2. the three.js texture is flipped on the Y (I know, webgl)

3. when resizing the window the texture gets stretched in an opposite way? probably I'm doing something wrong

Again, thank you for looking into this!

[ivanpopelyshev]

I fixed those issues in another project, I'll make a demo of that code later. There's patch for v4 that reverses the way pixi uses renderTargets (v5 renderTextures).

[marcofugaro]

@ivanpopelyshev could you fix the example now that v5 is out? 🙏

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[marcofugaro]

Stale bot don't

[Trung0246]

Any update so far? The codepen seems working but I'm not sure if it hacky or not.