How do I use stencil buffer on krom/hl target's graphics2? ( T ^ T )

[lampysprites]

Hello!

I'm trying to create a mask for drawing using stencil buffer. Works in html5 as intended but something seems to be missing in other cases. In addition to the example below, tried drawing to an explicitly created Depth24Stencil8 image before the buffer.

Appreciate the help.

os: windows10
graphics: geforce gtx 560
kha: main @ f4fd02a5fd9a4bbc8133d985d1646a16498c44de

How it looks

Code

package;

import kha.Assets;
import kha.Framebuffer;
import kha.graphics4.PipelineState;
import kha.graphics4.VertexData;
import kha.graphics4.VertexStructure;
import kha.math.FastMatrix3;
import kha.Shaders;
import kha.System;

using kha.graphics2.GraphicsExtension;


class Main {
	static var pipeMask:PipelineState;
	static var pipeOver:PipelineState;


	static function render(fb:Framebuffer) {
		var t = kha.System.time,
			g = fb.g2;

		g.begin();

		g.pipeline = pipeMask;
		g.color = 0xffffffff;
		g.fillCircle(400, 300, 200);

		g.flush();

		g.pipeline = pipeOver;
		g.color = 0x88ff00ff;
		g.pushTransformation(FastMatrix3.translation(400, 300).multmat(FastMatrix3.rotation(t)).multmat(FastMatrix3.translation(150, 0)).multmat(FastMatrix3.rotation(-2 * t)));
		g.fillRect(-150, -150, 150, 150);
		g.popTransformation();

		g.end();
	}


	public static function main() {
		System.start({title: "Kha", width: 800, height: 600}, function (_) {
			Assets.loadEverything(function () {
				var structure = new VertexStructure();
				structure.add("vertexPosition", VertexData.Float32_3X);
				structure.add("vertexColor", VertexData.UInt8_4X_Normalized);

				pipeMask = new PipelineState();
				pipeMask.inputLayout = [structure];
				pipeMask.vertexShader = Shaders.painter_colored_vert;
				pipeMask.fragmentShader = Shaders.painter_colored_frag;

				pipeMask.stencilBothPass = Replace;
				pipeMask.stencilMode = Always;
				// pipeMask.colorWriteMask = false;
				pipeMask.stencilReferenceValue = Static(1);

				pipeMask.compile();

				pipeOver = new PipelineState();
				pipeOver.inputLayout = [structure];
				pipeOver.vertexShader = Shaders.painter_colored_vert;
				pipeOver.fragmentShader = Shaders.painter_colored_frag;

				pipeOver.stencilMode = Equal;
				pipeOver.stencilReferenceValue = Static(1);
	
				pipeOver.compile();

				// Avoid passing update/render directly, so replacing them via code injection works
				System.notifyOnFrames(function (framebuffers) { render(framebuffers[0]); });
			});
		});
	}
}

[RobDangerous]

Probably something about the Direct3D 11 backend - please give a regular Windows build a try (without Krom or HL), then we can go from there.

[RobDangerous]

Oh, it's a discussion. Should probably be an issue instead.

[RblSb]

Thanks for nice mask example btw

[lampysprites]

Sure. Although for an example, better change stencil in draw(), without new pipelines, or with just one. The code I grabbed initially had them for different shaders.

  @:privateAccess var pipe = cast(g, kha.graphics4.Graphics2).coloredPainter.pipeline.get(null, Depth24Stencil8).pipeline;
  
  pipe.stencilBothPass = Replace;
  pipe.stencilReferenceValue = Static(1);
  
  g.color = 0xffffffff;
  pipe.stencilMode = Always;
  // pipeMask.colorWriteMask = false;
  g.fillCircle(400, 300, 200);
  
  g.flush();
  
  g.color = 0x88ff00ff;
  pipe.stencilMode = Equal;
  // pipeMask.colorWriteMask = true;
  g.pushTransformation(FastMatrix3.translation(400, 300).multmat(FastMatrix3.rotation(t)).multmat(FastMatrix3.translation(150, 0)).multmat(FastMatrix3.rotation(-2 * t)));
  g.fillRect(-150, -150, 150, 150);
  g.popTransformation();
  
  // restore default drawing
  g.flush();
  pipe.stencilMode = Always;
  pipe.stencilBothPass = Keep;
  pipe.stencilReferenceValue = Static(0);