SKIA for Android: GPU SkImage Rendering Failure on TGLCanvas vs. Vulkan

[Zeus64]

I'm using SKIA4delphi with Android and I want to draw a GPU SkImage directly onto a GPU SkSurface (IE: the canvas of the main form). Currently, the skia framework utilizes TBitmap, which employs a CPU SkImage. This approach is 4-5 times slower than using a GPU image.

When I switch to the default Skia Vulkan canvas (TVKCanvas), everything functions correctly, but the framerate is still 2-5 times slower compared to using the older TCanvasGpu with TTexture. I now want to evaluate the performance using the OpenGL canvas (TGLCanvas) to determine if the reduced speed is related to Vulkan. However, I'm facing an issue where I can't seem to draw a GPU SkImage on the TGLCanvas. There are no errors, but the image doesn't appear on the form.

I've attached a demo for reference. It's set with GlobalUseVulkan := False. When this is changed to GlobalUseVulkan := true, the rectangle is painted as expected.

// IF you set GlobalUseVulkan := True;
// in Project1.dpr then everything work
// fine but with GlobalUseVulkan := False
// nothing is draw :(

procedure TForm1.Button1Click(Sender: TObject);
begin
  var LImageInfo: sk_imageinfo_t;
  LImageInfo.width := 150;
  LImageInfo.height := 150;
  LImageInfo.color_type := {$IFDEF BIGENDIAN}sk_colortype_t.RGBA8888_SK_COLORTYPE{$ELSE}sk_colortype_t.BGRA8888_SK_COLORTYPE{$ENDIF};
  LImageInfo.alpha_type := sk_alphatype_t.PREMUL_SK_ALPHATYPE;
  LImageInfo.color_space := 0;
  TGrCanvas.SharedContext.BeginContext;
  var LGrBackEndTexture := gr4d_directcontext_create_texture(
                             TGrCanvas.SharedContext.GrDirectContext.Handle, // self: gr_directcontext_t;
                             LImageInfo.width, // width,
                             LImageInfo.Height, // height: int32_t;
                             LImageInfo.color_type, // color_type: sk_colortype_t;
                             // color: sk_color_t;
                             False, // is_mipmapped,
                             True, // is_renderable,
                             False); //is_protected: _bool
  var lsksurface := sk4d_surface_make_from_texture(
                      TGrCanvas.SharedContext.GrDirectContext.Handle, // context: gr_directcontext_t;
                      LGrBackEndTexture, // const texture: gr_backendtexture_t;
                      gr_surfaceorigin_t.TOP_LEFT_GR_SURFACEORIGIN, // origin: gr_surfaceorigin_t;
                      1, // sample_count: int32_t;
                      LImageInfo.color_type, // color_type: sk_colortype_t;
                      LImageInfo.color_space, // color_space: sk_colorspace_t;
                      nil); //const props: psk_surfaceprops_t));
  var LPaint := sk4d_paint_create;
  try
    var LDestRectF := TrectF.Create(0, 0, 150, 150);
    sk4d_canvas_draw_rect(
      sk4d_surface_get_canvas(lsksurface), // self: sk_canvas_t;
      @LDestRectF,  // const dest: psk_rect_t;
      LPaint); // const paint: sk_paint_t;
  finally
    sk4d_paint_destroy(LPaint);
  end;
  FSkImage := sk4d_surface_make_image_snapshot(lsksurface);
  invalidate;
end;

procedure TForm1.PaintBox1Paint(Sender: TObject; Canvas: TCanvas);
begin
  if FSkImage <> 0 then begin
    var LSrcRect := TRectF.Create(0, 0, 150, 150);
    var LDstRect := LSrcRect;
    LDstRect.Width := LDstRect.Width / Canvas.Scale;
    LDstRect.height := LDstRect.height / Canvas.Scale;

    var LSamplingoptions: sk_samplingoptions_t;
    LSamplingoptions.max_anisotropic := 0;
    LSamplingoptions.use_cubic := False;
    LSamplingoptions.Cubic.b := 0;
    LSamplingoptions.Cubic.c := 0;
    LSamplingoptions.Filter := sk_filtermode_t.NEAREST_SK_FILTERMODE;
    LSamplingoptions.Mipmap := sk_mipmapmode_t.NONE_SK_MIPMAPMODE;
    //--
    if FPaint = 0 then begin
      FPaint := sk4d_paint_create;
      sk4d_paint_set_antialias(FPaint, false);
      sk4d_paint_set_dither(FPaint, true);
      sk4d_paint_set_style(FPaint, sk_paintstyle_t.FILL_SK_PAINTSTYLE);
    end;
    sk4d_paint_set_alphaf(FPaint, 1);
    //--
    sk4d_canvas_draw_image_rect(
      TSkCanvasCustom(Canvas).Canvas.Handle, // self: sk_canvas_t;
      FSkImage, // const image: sk_image_t;
      @LSrcRect, // const src: psk_rect_t;
      @LDstRect,  // const dest: psk_rect_t;
      @LSamplingoptions, // const sampling: psk_samplingoptions_t;
      FPaint, // const paint: sk_paint_t;
      FAST_SK_SRCRECTCONSTRAINT); // constraint: sk_srcrectconstraint_t)

  end;
end;

Demo.ZIP

[viniciusfbb]

@Zeus64

In the context of OpenGL, the problem probably stems from the absence of the "Flush()" and "Submit()" calls, which depending on the layout should be executed synchronously when the backend is OpenGL.

Unfortunately, OpenGL doesn't have a robust design for multi-threading operations. This contrasts with Vulkan and Metal/Apple, where a single context can be shared between several threads. OpenGL relies heavily on synchronization mechanisms even though it uses shared contexts.

Skia is currently developing a backend called Graphite, which promises significant performance improvements in multi-threaded systems. However, I believe that before long the vast majority of devices in use will all be Vulkan or Metal/Apple compatible.

[Zeus64]

@viniciusfbb

Thanks! but even with Flush() and Submit() the textures are still not drawed :( Here i m not even in multithread as all operations are made in the main thread. Also when working directly with an OpenGL canvas (not Skia), I am able to create and manage textures in the background without encountering significant issues. This leads me to believe that the problem might not solely lie with OpenGL's architecture. Given this, I suspect the issue may be more specifically related to how Skia creates and manages the shared context in OpenGL.

[viniciusfbb]

@Zeus64 we'll be a bit overwhelmed with work over the next few days, but I'll look into your problem as soon as possible.

However, I can provide some preliminary insights. I'll have to look into it when time permits, I had some examples of this, I remember I had problems sharing texture between contexts (even if OpenGL contexts are created in a shared way) through Skia. Texture creation should use the same color type, number of samples (I see you used 1).

[Zeus64]

@viniciusfbb Thanks a lot for your help and your precious work ! I will try also to dig into as soon as possible, right now I m fighting with skparagraph and I need to finish this before to study why GPU image do not work with skia in openGL.

[viniciusfbb]

right now I m fighting with skparagraph

@Zeus64 You can check the implementation of FMX.Skia.Canvas.TSkTextLayout, which uses SkParagraph. However, the TSkTextLayout source has complexities that were only necessary to emulate the exact behavior of other TTextLayout implementations.

[Zeus64]

@viniciusfbb yes the work you have done is my bible right now :) :)

[GrooverMD]

Just a small comment Zeus64

Adding ```delphi before your code block

and ``` after your code block will format your code with Delphi Syntax Highlighting

// IF you set GlobalUseVulkan := True;
// in Project1.dpr then everything work
// fine but with GlobalUseVulkan := False
// nothing is draw :(

procedure TForm1.Button1Click(Sender: TObject);
begin
  var LImageInfo: sk_imageinfo_t;
  LImageInfo.width := 150;
  LImageInfo.height := 150;

Backquote characters are available with the <> button when writing a comment

[viniciusfbb]

I'm closing this topic due to inactivity but feel free to open new ones.

[Zeus64]

@viniciusfbb I will come back a little later regarding this, because the implementation of the OpenGL canvas in skia4delphi is little wrong, i made the correction the the skia units

[viniciusfbb]

@Zeus64 I'm very anxious for improvements! You can open a PR if you want.

Someone reported that it seems to have a memory leak on Android when closing the forms, something like that. So we have to investigate this too.
Another change that we planned in the next release is to enable vsync (at least on Android) in this case:

eglSwapInterval(TGlSharedContext(SharedContext).Display, 1);