Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Move the image downsampler from the SkiaSharp repo #101

Open
mattleibow opened this issue Sep 12, 2020 · 2 comments
Open

[FEATURE] Move the image downsampler from the SkiaSharp repo #101

mattleibow opened this issue Sep 12, 2020 · 2 comments

Comments

@mattleibow
Copy link
Contributor

mattleibow commented Sep 12, 2020

Description of Change

  • Currently uses Skia for upsampling (IMHO upsampling always looks blurry anyway)
  • Not optimized yet, just uses Vector4 from the System.Numerics package.
    • Most of the time is actually spent in Skia's color type conversion...
    • I have optimized AVX2 code ready that short circuits the most common case (speedup x5), but I first wanted to share a "minimal" working example, without unsafe pointers, to get feedback
  • The code currently always uses a Parallel.For, but by default only uses a single thread.
  • Since Skia native doesn't have this resampler, I decided to use extensions methods, and put the code in a separate directory.
  • The outputWidth and outputHeight are floats, to allow fractional sizes (just like SKCanvas.DrawImage can draw into a fractional rectangle).
  • The SKImageSamplingOptions.OutputOffset allows to control the sub-pixel offsets, if needed, and allows setting other options.
  • SKCanvas.DrawResampledBitmap(bitmap, sourceRect, targetRect, options, paint) and overloads could be a handy method too, I didn't add this yet.
  • I added an example in the gallery. The baboon image is actually a good image, clearly showing the difference in quality between Skia's native hybrid box-resampler/resizer and this Lanczos based resampler.
  • Currently always uses a Lanczos filter (good average filter), but allows to provide external ones.
  • To improve quality, the image is first upscaled x2, but this is completely integrated in the filter weights, so has zero overhead.
  • I haven't done any testing on ARM devices yet (not sure how to do this).
  • I was able to run all unit tests, but I didn't add any yet. The existing APIs don't change at all.
  • I struggled a lot to verify the results. I reported some bugs to Skia regarding PNG image encoding bugs. Thanks for the GIMP tip, that one always worked correctly.
  • I should add a lot more comments about how this all works ;-)

Bugs Fixed

Skia's default image resizing can be blurry.

API Changes

Added:

  • public static SKBitmap ResampledBy (this SKBitmap srcBitmap, float scaleFactor, SKImageSamplingOptions options = null)
  • public static SKBitmap ResampledBy (this SKBitmap srcBitmap, float widthFactor, float heightFactor, SKImageSamplingOptions options = null)
  • public static SKBitmap ResampledTo (this SKBitmap srcBitmap, float outputWidth, float outputHeight, SKImageSamplingOptions options = null)
  • public static SKBitmap ResampledBy (this SKPixmap srcPixmap, float scaleFactor, SKImageSamplingOptions options = null)
  • public static SKBitmap ResampledBy (this SKPixmap srcPixmap, float widthFactor, float heightFactor, SKImageSamplingOptions options = null)
  • public static SKBitmap ResampledTo (this SKPixmap srcPixmap, float outputWidth, float outputHeight, SKImageSamplingOptions options = null)

Behavioral Changes

None

PR Checklist

  • [n] Has tests (if omitted, state reason in description) -> Added sample (should add many unit tests some day)
  • [y] Rebased on top of develop at time of PR
  • [y] Changes adhere to coding standard -> formatted according to .editorconfig
  • [n] Updated documentation -> Not yet, until we agree on API, if this PR is accepted at all ;-)
@ziriax
Copy link

ziriax commented Jan 10, 2021

I've been doing some more tests, and this resampler isn't yet production ready, it is rather slow. Quality is great though.

Furthermore, it seems Halide already contains a really good and fast high quality image resizer:
https://github.com/halide/Halide/tree/master/apps/resize

Okay, Halide doesn't yet resample in linear color space, so I'm comparing apples with oranges, but for most scenarios, resampling in sRGB space is good enough.

Halide is just... insane. IMHO the way forward for most advanced imaging processing.

It would be awesome if we could somehow integrate the Halide LLVM-based JIT into C# :-)

As a start we could just AOT compile a lot of specialized native libs, and call the correct one at runtime.

NOTE: the existing image scaling filters in Skia are horrible, they produce blurry results when downscaling >0.5 (SkiaSharp 2.80.2), so this problem is not fixed. IMHO Skia should get a native high quality downsampling filter.

@ziriax
Copy link

ziriax commented Jan 19, 2021

I've managed to create an image downsampler that just uses System.Numerics.Vectors that approaches the performance of WIC's FANT downscaler (just a box filter), it is 3% slower on my machine (i7700k). Not sure how it performs on other CPUs.

I'm amazed I could get this performance out of C#, using only old common constructs!

Maybe when using the new intrinsics, I can beat the native implementation, not sure.

However, since this really has nothing to do with Skia, I'm not sure it should be integrated?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants