Full C# refactor: move to .NET 6, clean code, add NuGet packaging, more examples, efficient APIs

[KirillAldashkin]

Currenty, C# bindings are built using Mono, which is a completely obsolete toolchain in the .NET ecosystem.
This PR is aimed at updating C# bindings to use modern and well-known .NET tools and do necessary and relevant tasks, such as:

Tasks:

• Update projects format to .csproj
• Update to current LTS version of .NET
• Update the code to meet the C# conventions (also maybe do some optimizations)
• Add NuGet packaging
• Update docs (README, etc.)

This will bring the following benefits:

1. Much better support - can be used by most .NET developers with just one CLI command
2. Easier to build and maintain a library - latest versions of the language and CLI tools

[KirillAldashkin]

It remains only to publish the NuGet package (this needs to be discussed with @hzeller), then add information about it to the README.

[KirillAldashkin]

Implemented the SetPixels(int x, int y, int width, int height, Color *colors) method. It sets the colors of many pixels (or even the entire canvas) in one call.

1. Performance - The PInvoke calls are a bit expensive, and SetPixels() reduces amount of such from 2048 (for my matrix) to 1. Tests showed a threefold increase in the speed of copying a .gif image to a 32x64 matrix (.gif decoding was performed by a pure C# library without native dependencies).
2. Better integrability - This approach allows to use an arbitrary .NET library that works with images in memory, and then quickly copy this image to the matrix.

I am not a professional in C++, and maybe something can be done better.

[hzeller]

I don't know anything about c#, so I'll pas this to @digital-synapse who contributed the original c# binding to comment and review.

[marcokreeft87]

Can you add the option to set the drop privileges argument?

[hzeller]

Oh, and also I need someone to review the code, as I don't know anything about best practices in the C# environment.

[digital-synapse]

Sorry for the long delay!

I have reviewed the changes and they look fine but they break support on raspberry pi 0 and 0w. (This is the reason I implemented with mono for broader support in the first place even though I would have preferred .net core. See dotnet issue here dotnet/iot#998)

Anyways, can we please move these changes to a separate binding instead of changing the existing mono binding. That will preserve compatibility for rpi zeros.

Thanks!

[marcokreeft87]

You can install .NET 6 on a Raspberry Pi zero : https://www.nikouusitalo.com/blog/installing-net-6-on-a-raspberry-pi-zero-2-w/

[digital-synapse]

Oh that's good. That wasn't previously possible. Perhaps just .net core is an issue then.

Ok to merge.

[FabianAronsson]

I have been trying to compile using the mono binding, but it has been a lot of hassle for some reason. This would be a great addition! 👍

[Jan1503]

Well, that looks good!
Maybe its time to fire up my old 384x192 matrix again ;)

[marcogruhl]

Based on this i wrote App for my small 64x32 Matrix AND for faster development i build a LedMatrixSimulator. This will help c# development with this library.

[Jan1503]

Nice! Will definitely look into it if I find some spare time 😀
I wrote a custom usercontrol for winforms to simulate a matrix about 15 years ago.
It has hardware support using an FTDI RS-232 chip to send pixel-data over a serial line to some old RG-displays.

Time is tickin' so fast, it's really time to dig into this stuff again...

btw. I have started to write a simple "Canvas-Manager" for this library a few years ago. It supports an unlimited amount of layers, i.e. for scrolling text, media playback and so on...it's far from final but maybe it's worth sharing.

[Jan1503]

@KirillAldashkin
I just tried your .SetPixels method but can't spot any performance gains so I think the pInvoke's are not the bottleneck.
Tried it with a test-matrix of 64x32 and with my big one 384x192 on a regular rPi4.

I throw an SKBitmap using direct memory pointers to the matrix, which is the fastest method I know of, please see the code using your new method:

private void DrawBitmapPixelsToCanvas(RGBLedMatrix ledMatrix, RGBLedCanvas canvas, SKBitmap bitmap)
        {
            var colors = new List<Color>();                        
            var pixelAddr = bitmap.GetPixels();
            unsafe
            {
                var ptr = (byte*)pixelAddr.ToPointer();

                for (var col = 0; col < bitmap.Height; col++)
                    for (var row = 0; row < bitmap.Width; row++)
                    {
                        var b = *ptr++;
                        var g = *ptr++;
                        var r = *ptr++;
                        ptr++; //Skip alpha

                        //canvas.SetPixel(row, col, new Color(r, g, b));
                        colors.Add(new Color(b, g, r));                        
                    }
            }

            canvas.SetPixels(0, 0, bitmap.Width, bitmap.Height, colors.ToArray());
            ledMatrix.SwapOnVsync(canvas);
        }

[KirillAldashkin]

@Jan1503
Your code is very ineffective since you create two odd buffers and copy all pixel data two times.
new List<Color>() is the first odd buffer, and colours.ToArray() is the second (this method creates a new array).

Some sketch from the phone without any copying:

private unsafe void DrawBitmapPixelsToCanvas(RGBLedMatrix ledMatrix, RGBLedCanvas canvas, SKBitmap bitmap)
{
    var ptr = (Color*)bitmap.GetPixels();
    var span = new Span<Color>(ptr, bitmap.Width * bitmap.Height); //wraps a specified region of memory without any copying
    canvas.SetPixels(0, 0, bitmap.Width, bitmap.Height, span);
    ledMatrix.SwapOnVsync(canvas);
}

[Jan1503]

@KirillAldashkin
Thanks for your code. I didn't know I could fill a span using a ptr.
I'm not at home this weekend, so I cant try it, but at first sight I'm not sure if the colors are in the right order.

[Jan1503]

After a second view I don't think this will work ootb cause you ignore skipping the Alpha value of the SKBitmap pixel.

AfaIk Skia does not support 24-bit pixel format so you have to deal with the alpha value.

[KirillAldashkin]

@Jan1503
Color class from this binging represents 24-bit RGB color (source), so you'll have to create a new RGB buffer to copy SkBitmap pixels into it:

private unsafe void DrawBitmapPixelsToCanvas(RGBLedMatrix ledMatrix, RGBLedCanvas canvas, SKBitmap bitmap)
{
    //allocate buffer on stack, will work only with small image, otherwise
    //use ArrayPool<Color>.Shared.Rent() and ArrayPool<Color>.Shared.Return()
    Span<Color> data = stackalloc Color[bitmap.Width * bitmap.Height];
    var ptr = (int*)bitmap.GetPixels();
    for (var i = 0; i < bitmap.Height * bitmap.Width; i++)
    {
        data[i] = *(Color*)ptr;
        ptr++;
    }
    canvas.SetPixels(0, 0, bitmap.Width, bitmap.Height, data);
    ledMatrix.SwapOnVsync(canvas);
}

But I would just use SixLabors.ImageSharp library instead to manipulate images, which supports a lot of formats (and doesn't have any non-C# dependencies)

[Jan1503]

@KirillAldashkin
Yeah, just read about ImageSharp, will try to build a new version of my CanvasManager based on this.
Do you know how to get the pixel data using this library? Haven't read the entire docs yet but have seen it supports bgr24 and rgb24.

[KirillAldashkin]

@Jan1503
Can't find it out right now. And I think we'd better discuss this somewhere else, since it doesn't relate directly to this binding.

[Jan1503]

@KirillAldashkin
This seems to be the explanation for working with pixel buffers: pixelbuffers

But you're right, let's discuss in your repo.

Edit: Just tried the method above but the index "i" is not declared. Would you mind fixing it?
Could not test on real hardware but surely will on monday.
And where should we talk about this stuff?

[digital-synapse]

The bottleneck will always be the gpio, bus frequency, and the CPU imho. Any graphics processing as much as possible should be performed off CPU (on the GPU) and the frame buffer copied to memory. Following this approach, I was able to render realtime shadertoy pixel shaders on a 128x64 RGB led panel at a consistent 30fps with the mono implementation. (On raspberry pi zero)

…

On Sat, May 27, 2023, 2:33 AM Jan Wrage ***@***.***> wrote: @KirillAldashkin <https://github.com/KirillAldashkin> This seems to be the explanation for working with pixel buffers: <https://docs.sixlabors.com/articles/imagesharp/pixelbuffers.html> But you're right, let's discuss in your repo. — Reply to this email directly, view it on GitHub <#1513 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB6YU7JNUAHSNQY7SPZEG5LXIHC7FANCNFSM6AAAAAAUZC4TIA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[UniqueDaniel]

The code looks good to me, and is very similar to my own implementation where the only major difference is the addition of interfaces for the canvas, font and matrix so I could replace them for testing.

Possibly out of scope for this PR, but there are a couple of possible useful additional functions:

• Draw box of Color
  • Origin, Dimensions, Single color.
• Clone canvas region to canvas region
  • Source, Origin, Dimensions, Destination, Destination Origin
  • Use Case: Fitting text with appropriate cropping into middle of array

I can do the C# parts but c / c++ is well out of my comfort zone. I have some others I have already implemented and will look at getting them into shape and adding them once this PR is merged.

[KirillAldashkin]

@UniqueDaniel I don't think that such APIs are needed because:

1. This is a binding for original library, not extension of it (if you want to extend original library, it's better to create a separate issue)
2. For any complex image manipulation, it's better to use special libraries and then just copy pixel data into rgb-matrix buffer (SixLabors.ImageSharp will be good unless you need very high perfomance)

[Jan1503]

Just for a quick comparison I switched my CanvasManager to ImageSharp and it's about 20-30 fps slower than Skia, mostly because of the pure managed implementation I think.
Have not done real measurements yet, just use the fps-output from the library on console for a quick spot.

This is my implementation using Skia (thanks @KirillAldashkin for optimization, though it not seems faster ;)):

private unsafe void DrawBitmapPixelsToCanvas(RGBLedMatrix ledMatrix, RGBLedCanvas canvas, SKBitmap bitmap)
        {            
            Span<Color> data = stackalloc Color[bitmap.Width * bitmap.Height];
            var ptr = (int*)bitmap.GetPixels();
            for (var col = 0; col < bitmap.Height; col++)
                for (var row = 0; row < bitmap.Width; row++)
                {
                    data[row + (col * bitmap.Width)] = *(Color*)ptr;
                    ptr++;
                }
            canvas.SetPixels(0, 0, bitmap.Width, bitmap.Height, data);
            ledMatrix.SwapOnVsync(canvas);
        }

and this is ImageSharp:

private void DrawBitmapPixelsToCanvasImageSharp(RGBLedMatrix ledMatrix, RGBLedCanvas canvas, Image<Rgb24> img)
        {
            //Configuration customConfig = Configuration.Default.Clone();
            //customConfig.PreferContiguousImageBuffers = true;            
            {
                if (!img.DangerousTryGetSinglePixelMemory(out Memory<Rgb24> memory))
                {
                    throw new Exception(
                        "This can only happen with multi-GB images or when PreferContiguousImageBuffers is not set to true.");
                }

                using MemoryHandle pinHandle = memory.Pin();
                unsafe
                {
                    var ptr = (Color*)pinHandle.Pointer;
                    var span = new Span<Color>(ptr, img.Width * img.Height);
                    canvas.SetPixels(0, 0, img.Width, img.Height, span);
                }
            }
            ledMatrix.SwapOnVsync(canvas);
        }

[KirillAldashkin]

though it not seems faster

I meant that one SetPixels() call is faster then a lot of SetPixel() calls, since for every PInvoke call .NET runtime should make sure that GC will not mess things while unmanaged code is executing. In cases where processing images takes a lot of time, it is not so noticeable.

@Jan1503 Memory struct already has Span property, so you don't have to mess with unsafe

private void DrawBitmapPixelsToCanvasImageSharp(RGBLedMatrix ledMatrix, RGBLedCanvas canvas, Image<Rgb24> img)
{
    if (!img.DangerousTryGetSinglePixelMemory(out Memory<Rgb24> memory))
        throw new Exception("This can only happen with multi-GB images or when PreferContiguousImageBuffers is not set to true.");
    canvas.SetPixels(0, 0, img.Width, img.Height, MemoryMarshal.Cast<Rgb24, Color>(memory.Span));
    ledMatrix.SwapOnVsync(canvas);
}

(and Pinned memory should be released: pinHandle.Dispose())

[Jan1503]

@KirillAldashkin
Thanks, haven't investigate it further, was in a hurry and happy that it worked 😉

[hzeller]

Alright, looks like things are settled so far. @digital-synapse is fine with merging according to this comment.

If there are remaining things to work on, please just open a new pull request.