Is it possible to use MAUI as a GUI?

[MiheevN]

Millennium Challenge.
Maybe a simple answer is known, or maybe this topic will hang for decades until it is found.

It is required to render a complex interface both on top of graphics and in textures to create 3D widgets.
And MAUI is great, in my opinion.
Ideally, the 3D space itself could be one of the Maui graphic elements initially, like the Viewport in the program, but this is probably too much?

What are the options here?
Using your SwapChain, according to the principle described in the topic #435 (comment)
And trying to take the MAUI window through
var hwnd = ((MauiWinUIWindow)App.Current.Windows[0].Handler.NativeView).WindowHandle;
Sdl2Window window = new Sdl2Window(hwnd, true);
But more is needed...😈

The relevant topic in MIUI is https://techcommunity.microsoft.com/t5/app-development/maui-like-3d-gui-in-vulkcan-directx-etc/m-p/3259904

Or maybe there is something much better? Another method for creating a Complex Interface. No worse than in UE4.

[vpenades]

As far as I know, MAUI has two ways of rendering: One is through native UI APIs as found in Android or iOS.

The other one is by writing an implementation of the abstract layer using Maui Graphics.

The relevant code can be found here: https://github.com/dotnet/Microsoft.Maui.Graphics, where you can find backend implementations for native APIs as diverse as DirectX or Skia.

The problem: Maui.Graphics is still in development and the current backends seem to be more a proof of concept than something that can be used in production.

[StefanH-AT]

If you can get native controls in a Maui app and are able to get a handle from it, you should be able to do what I did in #435

[MiheevN]

@StefanH-AT Your decision is even mentioned in the post.

So far I can only get a window. What is also written in the post.
Handle elements are probably not even possible.
But the problem is that MAUI is launched as a separate layer on top of the window, and it is opaque.
What is said here: MicrosoftEdge/WebView2Feedback#286

It looks like this, I'll duplicate it here: This can be seen at the time of resizing:

[MiheevN]

The current simplest solution is this.
Use the CefSharp library and specifically CefSharp.OffScreen
While the color is wrong, I do not know how to solve it.

I'll keep looking until all the problems are solved.
Here are sample code snippets.
The main thing is the Paint event and its pointer, which must be sent to the texture.

browser = new ChromiumWebBrowser("http://zealoussanity.ru/");
browser.WaitForInitialLoadAsync().Wait();

browser.Paint += Browser_Paint;

        private void Browser_Paint(object sender, OnPaintEventArgs e)
        {
            PrimitiveMesh1.UpdateTexture(_graphicsdevice, e.BufferHandle);
        }

_textureData.UpdateTexture(gd, _texture, newdata, false);

        public void UpdateTexture(GraphicsDevice gd, IntPtr newdata)
        {
            _textureData.UpdateTexture(gd, _texture, newdata, false);
        }
        public unsafe void UpdateTexture(GraphicsDevice gd, Texture tex, IntPtr newdata, bool MipMaps)
        {
            for (int level = 0; level < MipLevels; level++)
            {
                Image<Rgba32> image = Images[level];
                Span<Rgba32> pixelSpan = image.GetPixelSpan();
                fixed (void* pin = &MemoryMarshal.GetReference(pixelSpan))
                {
                    gd.UpdateTexture(
                        tex,
                        newdata, //This parameter - Source
                        (uint)(PixelSizeInBytes * image.Width * image.Height),
                        0,
                        0,
                        0,
                        (uint)image.Width,
                        (uint)image.Height,
                        1,
                        (uint)level,
                        0);
                }
            }
        }

Equally important is the parameter

browser.GetBrowserHost().WindowlessFrameRate
Which will regulate the refresh rate.
I seem to have managed to achieve about 30 frames, although I set it to 120.

[Redhacker1]

Do you know if you are using Hardware acceleration with CEF, or if it is possible?

[MiheevN]

I need a very flexible visual interface.

And the browser, along with Blazor, is unfortunately the only and simplest way to implement it.
This is a forced decision, so I would like to do without it. it's just a catastrophic overhead.

I don't know yet if acceleration is used there

[mellinoe]

@MiheevN I've read through most of this and it's still not clear to me, are you trying to:

1. Render the output from a MAUI interface on top of a 3D viewport (e.g. as the interface layer for a game)

or

2. Render a Veldrid viewport embedded inside of what is otherwise a normal, 2D MAUI application?

or

3. something else?

Nevertheless, this is more of a question for the folks over at MAUI. Veldrid has a pretty decent extensibility model and swapchain abstraction, so the real question is what features are/are not supported by MAUI itself.

[vpenades]

@mellinoe I think both @MiheevN (and me) would like is option (1.), that is, to be able to use MAUI as an alternative to ImGUI.

MAUI has been able to write a backend for SharpDX, so doing the same for Veldrid should be possible

[MiheevN]

In fact, Maui is the same WebView.
That is browser rendering.

That's what needs to be rendered. Of course, Maui seems to make it faster and more efficient, but since it's complicated -

you can render just the Blazor Web server. And just run it locally is possible. And our interface becomes a site like https://localhost:9003/
That will allow you to open different tabs with information in many browser windows on additional monitors.

What is the repository for, and a link directly to the example https://github.com/SupinePandora43/UltralightNet/tree/upstream/Examples/gpu/UltralightNet.Veldrid.TestApp

• Allows you to render just any site, for example, there may be general statistics that are loaded not from the inside, but simply at the general address as a site, which will allow you to update it without updating the client.

Now it seems the best option, I will develop it.

Then, more likely, there will be another hybrid option, rendering the visual interface in native fast ways.

[PrestigeDevop]

Is there any update on this yet ?
I want to do most of the UI using shaders - in abstracted way as example i.e using render texture which reflect the native controls buffer and render styles interactively ( without disabling the touch functionality ) - animate them using shader in top of that . more preferably using blazor since it has easier programming model .

native MAUI workloads is out and It would be cool if we could support blazor. if blazorwebview controller can't use opengl-vulkan directly then why can't be used with webGPU since SPIR-V bytecode could be emitted ?

it's a cool idea to manipulate xaml dom tree into some sort of interactive widget which support shaders (like those frameworks which render the entire dom into canvas).

[MiheevN]

@PrestigeDevop But I mean, isn't what I described above what you need?
Have you studied it?

To reiterate: This is rendering a browser window to a texture with Veldrid, naturally using gsl shaders.
Which you can edit as you like, apply any effects and change any part of the rendering process.

but there are other libraries, OpenGL, pure Vulcan, etc.

I render Blazor in 3D space, being able to use it as a visual interface in at least menus and item terminals.
Since it will not be fast everywhere.

That is, this method is the current topical solution to this topic.

There, while studying Ultralight, you can repeat the same thing in C++.