Support Full System.Drawing Functionality on .NET Core #21980
Comments
|
@mellinoe Thanks for the additional information! I actually have a good experience with using Mono/libgdiplus via CoreCompat.System.Drawing. Here's why:
Another option would be to use the System.Drawing code from NetFX and have it consume libgdiplus instead of GDI+ on Linux. There's been some talk of open sourcing System.Drawing before, not sure if that's still an option? It would be fairly easy to implement, too. Regarding UWP, do you currently have demand for System.Drawing on UWP? /cc @akoeplinger |
That is good to know; thanks for the info. If we are able to treat it as a fully external dependency, like the other native libs we use, then I would have less hesitation about using it. I'm not an expert on libgdiplus, so I would certainly like to understand more about it. The folks that I have talked with have been hesitant to readily suggest that we go ahead and use it, but it may be okay in the face of less-promising alternatives. In general, though, I think we should be really cautious about any new native dependencies we adopt in .NET Core. They make a lot of things more complicated for us and for customers.
I was trying to express the idea that folks trying to submit fixes for System.Drawing will have a harder time contributing to an external C library than a managed C# library here. Just a different skill-set, is all.
At the very least, we will be using the .NET Framework version for Windows, so we will certainly open-source it. We can attempt to use that same implementation against libgdiplus on other platforms, but I fear that would involve reworking tons of the native code. Then again, that is just a guess on my part.
We view System.Drawing as one of the main libraries blocking old customers (and their code) from migrating to our new platforms, including UWP. That's not exactly the question you asked, but the end result is: we'd like to support System.Drawing everywhere, because it has proven to be an important dependency for existing libraries. |
My guess is that the .NET Framework version will work well with libgdiplus. It's an assumption based on my experience with libgdiplus and how it tries to be binary compatible with GDI+.
That's a fair point, although on the other hand the C libraries that libgdiplus uses - such as libpng, libjpeg,... have matured for a long time so there's some value to that, too. Graphics code requires a special skillset anyway, perhaps the bulk of people with that skillset are knowledgeable in C? Perhaps another option is to start with libgdiplus and move code from unmanaged to managed. It could help kick start the System.Drawing implementation by reusing libgdiplus and move towards the goal of cross-platform managed code step by step. |
If I was to build a managed 2D graphics library (I am) I wouldn't, by choice, re-implement the System.Drawing API. That said @mellinoe I'm happy to offer and assist in migration if the decision was made to use ImageSharp as an underlying managed library. In Jamestopia though we wouldn't bother and would actively push developers to migrate to better API's. Seems a lot cheaper and easier for MS. There are some fundamental differences in how the two libraries currently work which would have to be discussed and workarounds considered. We're still unfinished so we have great flexibility to alter the API to improve flexibility and fill gaps which would help a lot. Pixel Formats Image Formats
MetaData Affine Transforms - Resize, Crop etc Drawing I'm sure there's more...
@qmfrederik I had a very, very quick scan of the libgdi codebase and immediately found unsupported areas 1, 2. I've no idea what is out there. I remember having issues with
Btw I couldn't read/write C if you held a gun to my head plus I'm teaching myself graphics programming as I write ImagSharp. Shocking 😝 |
|
It is worth updating the original issue with the new information provided in the comments. Mono's System.Drawing implementation was used for Mono's Windows.Forms implementation and we spent a few years tuning it to work with various third party component vendors. We delegated the hard graphic problems to Cairo. It was also built so that on Windows, it would use the system provided GDI+ library, and on other platforms, it uses Mono's This means a couple of things:
While the implementation works, the font handling capabilities are complicated on Unix, so we used FreeType to get this information. While this works great on Linux systems, it is very slow on first use on a Mac that typically lacks FreeType, so if I was going to do the work in this space, I would start by improving libgdiplus to have a swappable font backend and use Apple's CoreText on OSX to avoid the initial font cache creation problems. Cairo and the underlying libraries are faster than any C# code you can write. That said, if I was going to take on the long-term maintenance of System.Drawing, today I would take a different approach. I would use SkiaSharp as the backing engine as it is actively maintained, actively developed, actively fine tuned and optimized for both Chrome and Android. It also addresses some of the design limitations of Cairo, both the bottlenecks of the API as well as issues with the coordinate space that happen due to loss of precision with huge matrix transformations (Mozilla went through this, and made essentially this switch as well). I would start with the sysdrawing-coregraphics which is a project that moved most of the logic from libgdiplus from C to C# and replaced the calls to Cairo with calls to CoreGraphics. And then I would replace the calls to CoreGraphics with calls to the Skia APIs. Last year I did some work along those lines on my spare time, where I mostly butchered the codebase so it would compile without CoreGraphics and I was planning on adding the Skia support. The time to do that never materialized. If you are interested in that fork, you can find it here. |
|
Just had a thought. What about server support. What are the plans there? That should factor into any decision making. |
|
So what is the goal here? Only API compatibility? This is what I understood from what @mellinoe said. If this is the case, @migueldeicaza comparison between native code and C# isn't very relevant, as long as C# performance is decent:
I mean if building the API in managed code is an option, let's go for it. On the other hand, if we settled on "let's just create something" option, let's not go with Win32-only option; this makes it a bit confusing. |
|
Great initiative!! Its better to have System.Drawing implemented on top of set of interfaces and let the community develop each implementation. |
|
You need good APIs for that that to be a viable idea. This is a stopgap not a solution. |
|
I'd personally prefer a more forward-looking approach with an officially supported modern cross-platform graphics API for all .Net platforms, and a generic and optional System.Drawing implementation on top of it. But i'm aware that's a rather extensive and expensive solution and, thus, probably unrealistic. |
|
I would love to see something like an implementation on top of ImageSharp. This hypothetical System.Drawing port isn't for high performance or high quality use - it's just to remove a blocker for porting legacy code in which image handling might be one small function. It would be fine if it was slower than the original netfx System.Drawing or didn't produce bit-identical output in all ops- it might even be able to get away with throwing NotSupported for some rarely-used apis. Building on top of a community library with a modern API would provide a path forward for people to move their code to make direct use of that implentation later on. |
|
Whilst I'm aware that a lot of people use System.Drawing, the WPF System.Windows.Media api is a lot more capable in many areas, and more similar to the UWP API so personally I would rather see a better, more flexible/modern API than System.Drawing available, although of course there may be valuable things in System.Drawing that may be worth implementing |
|
IMHO System.Drawing should be left as Windows only. It is already being used in .Net Framework projects using ASP.Net Core, but those will still be using Windows since those are enterprise applications and I don't see them moving their whole infrastructure "because now they can". |
I disagree. ASP.NET Core is being used by more and more open source projects and startups because of cross-platform support and its speed improvements in comparison to nodeJS. I think it's viable to support System.Drawing on all environments where .NET Core and ASP.NET Core is running. I'm a little worried about binary compatibility here. I think that we definitely need a System.Drawing implementation which is compatible with .NET Framework but I also think that we should build something (together with the community) which is modern, scalable and fast. It might be a good idea to 1. port the existing lib to core (managed) and 2. build a successor of System.Drawing which has all the innovations and improvements which aren't possible with the current API spec. |
|
@ViktorHofer it was said that there are other more modern alternatives like ImageSharp or SkiaSharp. That's why I think it's pointless since those open source projects and startups should use those other alternatives. |
|
@ViktorHofer I've literally spent the last two years of my life doing just that with ImageSharp. It's already faster than System.Drawing in many respects and has a much simpler/easier to extend API. It needs the community to step up to help me push it over the line. Building something else would be a crushing blow to me. |
|
@JimBobSquarePants Then we should definitely also talk about building upon existing libraries 👍 If you put a lot of effort into it we shouldn't exclude ImageSharp from the discussion. |
|
What's the goal?
I had initially imagined it was for (1) where people used System.Drawing in spite of the warnings; but maybe is for (2) and (3)? |
|
@JimBobSquarePants What needs to be done to remove the "in early stages (alpha). As such, we cannot support its use on production environments until the library reaches release candidate status." warning from your project? It's rather off-putting, and given the way that the .NET Core team have redefined 'Release Candidate' to mean what 'Alpha Experimental Technology Preview' might have once meant, it's now anyone's guess in the world of .NET what "early stages (alpha)" might mean, but it does sound like "not something I want to invest time in right now". If you want to mop-up the world of System.Drawing users - I'm one, with a important (to us) ASP.Net application which can't currently be ported to .NET Core because it depends on, inter alia, System.Drawing - then you perhaps need to work a little on the optics for that market.
I don't want to see anyone suffer 'a crushing blow', especially an unintended one from a myopic pachyderm like MS, but I went to look at ImageSharp this morning and came away discouraged. |
|
I'd go with path of least resistance - identify which scenario would be the cheapest and do that. Put warnings all around that it's just compatibility layer that should not be used for new projects. If feeling bad about leftover manpower invest it in improving ImageSharp :) |
|
@willdaen Out of curiosity, did you have a look at CoreCompat.System.Drawing (Mono's Sysyem.Drawing ported to Core)? Would that meet your requirements? |
|
@Kukkimonsuta Yeah, that kind of is what I thought this was about - a cross platform impkementation of System.Drawing which people can (temporarly) use as they move to .NET Core - basically removing one of the adoption blockers. It may not be the most pure solution, but it kind of is what libgdiplus was built for. |
Really? Well that's terribly disappointing. You didn't see the feature list linked to from the readme? I need to update that actually to add all our Porter Duff support and flesh out some other things.
It's in the readme.
The message means exactly what it says. The API is subject to change. (Btw I don't think MS have redefined anything, they just rushed things certainly but alpha, beta etc is not defined by them)
We need to do two things.
After that I'm happy to move to beta and add the library to Nuget. It's been a very slow alpha as I want to get this right. There's simply no point creating a half/baked graphics library. No one will use it and you'll end up wasting your time. P.S. I've no shame in admitting I learned by doing. It's the absolute best way to thoroughly understand things IMO and I see it as an encouragement that others can do the same. 😄 |
|
@JimBobSquarePants Thanks for the response - I am really reluctant to hijack this thread by extending discussion about ImageSharp specifically... I'll give it a try and file any issues on your repo. |
|
@qmfrederik I haven't looked at CoreCompat.System.Drawing - we're blocked on several things (SignalR is another) and I haven't really searched hard for alternatives to everything which is missing - I just keep a vague eye on what's happening to the things I know will cause us problems. |
|
Lots of great feedback from anyone, thank you! (I didn't catch up on all of it) Here's how I look at Priorities
Solutions(2) Mono needs it on Mac & Linux
(3) simplify migration from Windows to Linux/Mac
|
|
@karelz Regarding the Mono implementation on Linux, I don't think it is involved at all. In fact, libgdiplus, the native part, ships with all Linux distros .NET Core supports. So you don't need to port it, the distros have already done that for you. You can declare a dependency on libgdiplus pretty much the same way you do for openssl or curl - acquire it via the distro instead of NuGet. You can consider distributing libgdiplus and its dependencies as native components un NuGet packages. That does get complicated, fast - I tried with CoreCompat.System.Drawing. Mono/libgdiplus gets you 1, 2 and 3 at almost no cost - most of the work has already been done. The result is in the CoreCompat.System.Drawing NuGet package. |
|
@qmfrederik I was relying only on info from @marek-safar who works on Mono - so I let him chime in. |
|
Mr @qmfrederik is correct, libgdiplus is available on pretty much every Linux distribution already and it only relies on the unmanaged pieces. If Marek had any feedback on dealing with build complexity, it would be around you trying to ship a forked version of yours, and then you would have the same issues that you have in shipping CoreCLR - namely that you will want to compile for each system. So that should not be an issue if you want to take the fastest/easiest path to bring System.Drawing. You do not need to worry about iOS, for historical reasons System.Drawing is not used there and we do not even distribute it (beyond the basic Rectangle, Point types). On iOS, the world has gone with either CoreGraphics or our cross-platform SkiaSharp. Given the outlined priorities from @karelz, I would change my recommendation. Given these priorities, really System.Drawing wont be a long-term foundation for graphics, and I would essentially just reuse the Mono code. Rely on gdiplus, and use Mono's System.Drawing stack (and ifdef out parts that wont work on .NET Core 2). You can certainly choose a different path, but given the above goals, it sounds like investing on rewriting this code is a waste of time. |
That was my thinking as well. With the caveat that we might want to use Desktop implementation on Windows to have higher-confidence compat (I've been bitten by HttpListener Mono compat recently, so I'm a bit cautious now ;)).
Agreed. We didn't plan huge investment. And I am not fan of wasting effort. Overall, I don't think |
|
@danmosemsft I'm in process of porting a set of libraries to netstandard. Some part's of them relies on Drawing, eg document exporting/printing. On the other hand document generation/pdf export/dataaccess can work on pure netstandart. |
|
@FixBo Is your issue that System.Drawing.Common targets netcoreapp2.0 but not netstandard? |
|
@qmfrederik Correct. |
|
@qmfrederik At the moment, it actually relies on some private state in the Color structure which is not publically available in netstandard2.0, but are implemented (privately) in .NET Core 2.0. We would have to make some structural changes to the code to re-target to .NET Standard 2.0. Perhaps it is worth looking into after we are stable and solid. |
|
Will System.Drawing.ColorTranslator (I miss System.Drawing.ColorTranslator.FromHtml) be available at net core 2.1? |
|
@safern can you please comment on |
|
I can't use https://www.nuget.org/packages/System.Drawing.Common from a .NET Standard project: Do you have any plan to make it compatible with .NET Standard projects too. Library authors would appreciate it. |
Hi @VahidN thanks for bringing that up. We created issue: https://github.com/dotnet/corefx/issues/24675 and I'll be solving the issue in the upcoming days.
I will have to look why we didn't port it from the beginning. @jmroyb I'll update you on that. |
|
@mellinoe and others: it has taken me considerable time but I’ve read - carefully - through each article nd every comment here. Without weighing in on some of the implementation specific details, I have a question that was mentioned in passing but not addressed. What is the benefit to providing this as a part of .NET Core? Please do not misunderstand me - I appreciate the importance of System.Drawing as a blocking dependency preventing any legacy codebases from migrating to .NET Core. But why must this be addressed by adding a legacy, deprecated, highly system-specific library to the actual .NET Core/Standard specification and implementation? To rephrase: how would simply providing a NuGet package that implements, via one or more of the means discussed above, System.Drawing on these specific platforms not be a sufficient stopgap to help these developers that do not have access to the needed resources to migrate to a better/more modern alternative? By including this in .NET Core, we are not only committing to providing a solution for platforms currently supported by CoreFx but also severely restricting what platforms CoreFx can be easily ported to in the future. It’s all well and good to discuss how awesome SkiaSharp support is on Linux and Android and whatnot, but what happens when someone wants to port CoreFx to SPARC or even just some archaic ARM variant not supported by Skia or libgdiplus or whatnot? What about FreeBSD? Or another OS not yet developed? I am all for developing this library, but I think careful thought needs to be given to its repercussions. I am speaking as a die-hard SWF fan that shunned Silverlight, WPF, and XAML until Windows 10 Creators Update, when I finally embraced UWP. I deeply appreciate the difficulties in learning the subtleties and nuances of new toolkit’s and the difficulties in migrating to a platform with missing dependencies. But what is to be gained by bending over backwards to support System.Drawing? It seems this discussion has almost derailed into a conversation about support SWF on CoreFx and lost sight of the initial goal: to make it easier for applications consuming SD to migrate to CoreFx. I would really appreciate if you would take a step back and perhaps reevaluate this. I do appreciate the time and effort already invested in this undertaking, but none of it need by lost, simply publishing everything discussed but under the context of a separate package than part of .NET Core would accomplish all the same goals. Now to ask a different question that was previously asked but not answered: while SD is definitely a major blocking concern for users of legacy .NET FX apps on Windows blocking their migration to CoreFx, what is the actual number of projects that would benefit from this being a Windows-only NuGet package vs this giant effort to support it across all platforms? How many organizations currently using SD are expected to migrate to .NET Core if SD were available and expand to non-Windows platforms that haven’t already used a better alternative to SD/GDI+? |
|
@mqudsi I assume its being provided via the Windows Compatibility Pack for .NET Core rather than being part of .NET Standard |
|
If I understand clearly your concern is that by adding System.Drawing to .NET Core it would make really hard to extend/port this platform to another framework, platform using different libraries right? If that is your concern and I understood correctly, System.Drawing is not inbox in .NET Core neither netstandard, which means that it is shipped as an independent package in NuGet and if people want to use it they just reference that package in their project. This was done as a compatibility effort to make migration from .NET Framework to .NET Core easier. So we are accomplishing both goals, compatibility and we’re not adding this as part of the framework inbox. Which for your concern (which is reasonable) doesn’t affect you either from what I understood in your comment. Please feel free to correct me if that wasn’t your main concern. To answer your last question, the point of .NET Core is provide portability and extensibility to different platforms, if developers have a .NET Framework app that use System.Drawing but they don’t want to port it because they don’t gain much because it will be Windows-Only supported (the same as they have in their current app), what would be the pint of this effort vs if we support it in Unix also, they gain the portability of their app for free and is a good incentive to move to .NET Core. |
Correct, and also as an individual package in NuGet. |
|
Thank you! This is my second forray into the github page for CoreFX (first was my PR from earlier today) and presumed the issue being here meant it was part of “core” CoreFx. Thanks for clarifying! |
I would argue that offering System.Drawing support on Unix would offer zero incentive to move to .NET Core |
There seems to be demand for System.Drawing on .NET Core; CoreCompat.System.Drawing, CoreCompat.System.Drawing.v2 and System.Drawing.Common combined have over 300,000 downloads on NuGet. To put the "giant effort to support it across all platforms" in some context: Mono has ported System.Drawing to Linux and macOS more than 10 years ago. They've also made libgdiplus compatible with ARM-based OSes such as Android and iOS. What CoreFX is doing is consolidating the Windows, Linux and macOS implementations of System.Drawing. At the end of the day, it's another NuGet package which is available to you, and you're free to use any imaging library which meets your requirements 😄. |
Now that System.Drawing is out of the BCL (a good thing) I think it's important we have new basic (but quality) image manipulation support as part of the BCL, specifically for image resizing and rotation which does not mandate any external dependency. |
|
@safern is this issue still useful? We already have an issue to track marking the compat pack stable. If we want to track consolidating the Windows and Unix code further it would probably make sense to have a new more focused issue. |
|
Agree. We can close this issue. |
|
I was researching a way to migrate our apps toward net core with Platform Extensions and noticed that System.Drawing.ImageConverter is not present in Platform Extensions. Is it going to be suppported? Is this the best place to report such thing, or should I raise a separate issue or report it somewhere else ? |
|
@voltcode probably should open a new issue; for tracking, if its not already in the Windows Compatibility Pack |
|
@voltcode this would be a new issue, but actually it is already being added in dotnet/corefx#33092 which you can track. After that .NET Core should have everything that was in System.Drawing.dll in the System.Drawing namespace - I just checked. |
|
doenst work with blazor Webassembly |
ghost
locked as resolved and limited conversation to collaborators
and others
Update: The following is the current plan.
System.Drawing has been identified as one of the top most-used assemblies that are missing from .NET Core. Although there are 3rd-party alternatives that provide more modern versions of the same functionality (ImageSharp, SkiaSharp, etc.), we believe there is still value in providing a binary-compatible implementation of System.Drawing itself. This issue is intended to document and track the work items involved. There are still lots of open questions; feedback and suggestions are encouraged.
Priorities:
Plan
System.Drawing is, fundamentally, a very thin wrapper over the GDI+ component on Windows. Worse yet, it is not available for UWP applications, so support is limited strictly to Win32 applications. This makes cross-platform implementations difficult, and the largest chunk of work involved with this effort will be figuring out how we can support the library on all of our platforms, and maintaining compatibility between them. There are several existing implementations that we can leverage, but none will support all of our platforms by themselves. We may need several distinct backends in order to get full cross-platform support, although some options could avoid that. The below chart maps implementation approaches (rows) with platform applicability (columns):
.NET Framework Version (GDI+)
This is the existing implementation from the .NET Framework on Windows. It is a very thin wrapper around GDI+.
Direct2D/Win2D
In order to get UWP support, we could implement a compatibility layer over Direct2D, or Win2D.
Mono/libgdiplus
This is the implementation used in mono. It uses a complicated native shim called "libgdiplus", which relies on a large set of native binaries.
Xamarin/CoreGraphics
This is a specialized implementation used by Xamarin, in support of macOS and iOS. Unlike the "libgdiplus" mono version, it only uses CoreGraphics (a system component).
Managed (ImageSharp or custom)
One option is to implement the library in managed code. We can call into an existing library, like ImageSharp, for image-processing and graphics functionality. This is very desirable for a lot of reasons, but it does have a higher implementation cost.
Windows-Specific
One option is to treat the library as a Windows-specific component, similar to Registry. Attempting to install and load the library on other platforms would throw PlatformNotSupportedException's, as Registry does.
Tests
We do not have a very good existing test suite in the .NET Framework. The library is mainly covered indirectly, through other components like WinForms. Mono has a good set of conformance tests for their implementation, which we should be able to re-use. Where appropriate, we should also consider test cases which directly compare output from the .NET Framework implementation and other implementations (assuming we choose to support such things).
Location
An open question is where this library should live. If we take third-party dependencies to support some platforms, we may not want this code to live in corefx.
Printing Functionality
One thing skimmed over above is the fact that System.Drawing exposes a lot of functionality for printing images and documents. This poses a different problem from image-processing and graphics, which can be implemented portably. We may decide to exclude all printing-related portions of the library.
Future investments
Once the library is supported on all platforms, we will accept small, targeted improvements to the library, keeping in mind that legacy compatibility is the only important metric for the library. We will not want to merge any code that refactors, optimizes, or augments the Windows version until we have a fully-working cross-platform implementation. This matches our strategy with other compatibility libraries, like System.IO.SerialPort.
@marek-safar @JimBobSquarePants @ViktorHofer @qmfrederik @karelz
The text was updated successfully, but these errors were encountered: