Same code starting wkhtmltopdf process works on netcoreapp3.1 but fails on net5.0 on Linux

[gurustron]

Description

Repro.

Following code starting process and reading stdout works fine in netcoreapp3.1:

    internal class Program
    {
        private static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");
            using (var process = new Process
            {
                StartInfo = new ProcessStartInfo
                {
                    FileName = "wkhtmltopdf",
                    Arguments =
                        "-q -B 8 -L 8 -R 8 -T 8 --print-media-type --enable-local-file-access -s Letter -O Portrait - -",
                    UseShellExecute = false,
                    RedirectStandardOutput = true,
                    RedirectStandardError = true,
                    RedirectStandardInput = true,
                    CreateNoWindow = true
                }
            })
            {
                process.Start();

                byte[] array;

                var html = "test";
                using (var standardInput = process.StandardInput)
                {
                    standardInput.WriteLine(html);
                }

                using (var memoryStream = new MemoryStream())
                {
                    using (var baseStream = process.StandardOutput.BaseStream)
                    {
                        var buffer = new byte[4096];
                        int count;
                        while ((count = baseStream.Read(buffer, 0, buffer.Length)) > 0)
                            memoryStream.Write(buffer, 0, count);
                    }

                    var end = process.StandardError.ReadToEnd();
                    if (memoryStream.Length == 0L)
                        throw new Exception(end);
                    process.WaitForExit();
                    array = memoryStream.ToArray();
                }

                Console.WriteLine(array.Length);
            }
        }
    }

but fails in net5.0 with:

System.Exception: QPainter::begin(): Returned false
Exit with code 1, due to unknown error.

Configuration

• dotnet --version: 5.0.101
• Ubuntu 20.04.1 LTS
• wkhtmltox_0.12.6-1.bionic_amd64

Regression?

Works in netcoreapp3.1

Other information

related:

1. Linux version cannot write on STDOUT when wkhtmltopdf used as a child-process. wkhtmltopdf/wkhtmltopdf#3119
2. Failure on .NET 5.0 fpanaccia/Wkhtmltopdf.NetCore-deprecated#46 (comment)
3. QPainter::begin(): Returned false Exit with code 1, due to unknown error. .net 5 docker linux fpanaccia/Wkhtmltopdf.NetCore.Example-deprecated#14

[ghost]

Tagging subscribers to this area: @eiriktsarpalis
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

https://github.com/gurustron/repro-process-wkhtml-net-5.

Next code starting process and reading stdout works fine in netcoreapp3.1:

    internal class Program
    {
        private static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");
            using (var process = new Process
            {
                StartInfo = new ProcessStartInfo
                {
                    FileName = "wkhtmltopdf",
                    Arguments =
                        "-q -B 8 -L 8 -R 8 -T 8 --print-media-type --enable-local-file-access -s Letter -O Portrait - -",
                    UseShellExecute = false,
                    RedirectStandardOutput = true,
                    RedirectStandardError = true,
                    RedirectStandardInput = true,
                    CreateNoWindow = true
                }
            })
            {
                process.Start();

                byte[] array;

                var html = "test";
                using (var standardInput = process.StandardInput)
                {
                    standardInput.WriteLine(html);
                }

                using (var memoryStream = new MemoryStream())
                {
                    using (var baseStream = process.StandardOutput.BaseStream)
                    {
                        var buffer = new byte[4096];
                        int count;
                        while ((count = baseStream.Read(buffer, 0, buffer.Length)) > 0)
                            memoryStream.Write(buffer, 0, count);
                    }

                    var end = process.StandardError.ReadToEnd();
                    if (memoryStream.Length == 0L)
                        throw new Exception(end);
                    process.WaitForExit();
                    array = memoryStream.ToArray();
                }

                Console.WriteLine(array.Length);
            }
        }
    }

but fails in net5.0 with:

System.Exception: QPainter::begin(): Returned false
Exit with code 1, due to unknown error.

Configuration

• dotnet --version: 5.0.101
• Ubuntu 20.04.1 LTS
• wkhtmltox_0.12.6-1.bionic_amd64

Regression?

Works in netcoreapp3.1

Other information

related:

1. Linux version cannot write on STDOUT when wkhtmltopdf used as a child-process. wkhtmltopdf/wkhtmltopdf#3119
2. Failure on .NET 5.0 fpanaccia/Wkhtmltopdf.NetCore-deprecated#46 (comment)
3. QPainter::begin(): Returned false Exit with code 1, due to unknown error. .net 5 docker linux fpanaccia/Wkhtmltopdf.NetCore.Example-deprecated#14

Author:	gurustron
Assignees:	-
Labels:	area-System.Diagnostics.Process, untriaged
Milestone:	-

[yakeer]

Following, facing the same issue.

[imb590]

wkhtmltopdf, when instructed to output in stdout, tries to open the file /dev/stdout and that fails with ENXIO.

/dev/stdout is a symlink to /proc/PID/fd/1, which is a socket for the process started from .NET 5 with System.Diagnostics.Process with RedirectStandardInput = true. In .NET Core 3.1, it was a pipe, and Linux syscall open works fine with that; the change was made in #34861.

As far as I understand, there is no way to start a process from .NET 5 through System.Diagnostics.Process with input redirection that works with wkhtmltopdf's way of opening /dev/stdout.

[yakeer]

wkhtmltopdf, when instructed to output in stdout, tries to open the file /dev/stdout and that fails with ENXIO.

/dev/stdout is a symlink to /proc/PID/fd/1, which is a socket for the process started from .NET 5 with System.Diagnostics.Process with RedirectStandardInput = true. In .NET Core 3.1, it was a pipe, and Linux syscall open works fine with that; the change was made in #34861.

As far as I understand, there is no way to start a process from .NET 5 through System.Diagnostics.Process with input redirection that works with wkhtmltopdf's way of opening /dev/stdout.

so basically, as far as I understand, there is no current way to support wkhtmltopdf in .NET 5 in the current situation?

That's a big thing for me, and as far as I believe for other developers that are exporting PDF's in their .NET applications.

[eiriktsarpalis]

Agree with @mbashov's analysis. However the error in the linked reproduction is caused by the fact that it is disposing the StandardInput object prematurely, which results in the corresponding socket being closed. Once I removed the unneeded using statement, the error went away.

Edit: fixed typo.

[gurustron]

@eiriktsarpalis changing code to:

                using (var memoryStream = new MemoryStream())
                {
                    var baseStream = process.StandardOutput.BaseStream;
                    // using (var baseStream = process.StandardOutput.BaseStream)
                    //{
                        var buffer = new byte[4096];
                        int count;
                        while ((count = baseStream.Read(buffer, 0, buffer.Length)) > 0)
                            memoryStream.Write(buffer, 0, count);


                        var end = process.StandardError.ReadToEnd();
                        if (memoryStream.Length == 0L)
                            throw new Exception(end);
                        process.WaitForExit();
                        array = memoryStream.ToArray();
                    //}
                }

still throws the same error for me. And it works in .NET Core 3.1 without any changes.

[eiriktsarpalis]

Apologies, my original comment should have said "StandardInput", namely making the following change:

-                using (var standardInput = process.StandardInput)
-                {
-                    standardInput.WriteLine(html);
-                }
+                process.StandardInput.WriteLine(html);

In general, I would say there are too many using statements in this code. Just applying it to the Process object should suffice.

[gurustron]

@eiriktsarpalis thank you! this works for 5.0. Still wonder what actually changed since 3.1 cause it worked there with disposes =)

Agreed on the disposes - code was written by someone else and I have not read it thoroughly enough.

[eiriktsarpalis]

Great to know! In that case I'm going to close this issue. Thank you for your contribution.

[gurustron]

@eiriktsarpalis

I was running wrong project. Removing the using from standardInput leads to repro project just hanging indefinitely on first baseStream.Read(buffer, 0, buffer.Length) (which kind of makes sense, using Close instead of disposing leads to the same exception).

can you please reopen the issue?

[eiriktsarpalis]

I was seeing the same, however the exact same behaviour happens when I call it with the same arguments directly from the shell, so I'm assuming this is the intended behaviour?

[gurustron]

@eiriktsarpalis

echo "hello world" | wkhtmltopdf -q -B 8 -L 8 -R 8 -T 8 --print-media-type --enable-local-file-access -s Letter -O Portrait - - > test.pdf

works fine for me. The latest to switches (" - -") switch input to stdin and output to stdout.

And again - 3.1 works with closing input, and sample in the docs closes the StandardInput also.

[yakeer]

@eiriktsarpalis thank you! this works for 5.0. Still wonder what actually changed since 3.1 cause it worked there with disposes =)

Agreed on the disposes - code was written by someone else and I have not read it thoroughly enough.

Could you please share your code which made it work?

[tmds]

/dev/stdout is a symlink to /proc/PID/fd/1, which is a socket for the process started from .NET 5 with System.Diagnostics.Process with RedirectStandardInput = true. In .NET Core 3.1, it was a pipe, and Linux syscall open works fine with that; the change was made in #34861.

This is indeed the root cause for the issue.
open for /dev/stdout fails when it is a socket.

@stephentoub this is a regression caused by changing the Process Streams from pipes to sockets.

[stephentoub]

open for /dev/stdout fails when it is a socket.
@stephentoub this is a regression caused by changing the Process Streams from pipes to sockets.

That's unfortunate. How do other platforms deal with this? I've seen others use socketpair as well.

Suggestions? It'd be a shame to need to revert. (Though if we can land #44647, maybe we could switch back to using pipes and use those types instead.)

[tmds]

Other platforms using socketpair will run into the same problem.
Yes, we can use #44647 to switch back to pipes.

[eiriktsarpalis]

FWIW here's a minimal repro highlighting the regression:

#include <stdio.h>
#include <fcntl.h>

int main()
{
    if (open("/dev/stdin", O_RDONLY) < 0)
    {
        fprintf(stderr, "failed to open /dev/stdin.\n");
    }

    if (open("/dev/stdout", O_WRONLY) < 0)
    {
        fprintf(stderr, "failed to open /dev/stdout.\n");
    }
}

static void Main()
{
    var process = new Process
    {
        StartInfo = new ProcessStartInfo
        {
            FileName = "<path to executable>",
            RedirectStandardOutput = true,
            RedirectStandardInput = true,
        }
    };

    process.Start();
    process.WaitForExit();
}

The above prints the error messages in net5.0 but works in netcoreapp3.1.

[TingluoHuang]

👋 I think I am also hitting this in the GitHub Actions runner, I updated the runner from 3.1 to 5.0.
We also redirect both Stdin/out/err in the runner

            _proc.StartInfo.RedirectStandardInput = true;
            _proc.StartInfo.RedirectStandardError = true;
            _proc.StartInfo.RedirectStandardOutput = true;

What i see now is when the runner invoke a process, like bash -e /path/to/script and the script has printenv > /dev/stderr
the script will fail with

/dev/stderr: No such device or address
printenv: write error: Broken pipe

Is there any workaround I can try? or I will need to revert the runner back to 3.1. 😭

[eiriktsarpalis]

I can't think of a good workaround other than changing the code in the child process, which is not feasible in many cases. @stephentoub should we consider reverting #34861 for .NET 5?

[eiriktsarpalis]

cc @adamsitnik @carlossanlop

[stephentoub]

open for /dev/stdout fails when it is a socket.

@tmds, why is this the case? That's expected?

I see the docs for open on Linux do state ENXIO The file is a UNIX domain socket, and presumably socketpair is creating a UDS, which suggests it's "by design" for whatever reason :( (or it's just documenting a bug).

[stephentoub]

should we consider reverting #34861 for .NET 5?

It's unfortunate, but if scenarios involving opening stdin/stdout/stderr via procfs are broken on Linux, we probably should (not a full revert, but a revert of the relevant pieces). If it works as-is on macOS, we could consider reverting just for Linux, too.

The original PR was solving a real problem, though. Hopefully we could fix it again with #44647.

[tmds]

I see the docs for open on Linux do state ENXIO The file is a UNIX domain socket, and presumably socketpair is creating a UDS, which suggests it's "by design" for whatever reason :( (or it's just documenting a bug).

POSIX open documentation mentions it may fail with EOPNOTSUPP when it is a socket.

When we changed to sockets, I expected this to be fine for the handles we provide in the child process. I hadn't considered this way of getting hold of the standard streams.

If it works as-is on macOS

Someone with a mac can check using @eiriktsarpalis reproducer: #46469 (comment).

[jbenden]

Hi,

Just wondering why pipe didn't get modified to use flag O_NONBLOCK, which resolves the original problem? PS: Mac might not have this flag, but you should be able to set it still via fcntl - see example code here.

Thanks,
-Joe

[stephentoub]

Just wondering why pipe didn't get modified to use flag O_NONBLOCK, which resolves the original problem

By "original issue", do you mean what #34861 was solving? Simply making the file descriptor non-blocking doesn't fix that. The hard part is all of the infrastructure necessary to enable efficiently performing the cancelable asynchronous I/O on top of non-blocking file descriptors via a mechanism like epoll on Linux or kqueues on macOS. There's already support for that under System.Net.Sockets.Socket, which is why #34861 changed Process to use socketpair, so that all of that infrastructure could be utilized. As an alternative, #44647 is working to change all of that infrastructure to support arbitrary file descriptors, at least for basic read and write operations, at which point it could be used with a pipe.

[jbenden]

@stephentoub Thank you for the explanation; maybe I need to go to sleep or at least step away from the computer - it didn't sink in when I read all of these reports. But, yup, using select, epoll, or kqueue is the answer. I'm glad a solution is being developed! Thanks!!

[eiriktsarpalis]

If it works as-is on macOS

Someone with a mac can check using @eiriktsarpalis reproducer: #46469 (comment).

I confirm that the particular code doesn't reproduce on macOS.

[eiriktsarpalis]

Fixed in #47461.