rdmd file-descriptor issues with snap packages

[dlang-bugzilla-migration]

Transferred from: https://issues.dlang.org/show_bug.cgi?id=17838

User @WebDrake reported (2017-Sep-18 20:02:36):

rdmd uses spawnProcess to launch the D compiler of choice, and passes a custom File instance to the stdout parameter.

This means that the D compiler has to inherit a file-descriptor from rdmd. For traditional Linux packages, this is not a problem. However, where snap packages are concerned, this creates an issue: when the D compiler is itself a snap package, its confinement rules prevent it from inheriting the file descriptor, and rdmd will exit with an error message:

$ rdmd --compiler=ldc2 hello.d
Failed to flush stdout: Permission denied
Failed: ["ldc2", "-v", "-o-", "hello.d", "-I."]

Some extended discussion of this issue can be found in this thread on the snapcraft forums:
https://forum.snapcraft.io/t/failed-to-flush-stdout-permission-denied/485

This is considered at least partly a bug in snapd, hence I've held off on reporting it as an rdmd issue, but it looks like this is going to be long-lived enough for me to think it might be worth addressing in rdmd itself if possible.

A recent patch to rdmd (f7e6f4e) partially addresses this issue, as it ensures that when rdmd is invoking a D compiler within the same snap package as itself, the confinement rules do not come into play (because the compiler executable is launched directly, instead of indirectly via the wrapper script that the snap package exposes to the host system).

However, this still leaves unaddressed the broader issue of rdmd invoking a compiler provided by a separate snap package (e.g. the ldc2 snap package).

So: can rdmd be either patched or invoked in order to avoid the D compiler it invokes needing to inherit a file descriptor from it? :-)

User @wilzbach responded (2018-Mar-24 07:32:32):

So: can rdmd be either patched or invoked in order to avoid the D compiler it invokes needing to inherit a file descriptor from it? :-)

Yes it can (in theory). With the improvements in 2.079, DMD now supports JSON output via stdout (see e.g. https://run.dlang.io/is/DiSyCl).
That's what #292 was supposed to do.
With the JSON output, no temporary .deps file would be needed and the stdout could be parsed directly, but I think 292 would just instruct dmd to save the JSON output to the temp folder, s.t. no further complications are needed.

So: can rdmd be either patched

Sure, have you tried patching spawnProcess out with
a) execute and saving the output manually
b) a custom implementation that uses real files instead of file pointers?

User @WebDrake responded (2018-Mar-24 13:25:53):

With the improvements in 2.079, DMD now supports JSON output via stdout

That's hardly a viable solution. The problem only arises when one attempts to specify a different D compiler with the --compiler flag, and that compiler is itself snap-packaged.

That would usually mean one is trying to use a different compiler than dmd -- i.e. ldmd2 or gdmd -- neither of which uses the 2.079 frontend yet. And even allowing for that, it's desirable to find a solution that does NOT work only for future compiler releases.

We have to get away from this notion that a release of rdmd only has to work with the dmd it's bundled with, and that it can ignore the question of working properly with other D compilers on the same system, or D compilers using an older frontend version.

This is ultimately an upstream bug that causes problems with the --compiler flag in a narrow context. We shouldn't work around it by causing problems with the --compiler flag in almost EVERY context.

Sure, have you tried patching spawnProcess out with
a) execute and saving the output manually

Yes, but with limited success, probably because I didn't find a way that worked with spawnProcess. Probably the way forward here is to use spawnShell instead, and to request that stdout be piped to a specified file name. That might also address other issues that people have had (e.g. one person mentioned problems on Windows using a dmd.bat script that wrapped the true dmd compiler).