Redirection of stdout/stderr in pybullet

[taiya]

Would be great if pybullet would not (unconditionally) pollute the standard streams. Had to resort to this hack to silence the pybullet build time: Oct 8 2020 00:10:46 that was sent to stdout upon import pybullet as pb

# hide diagnostic output
with open(os.devnull, 'w') as devnull:
  # suppress stdout
  orig_stdout_fno = os.dup(sys.stdout.fileno())
  os.dup2(devnull.fileno(), 1)
  # suppress stderr
  orig_stderr_fno = os.dup(sys.stderr.fileno())
  os.dup2(devnull.fileno(), 2)
  import pybullet as pb
  os.fsync(devnull.fileno())
  os.dup2(orig_stdout_fno, 1)  # restore stdout
  os.dup2(orig_stderr_fno, 2)  # restore stderr

This was the only solution that worked out of what listed here (with an extra fsync needed):
https://stackoverflow.com/questions/4178614/suppressing-output-of-module-calling-outside-library
https://stackoverflow.com/questions/42952623/stop-python-module-from-printing
https://stackoverflow.com/questions/5081657/how-do-i-prevent-a-c-shared-library-to-print-on-stdout-in-python/14797594#14797594

Would have been much nicer if the library would either

1. respect the python sys.stdout / sys.stderr specification
2. allow to specify a different stream with a function

[taiya]

Interestingly the fragment above only works when both stdout and stderr are redirected.
It seems pybullet only pollutes stdout, but if I modify the fragment above:

# hide diagnostic output
with open(os.devnull, 'w') as devnull:
  # suppress stdout
  orig_stdout_fno = os.dup(sys.stdout.fileno())
  os.dup2(devnull.fileno(), 1)
  import pybullet as pb
  os.fsync(devnull.fileno())
  os.dup2(orig_stdout_fno, 1)  # restore stdout

annoyingly the results becomes:

Traceback (most recent call last):
  File "/workspaces/kubric/helloworld.py", line 44, in <module>
    os.fsync(devnull.fileno())
OSError: [Errno 22] Invalid argument

(and of course using sys.stdout.flush() instead of os.fsync does not seem to work...)

[taiya]

A more cleaned up version, including a potential leak prevention caused by dup.
Not exactly sure how safe and cross-platform _flush_c_stream is...

class RedirectStream(object):

  @staticmethod
  def _flush_c_stream(stream):
    streamname = stream.name[1:-1]
    libc = ctypes.CDLL(None)
    libc.fflush(ctypes.c_void_p.in_dll(libc, streamname))

  def __init__(self, stream=sys.stdout, file=os.devnull):
    self.stream = stream
    self.file = file

  def __enter__(self):
    self.stream.flush()  # ensures python stream unaffected 
    self.fd = open(self.file, "w+")
    self.dup_stream = os.dup(self.stream.fileno())
    os.dup2(self.fd.fileno(), self.stream.fileno()) # replaces stream
  
  def __exit__(self, type, value, traceback):
    RedirectStream._flush_c_stream(self.stream)  # ensures C stream buffer empty
    os.dup2(self.dup_stream, self.stream.fileno()) # restores stream
    os.close(self.dup_stream)
    self.fd.close()

with RedirectStream(sys.stdout):
  import pybullet as pb

Anyhow, TL;DR dealing with this externally to pybullet is really painful, would be great if we did not have to?

[JakobThumm]

This caused pytest to run into an OSError: [Errno 22] Invalid argument when collecting the tests. I guess pytest uses a different IO stream. I'm not sure it's exactly what you were looking for, but here is an alternative that works together with pytest:

from contextlib import redirect_stdout

with open(os.devnull, 'w') as f, redirect_stdout(f):
    import pybullet as pb

[erwincoumans]

Thanks, didn't realize anyone cares about the output messages. Some of the printfs go through a common C++ function, so it would be possible to clean up the code (and some people don't like the build time output etc).
I'll leave it open for now, hope to get to this, since the project severely lacks resources.

[ZhiquanW]

I hope that pybullet can provide a parameter to control the output of these annoying messages as the outputs can be a mess while running with multiprocess.

[boris-il-forte]

@erwincoumans, is there any news about this feature? It is particularly bothering when using PyBullet with Joblib in Python, as the message is displayed for any new thread importing the library.
It would be nice if an environment variable could suppress this print.

[taiya]

I have hacked this out within Kubric. Please see the redirect magic therein: google-research/kubric#96

…

On Wed, Jun 9, 2021 at 5:23 AM Davide Tateo ***@***.***> wrote: @erwincoumans <https://github.com/erwincoumans>, is there any news about this feature? It is particularly bothering when using PyBullet with Joblib in Python, as the message is displayed for any new thread importing the library. It would be nice if an environment variable could suppress this print. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3441 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABCF4POQCRGQBY25WPUJWHTTR5MN3ANCNFSM46L5Z5AQ> .