Redirect Debug/Warning/... to OutputDebugString on Windows

[mosra]

When building a Win32 GUI app (CMake add_executable(... WIN32 ...), standard output isn't printed to the IDE output view, and one has to use OutputDebugString() because ... Windows is special! So we need to comply with that. Fortunately people learned to use Debug etc. quite extensively, Debug can have scoped output redirection, and there's now Corrade::Main that could do this automagically in the background.

Things to do:

• Rework Debug to be able to output to a callback, not just ostreams
  • Or temporarily implement an UGLY std::basic_stringbuf as seen on https://stackoverflow.com/questions/5248704/how-to-redirect-standard-output-to-output-window-from-visual-studio (but there it seems to be called char-by-char, isn't that slow?!)
  • This also needs the same fix as Android logcat redirection, to make asserts still print to it and not the default std::cerr
• https://docs.microsoft.com/en-us/windows/win32/api/debugapi/nf-debugapi-outputdebugstringw seems to be saying something about WaitForDebugEventEx, the heck is that?!
• Can I use IsDebuggerPresent() to enable this conditionally only when running from an IDE? Does that function return true also when running a Release build / a build without debugger? Ugh.
• Add an ability to open a console for a WIN32 app https://justcheckingonall.wordpress.com/2008/08/29/console-window-win32-app/
• Also expose _CrtSetReportMode(), _CrtSetReportFile() or _CrtSetReportHook() which control whether the assertion dialog box gets open or if the message gets printed to stdout etc

Cc: @hsdk123

[hsdk123]

IsDebuggerPresent()

Currently testing in vs2019 - seems to work fine with RelWithDebInfo, and doesn't return true if no debugger attached.

[mosra]

So it means ... if I run with F5, it says true, if I run with Ctrl-F5, it says false? In both cases I'd like the output to be shown.

[hsdk123]

if I run with F5, it says true, if I run with Ctrl-F5, it says false

Yup, I think that's exactly the case with IsDebuggerPresent() (you could of course do Ctrl-F5 and then reattach the debugger, but I don't think that's what you're implying..?)

[mosra]

I thought I could use IsDebuggerPresent() to check if the app is being run from within Visual Studio. Because I imagine, if the app is run outside of the IDE, people would want the debug messages to be again in the standard output (or not?).

Well, Windows is not my native system, so I'm just guessing -- but I imagine everything sent to OutputDebugString is lost when you don't have the VS debugger attached? Or does it go somewhere else? System log?

[LB--]

OutputDebugString isn't just for GUI apps, it works for console apps too. You can think of Windows apps as always having two different output streams - standard out and debug out - and one of them is only relevant while you have a debugger attached. Debugging-related messages should always go to OutputDebugString and normal input/output (such as piping between parent and child process) should be unaffected by it. Perhaps it should be configurable?

[williamjcm]

I tried here with MinGW-w64 GCC/GDB on Windows, using both CLion and the MSYS2 build of Qt Creator (I don't use VS because it can't be used with GCC and because it's really bloated).

IsDebuggerPresent() returns false when run normally through the IDE, and true when run with GDB attached. This happens both in Debug and Release (I don't use the other CMake build types).

Notes:

• Qt Creator manages to grab the app's output anyway, I didn't need to do anything over there for some reason.
• However, in CLion, I had to open its settings registry (the equivalent of Firefox' about:config screen), and switch run.processes.with.pty to false to see output when running an app normally (output when debugging worked out of the box).

[dween]

Note that you can just always send output to OutputDebugString and, when the application is running outside of Visual Studio, see it using the Sysinternals tool DebugView - see the download page. It was developed outside Microsoft but Microsoft acquired them. I've been using it for years - it's got some handy redirection and filtering capabilities, but you can also run it from the CMD CLI and redirect its output to a file.

[mosra]

OutputDebugString() is apparently implemented via exceptions, so I imagine using it always could have some bad perf implications compared to the regular stdout.

OTOH, thanks to that tweet now I probably know how to catch it even without VS debugger or DebugView attached :)