[Windows] Static with -o to stdout

[whothisbeme]

Hey there,

I'm not certain but I believe this is a bug encountered on the Windows Builds.

The below command on Ubuntu 18.04 will redirect stdout to the testredirect.wav file.

nrsc5 -o - 101.5 0 > _testredirect.wav

This works on Ubuntu 18.04 and will provide a .wav file which opens in VLC and plays without issue.

If you run the same command using a Windows build it will redirect to the testredirect.wav file but the produced file is un-usable.

When attempting to open via Groove Music you encounter an error:

At first it seems you are unable to play this produced file but upon more investigation it may be a header issue. I produced another .wav file using the following command:

nrsc5 -o testsavetofile.wav 101.5 0

This produced a file testsavetofile.wav that does play and works perfectly. I then compared the two files using HxD and it seems the broken file testredirect.wav has a "funky" header which I thought was causing the issues. I created a new file called franken.wav which uses the header from testsavetofile.wav and the data from testredirect.wav. This leaves me with a working .wav file which will play but has an extreme amount of static/noise however you can hear the station in the background.

I originally encountered this bug yesterday when attempting to run the following NodeJS code using the Windows build:

var wav = require('wav');
var Speaker = require('speaker');
var child_process = require('child_process');

var totalData = 0;

var outputWavWriter = new wav.FileWriter('./test.wav', {
    sampleRate: 44100,
    channels: 2
})

var wavreader = new wav.Reader();

const speaker = new Speaker({
    channels: 2,
    bitDepth: 16,
    sampleRate: 44100
})

// the "format" event gets emitted at the end of the WAVE header
wavreader.on('format', function (format) {
    console.log('Format event fired.');
    console.log(format);
    // the WAVE header is stripped from the output of the reader
    wavreader.pipe(speaker);
})

// START SCRIPT
run_script('nrsc5', ['-o', '-', '104.7', '0'], function(output, exitcode) {
    console.log('Process finished.');
    console.log(`Closing code: ${exitcode}`);
    console.log(`Output: ${output}`)
})

console.log(`Node Version: ${process.version}`);

console.log(`Starting NRSC5 NodeJS child_process testing...`);

function run_script(command, args, callback) {
    console.log('Starting Process.');
    var child = child_process.spawn(command, args);

    child.stdout.pipe(outputWavWriter) // Write to wav file
    child.stdout.pipe(wavreader); // Play right to speaker

    child.stdout.on('data', async function (data) {
        console.log(`Got Data: ${data.length}`);
        totalData += data.length;
        console.log(`Total Data: ${totalData}`)
    })

    child.stderr.on('data', function (data) {
        console.log(`stderr: ${data}`);
    })
    
    child.on('close', function(code) {
        callback('datatopassback', code)
    })
}

To run the above code save as index.js and run the following commands within an empty folder using NodeJS v14.15.1:

npm init -y
npm install wav speaker
node index.js

I ran the above NodeJS code using v14.15.1 on Windows first.

It will play the audio over speakers and save a .wav file with the proper headers but the produced audio contains a lot of noise.

This morning I ran the same code but using a PC running Ubuntu 18.04 with the Linux build. The above NodeJS code and also the commands work perfectly under linux. The audio contains no noise and is clear.

I've tested this across two Windows 10 PCs but using the same RTL2832U + R820T SDR receiver.

Is it possible I'm doing something wrong within my setup or maybe this is a known/expected output on Windows?

I've attached a samples.zip which includes the produced files above for troubleshooting.
samples.zip

Also, I've tried the MSYS2 64bit build, Ubuntu Cross-Complied mingw-w64 (64/32bit) builds both contain noise in the audio when using -o - in the command args.

Here are some screenshots of the Hex for a working .wav and the broken noisy .wav:

Thank you so much for this sweet open source software and all of the time invested,
Ryan Merritt

[argilo]

I suspect this might be the cause:

nrsc5/src/main.c

Lines 699 to 701 in b004fca

	#ifdef __MINGW32__
	SetConsoleOutputCP(CP_UTF8);
	#endif

That is needed so that track titles containing non-ASCII characters display correctly.

Perhaps you could try removing that line to see if the problem goes away.

[whothisbeme]

I've tested with the line removed and the noise is still present on 32/64bit Windows builds.

Are you able to duplicate the noise issue I'm encountering?

[argilo]

Does the output file still look the same in your hex editor after that change? It could also be that newlines are problematic.

I don't often use Windows, so I haven't tested this. Any information you can gather would be helpful.

[whothisbeme]

It does still look the same after that change. I removed the line, removed build-win32 folder and then ran support/win-cross-compile 32 which I believe would be a clean build.

I've attached some screenshots of what I'm seeing via the hex editor and it looks fairly unchanged. Also here are the sample
wav files.

more_samples.zip

Original Redirect (Doesn't open, and if header fixed contains noise)

After Line Removal Redirect (Same, Doesn't open, and if header fixed contains noise)

Test Wav Original (No noise)

Test Wav After Line Removal (No noise)

I attempted to look through the source a bit more but so far I've not come across anything obvious however I've never written anything in C.

I was thinking it could be something with the cli.py file and

nrsc5/support/cli.py

Line 91 in bb41b4f

self.wav_output = wave.open(self.args.o, "wb")

However, after further thought I don't think its relevant considering it works perfectly fine on Linux so it must be something else with the way Windows is handling the stdout. Is it possible that it has something to do with binary/text modes when the file is opened on Windows? I don't believe Unix has this distinction between binary and text modes but I could be mistaken.

Thanks again for your help with this 👍

[argilo]

Text mode does sound like a likely candidate.

Could you try adding setmode(fileno(stdout),O_BINARY); right after SetConsoleOutputCP(CP_UTF8);? If that works, we could probably do that conditionally if -o - is specified.

[whothisbeme]

I attempted to use setmode(fileno(stdout),O_BINARY); right after SetConsoleOutputCP(CP_UTF8); but I was greeted by did you mean ‘REG_BINARY‘ error.

I added #include <fcntl.h> & #include <io.h> after #ifdef __MINGW32__ which resolved the error but after testing the produced redirect file is looking the same. It looks as if setmode(fileno(stdout),O_BINARY); changed nothing with the output, but I'm not certain its working?

I'm starting to think we are on the right track. Is it possible we need to call setmode in another location?

Or Is it possibly an issue with:

nrsc5/src/main.c

Line 94 in b004fca

return ao_open_file(ao_driver_id("wav"), name, 1, &sample_format, NULL);

https://xiph.org/ao/doc/ao_open_file.html
https://xiph.org/ao/doc/drivers.html

Maybe the way Windows is treating stdout via libao? Do we need to call set_mode right before we send to stdout? I see it was recently resolved to use Binary mode for file output on Windows but I don't see any mention of stdout other than that it may not be supported to write directly to it, IE: may result in broken files?

From AO Driver Docs:

Writing WAVs to stdout will result in broken files. Use either the raw or the au driver instead.

However it seems the 'wav' driver is used on Linux as well and writing to stdout then redirecting to a file works without issue.

[pclov3r]

On Windows what I've always done to write to wave file on Windows is nrsc5 102.5 0 -o test.wav

Does this solve the issue for you?

I just tested this here with a fresh build and it appears to work fine here exactly like this.

Tho, This isn't stdout but i figured I'd add my two cents in :)

[whothisbeme]

@pclov3r I understand that nrsc5 102.5 0 -o test.wav works perfectly fine on Windows. This issue is specifically related to stdout on Windows and redirecting output to a file.

Thanks for your two cents though.

[whothisbeme]

It seems that the output is being polluted by quite a lot of C2 A0 which I believe is the hex for the UTF8 encoding of a Non-breaking space.

I'm only showing 6 occurrences of C2 A0 in the hex from a working test.wav file but the testredirect.wav file contains 19,680 hits.

[whothisbeme]

@pclov3r Can you possibly try nrsc5 -o - 101.5 0 > testredirect.wav with your build to see if the resulting testredirect.wav file is playable or not?

Thanks for your help and time spent investigating 👍

[pclov3r]

That seems to have made no difference here.

Hum...

[whothisbeme]

That is strange, what version of Windows are you running?

So you are saying that the resulting testredirect.wav file is playable? Would you be able to provide a sample of the output with nrsc5 -o testsavetofile.wav 101.5 0 and nrsc5 -o - 101.5 0 > testredirect.wav to compare?

Thanks again!

[pclov3r]

@whothisbeme

Sorry! I meant to say that it made no difference form the issue your having. Trying to play the file in VLC results in the same static audio that your having. :)

[whothisbeme]

Ah gotcha, thank you for clarifying 👍