Implement autoplay-policy conformance for main ShinySDR UI

[ofadam]

When I initially launch ShinySDR via any web browser, the audio is muted although the waterfall display is active and showing signals. I must first click on the "Direct link to audio stream" first, then go back to the dashboard to have working audio. Any idea what may be causing this?

[kpreid]

Which browsers, exactly? In Chrome, I would guess this is due to the newly enabled autoplay restrictions, but that shouldn't trigger on repeated visits to the page.

Is it really the "direct link" that makes it work, or is it just navigating away and back? (Try the other sidebar links, like "ShinySDR Manual".)

Anything in the console?

[ofadam]

I'm using the latest version of Chrome for OSX, but it also is happening on Safari. I tried using other sidebar links and it doesn't free up the audio like clicking on that "direct link."

…

On Wed, Jan 23, 2019 at 6:49 PM Kevin Reid ***@***.***> wrote: Which browsers, exactly? In Chrome, I would guess this is due to the newly enabled autoplay restrictions <https://developers.google.com/web/updates/2017/09/autoplay-policy-changes>, but that shouldn't trigger on repeated visits to the page. Is it really the "direct link" that makes it work, or is it just navigating away and back? (Try the other sidebar links, like "ShinySDR Manual".) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#125 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFP_rIwrZ1u2i5xdQXJyenfgDDtJIJr7ks5vGQMGgaJpZM4aPbfo> .

[ofadam]

There's a strange sawtooth shape present when no audio.

[ofadam]

Firefox works, though!

[kpreid]

That sounds like the AudioContext isn't running (the sawtooth is the audio buffer repeatedly filling up and being thrown out). Please check the console for messages.

[ofadam]

The only console items are the following:

gr::log :INFO: audio source - Audio source arch: alsa

INFO:shinysdr:Flow graph: Rebuilding connections because: switched device

INFO:shinysdr:Flow graph: Rebuilding connections because: added receiver a

[kpreid]

Sorry, I mean the browser console.

[ofadam]

Ah, right.

"The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page." https://developers.google.com/web/updates/2017/09/autoplay-policy-changes#webaudio

[kpreid]

In my experience, that message can appear even if the AudioContext is working normally. Could you please copy the entire console text so I can see the sequence of messages?

(I'll probably need to add some extra debugging to figure this out, but the complete log might have hints.)

[ofadam]

How's this?

main.js:95 Loading plugins…
main.js:95 Connecting to server…
ws-stream.js:68 The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page. https://goo.gl/7K7WLu
main.js:95 Downloading state…
ws-stream.js:252 Streaming using ScriptProcessor 48000 1ch audio and converting to 48000
ws-stream.js:277 The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page. https://goo.gl/7K7WLu
bufferer.js:106 Extreme audio overrun.
widget.js:225 Widget Banner did not handle title attribute
main.js:95 Ready.

[kpreid]

OK, unfortunately, that's essentially the same set of messages I see routinely. I've got some ideas, but it'll probably be a few days before I have them out for you to test.

[ofadam]

Understood! I really appreciate all the attention you've give it already. It's been an impressive piece of software.

[kpreid]

I pushed 446e6dd which adds debug logging.

For me, on reloading, I get this for a log:

main.js:95 Loading plugins…
main.js:95 Connecting to server…
ws-stream.js:72 The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page. https://goo.gl/7K7WLu
ws-stream.js:68 audio playback debug: initial state is suspended
main.js:95 Downloading state…
widget.js:225 Widget Banner did not handle title attribute
main.js:95 Ready.
[Violation] 'requestAnimationFrame' handler took 405ms
ws-stream.js:257 Streaming using AudioWorklet 44100 2ch audio and converting to 44100
ws-stream.js:68 audio playback debug: attempting resume, immediate state is suspended
ws-stream.js:283 The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page. https://goo.gl/7K7WLu
ws-stream.js:68 audio playback debug: attempted resume, immediate state is suspended
ws-stream.js:68 audio playback debug: resume resolved
[Violation] 'requestAnimationFrame' handler took 302ms

So the Chrome autoplay policy seems to be not doing what it's supposed to do, but for me the final result is that playback works anyway. Let me know what your log looks like. No need to copy the stack traces.

[kpreid]

I also notice in your log that the audio stream is running in ScriptProcessor mode rather than AudioWorklet mode. This is surprising. Could you specify exactly which Chrome version you're using?

[ofadam]

I'm using 71.0.3578.98 (64-bit) for OS X.

…

On Fri, Jan 25, 2019 at 11:30 AM Kevin Reid ***@***.***> wrote: I also notice in your log that the audio stream is running in ScriptProcessor mode rather than AudioWorklet mode. This is surprising. Could you specify exactly which Chrome version you're using? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#125 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFP_rE7F22rDk-tPr2tUEl8RfQkTfjqDks5vGz8qgaJpZM4aPbfo> .

[kpreid]

@ofadam Please rerun with the latest ShinySDR revision and report the new log results. No need to include the stack traces.

[ofadam]

It's been a while since I worked on this, but the issue remains even in the latest release. Starting the webapp results in silence with a sawtooth audio wave, which is only fixed after clicking on that direct audio link.

[kpreid]

I've been trying to make progress towards a robust fix for this.

Ignoring the question of any issues with Chrome's actual implementation of the stated autoplay policy, in order to make this work in general we definitely need a client-side state of “muted / not playing” which can then be “unmuted” by the user. I'm making some changes to the UI layout to make a good place for that mute button.

[mathisono]

It would be nice to enable the audio stream for a specific recever. Say i dont audio in the browser, but just on the stream? Then open up a second recever and hear that in the browser. I notced a few bug... But i will wait untill you have pushed the update to report on bugs. Mathison

…

On Mon, May 20, 2019, 9:26 PM Kevin Reid ***@***.***> wrote: I've been trying to make progress towards a robust fix for this. Ignoring the question of any issues with Chrome's actual implementation of the stated autoplay policy, in order to make this work in general we definitely need a client-side state of “muted / not playing” which can then be “unmuted” by the user. I'm making some changes to the UI layout to make a good place for that mute button. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#125?email_source=notifications&email_token=AAEW6LDL37AF2MERJ72NUXDPWN2XLA5CNFSM4GR5W7UKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV2WXLQ#issuecomment-494234542>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEW6LDNEUPKWWRPLTPEVBLPWN2XLANCNFSM4GR5W7UA> .

[kpreid]

Having audio streams with different content is a bigger problem — right now, the server-side signal flow is completely shared, the only exception being the sample rate conversion for audio. Changing that is covered by issue #73.

And if you've observed bugs, please do report them as separate issues. ShinySDR development is mainly limited by having enough free time to think about how to refactor things, and user-visible bugs are often really simple to tackle so it'll actually get things done faster in total if I know about more of them.

[kpreid]

I was doing some refactoring to prepare to have the above-mentioned mute state and I discovered that it wouldn't actually be guaranteed to resume the context on startup if there's supposed to be audio playing. Oops. Commit aeaaabe fixes that, I hope.

@ofadam Give this update try and let me know if it does anything different?

[ofadam]

Hello, I'm back! And still having trouble with audio not playing on initial launch with Chrome. It does with Firefox, somehow. Any further suggestions for ironing this out? I get the sawtooth-like waveform in the audio panel.

[kpreid]

@ofadam Please update ShinySDR to the latest version and post the contents of your console on one of the loads that fails to play audio (no stack traces needed, just the log messages for now).

[ofadam]

Loading plugins…
main.js:95 Connecting to server…
ws-stream.js:72 The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page. https://goo.gl/7K7WLu
ws-stream.js:68 audio playback debug: initial state is suspended
main.js:95 Downloading state…
ws-stream.js:206 Streaming using ScriptProcessor 44100 2ch audio and converting to 44100
ws-stream.js:68 audio playback debug: attempting resume, immediate state is suspended
ws-stream.js:233 The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page. https://goo.gl/7K7WLu
ws-stream.js:68 audio playback debug: attempted resume, immediate state is suspended
widget.js:235 Widget Banner did not handle title attribute
[Violation] 'requestAnimationFrame' handler took 491ms
[Violation] Forced reflow while executing JavaScript took 112ms
[Violation] 'requestAnimationFrame' handler took 60ms
[Violation] Forced reflow while executing JavaScript took 59ms
[Violation] Forced reflow while executing JavaScript took 39ms
bufferer.js:106 Extreme audio overrun.
[Violation] 'requestAnimationFrame' handler took 277ms

[kpreid]

[I've edited your comment to strip out the stack traces which make it hard to read.]

I don't have any good ideas about why this is happening, but apparently the autoplay policy isn't behaving the same for you as others. (Do you routinely do a 'clear browsing data', or use some other sort of privacy tool, maybe?)

The correct solution here is for ShinySDR to implement a UI to prompt for starting playback when it is needed (as the "audio toolbox" pages already do). I don't seem to have it recorded as an issue already, but the plan I had in mind was to add a “mute/pause” feature to the audio UI. Then, if autoplay is not permitted (or, perhaps, if the user prefers it), the page will start in muted state and the user will have to click to unmute it, which should satisfy the autoplay policy without extra dialogs.

[redge76]

Temporary and local solution: #148 (comment)