Audio Screenshare

[ghost]

Hi there!!

I found this god send of a project somehow and i'm so glad it did, your work is absolutely amazing and I am extremely greatful! Thank you.

Is audio with screen sharing planned too? Sorry i dont mean to demand stuff just curious.

Thank you again for your hard work, you're awesome!

All the best,
pink.

[kyleraykbs]

I second this! Your doing gods work here bud looking forward to the possibility of audio screensharing.

[itsfolf]

Audio is definitely planned! Still trying to figure it out though

[ghost]

amazing to hear! Not sure if this helps at all but the WebCord devs are figuring out how to implement this now. SpacingBat3/WebCord#154

Again, no rush keep up the great work and take care :)

[itsfolf]

amazing to hear! Not sure if this helps at all but the WebCord devs are figuring out how to implement this now. SpacingBat3/WebCord#154

Again, no rush keep up the great work and take care :)

Unfortunately that doesn't help, the issue is that on the desktop app Discord implements it's own native module for voice and video. Capturing the desktop audio on it's own is easy, the issue is that I have to figure out when, where, and what needs to be set in order to make that library transmit the captured audio.

The implementation of the "StartSoundshare" function in Discord's library depends on the platform. On MacOS it's completely different and uses some fancy proprietary libraries; on Windows all the debug symbols are stripped, meaning function names are unavailable and figuring things out is a gruesome task; and on Linux the function is, well, empty.
TL:DR: We're on our own and I haven't had the time to get to it.

[itsfolf]

On unrelated and definitely more welcome news though, I have (very dirtily) implemented hardware encoding through ffmpeg, meaning screen sharing should no longer turn your CPU into a room heater as soon as I can clean that up and push it

[ghost]

I see, thank you for looking into it! No rush with it, keep up the amazing work!!

Awesome to hear, i'll test it out when its pushed :)

[NyaomiDEV]

Glad to see someone is going the extra mile to implement proper screenshare audio on the official client!

+1

[itsfolf]

Small update: I've successfully hooked Discord's device attach and stream functions and grabbed everything I need. If everything goes to plan, all that's left is grabbing a pipewire monitor and somehow piping it into the stream (which is easier said than done because this thing has no fucking documentation, but Telegram gotta have an implementation somewhere)

[NyaomiDEV]

@fuwwy Telegram does not support audio screen sharing at all on Linux, rip. That would be a nice PR to make after this project gets it though.

But this exists: https://github.com/dimtpap/obs-pipewire-audio-capture

[itsfolf]

I've cracked it. The soundshare audiodevice interface receives a EngineAudioTransport class which contains a RecordedDataIsAvailable method. I haven't had a full look at it but all required arguments seem to be available.

[kiosion]

Third-ed, thanks so much for working on this! One question, will this work for audio sharing w/ Xorg/Pipewire as well?

[itsfolf]

Third-ed, thanks so much for working on this! One question, will this work for audio sharing w/ Xorg/Pipewire as well?

Theoretically yes, I just need to check for wayland and disable all the related hooks, but I don't have any xorg system to test atm

[NyaomiDEV]

Is there a Discord server you hang around in, so that we can chat about this modification?

[itsfolf]

Is there a Discord server you hang around in, so that we can chat about this modification?

I have an old server for projects, kinda dead tho https://discord.gg/hDj42dMhn9