Chromecast (wireless?) speakers disconnecting randomly

[jappish84]

Ok, so this seems to be an issue with wireless devices since I only have this issue when streaming to my Google home speakers directly from owntone to the speakers. The issue is that the stream disconnects at random times, sometimes after 5 minutes, sometimes (the longest I've noticed) after 20 minutes.

I'm running version 28.4 of owntone in a docker container

Relevant log entries that I could find.

[2022-07-07 22:42:36] [DEBUG]     cast: Retransmission to 'Köket' of lost RTCP frame_id 63337, packet_id 65535, bitmask 00
[2022-07-07 22:42:36] [DEBUG]     cast: Retransmission to 'Köket' of lost RTCP frame_id 63338, packet_id 65535, bitmask 00

AFTER SOME TIME

[2022-07-07 22:43:00] [DEBUG]   player: Incomplete read, wanted 3528, got 2664 (samples=333/time=7551020), deficit 3600
[2022-07-07 22:43:00] [DEBUG]     dacp: Update request: client closed connection

The interesting thing is that for testing I setup AirCast (https://github.com/hassio-addons/addon-aircast) in my Home assistant instance that publishes my Chromecast as Airplay speakers and streaming to the "Airplay" chromecast speaker results in no disconnects, works perfect except for the fact that the music is way out of sync of course.

I can provide the full debug log if needed - noticed that some potentially sensitive data is logged and I need some time to go through it thoroughly

[ejurgensen]

I don't really know what might be causing the disconnects, but I think I can rule out that the last two log messages have any relation to the issue. And if you don't have many retransmission log messages I suppose packet loss is an unlikely cause. So my uncertain guess would be underrun, which could happen if the Chromecast speaker has a slightly faster clock than the sender, leading it to consume audio at a higher pace and ultimately emptying its buffer. Unfortunately, I never figured out how to do clock sync with a Chromecast.

I don't recall if it is possible to get logs from Chromecasts, but that would of course be helpful.

AirCast seems to be based on AirConnect, which doesn't use RTP for Chromecast, so it's a bit of different thing.

[jappish84]

I have 8 retransmission logs during the approx. 20 minutes stream before it disconnected, this is what they look like.

[2022-07-07 22:22:36] [DEBUG]     cast: Retransmission to 'Köket' of lost RTCP frame_id 3359, packet_id 65535, bitmask 00
[2022-07-07 22:22:36] [DEBUG]     cast: Retransmission to 'Köket' of lost RTCP frame_id 3360, packet_id 65535, bitmask 00
...
[2022-07-07 22:28:36] [DEBUG]     cast: Retransmission to 'Köket' of lost RTCP frame_id 21350, packet_id 65535, bitmask 00
...
[2022-07-07 22:30:35] [DEBUG]     cast: Retransmission to 'Köket' of lost RTCP frame_id 27351, packet_id 65535, bitmask 00
...
[2022-07-07 22:36:36] [DEBUG]     cast: Retransmission to 'Köket' of lost RTCP frame_id 45356, packet_id 65535, bitmask 00
[2022-07-07 22:36:36] [DEBUG]     cast: Retransmission to 'Köket' of lost RTCP frame_id 45357, packet_id 65535, bitmask 00
...
[2022-07-07 22:42:36] [DEBUG]     cast: Retransmission to 'Köket' of lost RTCP frame_id 63337, packet_id 65535, bitmask 00
[2022-07-07 22:42:36] [DEBUG]     cast: Retransmission to 'Köket' of lost RTCP frame_id 63338, packet_id 65535, bitmask 00
...

Could it be that this issue got introduced in an update a while back because I think that I didn't have this issue in the (pretty?) early days of forked-daapd a few years back. I should have reported the issue as soon as I noticed it, but unfortunately didn't.

[ejurgensen]

Older versions used the same non-RTP method as AirConnect, so the issue probably came with the switch to RTP.

[frankusb]

I can confirm this, my playback lasted from 8:58 to 9:19 on a JBL Link 500 (Chromecast). Nothing in my log.

[ejurgensen]

Tried if I could reproduce, but my Google Home Mini has played for an hour now. Of course, if it is clock related different results would be expected. I will try to starve it on purpose and see how it reacts.

[jappish84]

Really happy you are looking in to it, would be awesome if it could be resolved. I'd be glad to contribute with testing if needed

[ejurgensen]

Sounds good. Does that mean you can build from source if I make a branch with a modification?

[jappish84]

I think I can manage that, yeah. I've done it a few times on other projects and I'm familiar with VEs for testing so should be good

[ejurgensen]

I had a closer look at this, and found it hard to come up even with modifications that would help diagnose the issue. So I don't have anything you can test. The casting that OwnTone does is more complicated than I remembered.

[jappish84]

Thanks for taking a closer look. Appreciate the work.
Since I've posted this issue I've also had one speaker play for more than an hour, not sure that has ever happened before since the switch.

How in sync are your wireless chromecast devices with the airplay devices @ejurgensen? Because my wireless CC devices are most often slightly out of sync with airplay devices. I think ahead, but I'll have to check and your theory above with CC devices starving seems to make sense

[ejurgensen]

How in sync are your wireless chromecast devices with the airplay devices?

Since I never found out how to do clock sync with a CC, the only sync I could do was controlling the delay by which playback is started. I set this to match to match my Airplay speakers, so for me the sync is quite good when playback starts. There is a bit of drift, however. And actually that gives me an idea on how you can check if underrun is the cause of your issue. If you try starting a playback session with both Airplay and CC, and you observe that the CC is gradually moving more and more out of sync, then it is an indication of possible underrun or overrun. If the CC is getting more and more ahead I suppose it equals an underrun in the making.

You can use the "offset_ms" setting to control when your CC starts. You can set it in the config file like this:

chromecast "My Speaker" {
offset_ms = 123
}

[jappish84]

The drift you are describing is absolutely happening with my setup where the CC always seems to want to drift ahead of the Airplay speakers. I'll pay some extra attention the coming days to see if CC always wanna drift ahead or if the fall behind the Airplay speakers aswell.

Both start synced so I don't think I need to mess with the offset_ms

[jappish84]

After some more testing I can confirm that the wireless CC devices always drift ahead of the Airplay devices. This must mean that they are starved after a while like you mentioned earlier

[ejurgensen]

Were you able to assess how much ahead the CC is when the disconnect happens? I think starvation would be expected to happen at around 1-2 sec difference.

I'm not sure what can be done about this. As mentioned, I don't know how to tell the CC to slow down/sync its clock.

[jappish84]

Hard to tell exactly but it's max 1sec
My best guess would be around 0.5s before it disconnects.

It just hit me that Google speaker delay offset can be set in the g home app, I'll try and mess with that when they drift just to see what happens

[ejurgensen]

I can add that Chromium uses the same way of streaming to a CC if you set it to play local content - so it's RTP like OwnTone. Perhaps something can be learnt from experimenting with that, but I'm not sure exactly what experiments would be useful.

[dashdsrdash]

I have had this happen in the past, irregularly, and had it clear up after I improved my wifi network by relocating access points and selecting low-noise channels.

[RomeoNotaLoka]

I try to keep my Google Home speaker in sync with a Sonos One. My GH disconnects about every 22 mins. For now, my solution to keep GH playing is by making an automation in HomeAssistant to reconnect GH if Sonos one is still playing.

[jappish84]

I had a similar workaround using homeassistant to re-enable streaming for active CC clients but just ended up not using my G speakers since they always drifted away from the sync.

Not familiar with the RTP and non-RTP method used by forked-daapd in the early days, but I'm guessing RTP has some advantages over the previously used method for streaming and switching back to the old method would be a bad idea?