mpd 0.24 (0.23.15) will not play via icecast as a drop-in replacement f/0.23.14 - mpd hangs until SIGTERM

[vargobj]

Bug report

Describe the bug

mpd 0.23.14 was installed via ubuntu repository to /bin/mpd; /usr/bin/mpd; later renamed mpd.23
mpd 0.23.15-1177-g1efb9d41d was compiled and installed at /usr/local/bin/mpd; later renamed mpd.24
/etc/mpd.conf contains audio_outputs to icecast2 that are set as enabled "no"
/etc/mpd.conf contains audio_output to null, enabled (5)
/etc/mpd.conf was copied to /etc/mpd.23.conf for testing purposes as described below

NB throughout this discussion I will be using 0.24, 0.23.15, and 0.23.15.177-g1efb9d41d interchangeably;
0.23 will be used to refer to 0.23.14 only.

Holding everything else the same, when /usr/local/lib/systemd/system/mpd.service is modified to change from
ExecStart=/usr/bin/mpd.23 --systemd
to
'ExecStart=/usr/local/bin/mpd.24 --systemd'

$ systemctl start mpd
mpd will run, but when the icecast output is enabled, it hangs. mpc can no longer communicate with it and systemctl cannot stop/restart mpd; it must be killed via SIGTERM (i.e,. kill -9; I don't think it will exit without SIGTERM)

This was the state of affairs prior to 2024.04.19.
Kaliko suggested running via the cli so I changed to a minimal mpd.24.conf and an included mpd.24.outputs.conf and I've separated out the database file, i.e., so the database is not overwritten each time the version changes. This was done at the systemd script level so that the DBs don't have to get rebuilt. Snippet of /usr/local/lib/systemd/system/mpd.service:

⋮
# DO NOT SWITCH 
# VERSIONS OF MPD 
# WITHOUT DEALING 
# WITH THE DB FILE!
#
# The different config files below address two things:
# 1) The db files are different so that switching versions
#    does not require they be rebuild
# 2) The port for mpd 0.24 is now 6624 instead of 6600 so
#    the two versions of mpd can be running simultaneously
#
#ExecStart=/usr/local/bin/mpd.24 --systemd /music/.share/mpd/mpd.24.conf
ExecStart=/usr/bin/mpd.23 --systemd /etc/mpd.23.conf
⋮

Fun from the cli, it was invoked by

runuser -g audio -G media -l mpd -c 'mpd.24 --stderr --no-daemon --verbose /music/.share/mpd/mpd.24.conf'

The mpd.24.conf files are below, but regardless, mpd still hangs when icecast2 is enabled.

Expected Behavior

Expect 0.23.15 to work as a drop in replacement for 0.23.14, i.e., such that the icecast streaming is operative and mpd does not hang.
Icecast has no issue with 0.23.14.

Actual Behavior

mpd hangs for more than 10 minutes and the user kills it.

Version

Linux 6.8.4-060804-generic #202404041833 SMP PREEMPT_DYNAMIC Thu Apr 4 18:46:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

mpd.24:

# mpd.24 --version
Music Player Daemon 0.24 (v0.23.15-1177-g1efb9d41d)
Copyright 2003-2007 Warren Dukes <warren.dukes@gmail.com>
Copyright 2008-2021 Max Kellermann <max.kellermann@gmail.com>
This is free software; see the source for copying conditions.  There is NO
warranty; not even MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Database plugins:
 simple proxy upnp

Storage plugins:
 local udisks nfs curl

Neighbor plugins:
 upnp udisks

Decoder plugins:
 [mad] mp3 mp2
 [mpg123] mp3
 [vorbis] ogg oga
 [oggflac] ogg oga
 [flac] flac
 [opus] opus ogg oga
 [sndfile] wav aiff aif au snd paf iff svx sf voc w64 pvf xi htk caf sd2
 [audiofile] wav au aiff aif
 [dsdiff] dff
 [dsf] dsf
 [faad] aac
 [mpcdec] mpc
 [wavpack] wv
 [openmpt] mptm mod s3m xm it 669 amf ams c67 dbm digi dmf dsm dtm far imf ice j2b m15 mdl med mms mt2 mtm nst okt plm psm pt36 ptm sfx sfx2 st26 stk stm stp ult wow gdm mo3 oxm umx xpk ppm mmcmp
 [modplug] 669 amf ams dbm dfm dsm far it med mdl mod mtm mt2 okt s3m stm ult umx xm
 [mikmod] amf dsm far gdm imf it med mod mtm s3m stm stx ult uni xm
 [sidplay] sid mus str prg P00
 [fluidsynth] mid
 [adplug] amd d00 hsc laa rad raw sa2
 [gme] ay gbs gym hes kss nsf nsfe rsn sap spc vgm vgz
 [ffmpeg] 264 265 266 302 3g2 3gp 4xm 669 722 aa aa3 aac aax ac3 ac4 ace acm act adf adp ads adx aea afc aiff aix al alias_pix alp amf amr amrnb amrwb ams anm ans apc ape apl apm apng aptx aptxhd aqt argo_asf argo_brp argo_cvg art asc asf asf_o ass ast au avc avi avif avr avs avs2 avs3 bcstm bethsoftvid bfi bfstm bin bink binka bit bitpacked bmp_pipe bmv boa bonk brender_pix brstm c2 c93 caf cdata cdg cdxl cgi cif cine codec2raw concat cri_pipe dash dat data daud dav dbm dds_pipe dfa dff dfpwm dif digi dirac diz dmf dnxhd dpx_pipe dsf dsicin dsm dss dst dtk dtm dts dtshd dv dvbsub dvbtxt dxa ea eac3 ec3 evc exr_pipe f32be f32le f4v f64be f64le fap far ffmetadata film_cpk fits flac flic flm flv frm fsb fwse g722 g723_1 g726 g726le g729 gdm gdv gem_pipe genh gif gif_pipe gsm gxf h261 h263 h264 h265 h266 h26l hca hcom hdr_pipe hevc hls hnm ice ico idcin idf idx iff ifv ilbc image2 image2pipe imf imx ipmovie ipu ircam ism isma ismv iss it iv8 ivf ivr j2b j2k j2k_pipe jacosub jpeg_pipe jpegls_pipe jpegxl_pipe jv jxl kux kvag laf libgme lmlm4 loas lrc lvf lxf m15 m2a m4a m4b m4v mac mca mcc mdl med mgsts microdvd mj2 mjpeg mjpg mk3d mka mks mkv mlp mlv mm mmcmp mmf mms mo3 mod mods moflex mov mp2 mp3 mp4 mpa mpc mpc8 mpeg mpegts mpegtsraw mpegvideo mpl2 mpo mptm msbc msf msnwctcp msp mt2 mtaf mtm mtv musx mv mvi mxf mxg nfo nist nsp nst nsv nut nuv obu ogg okt oma omg osq paf pam_pipe pbm_pipe pcx_pipe pdv pfm_pipe pgm_pipe pgmyuv_pipe pgx_pipe phm_pipe photocd_pipe pictor_pipe pjs plm pmp png_pipe pp_bnk ppm ppm_pipe psd_pipe psm psp psxstr pt36 ptm pva pvf qcif qcp qdraw_pipe qoi_pipe r3d rco rcv rgb rka rl2 rm roq rpl rsd rso rt rtp rtsp s16be s24be s24le s32be s32le s337m s3m sami sap sb sbc sbg scc scd sdns sdp sdr2 sds sdx ser sf sfx sfx2 sga sgi_pipe shn sln smi smk smush sol son sox spdif sph srt ss2 st26 stk stl stm stp str sub sunrast_pipe sup svag svg_pipe svs sw swf tak tco tedcaptions thd thp tiertexseq tiff_pipe tmv tta txd txt ty ty+ u16be u24be u24le u32be u32le ub ul ult umx usm uw v v210 vag vb vbn_pipe vc1 vidc viv vividas vmd voc vpk vqe vqf vql vt vtt vvc w64 wa wav way wc3movie webm webm_dash_manifest webp_pipe wow wsaud wsd wsvqa wtv wv wve xa xbin xbm_pipe xl xm xmd xmv xpk xpm_pipe xvag xwd_pipe xwma y4m yop yuv yuv10 rtp:// rtsp:// rtsps://
 [pcm]

Filters:
 libsamplerate soxr

Tag plugins:
 id3tag

Output plugins:
 shout null fifo sndio pipe alsa ao openal pipewire pulse jack httpd snapcast recorder

Encoder plugins:
 null vorbis opus lame twolame wave flac shine

Archive plugins:
 [bz2] bz2
 [zzip] zip
 [iso] iso

Input plugins:
 file io_uring archive alsa qobuz curl ffmpeg nfs mms cdio_paranoia

Paylist plugins:
 extm3u m3u pls xspf asx rss soundcloud flac cue embcue

Protocols:
 file:// alsa:// cdda:// ftp:// ftps:// gopher:// hls+http:// hls+https:// http:// https:// mms:// mmsh:// mmst:// mmsu:// nfs:// qobuz:// rtmp:// rtmpe:// rtmps:// rtmpt:// rtmpte:// rtmpts:// rtp:// rtsp:// rtsps:// scp:// sftp:// smb:// srtp://

Other features:
 avahi dbus udisks epoll icu inotify ipv6 systemd tcp un

mpd.23:

 mpd.23 --version
Music Player Daemon 0.23.14 (0.23.14)
Copyright 2003-2007 Warren Dukes <warren.dukes@gmail.com>
Copyright 2008-2021 Max Kellermann <max.kellermann@gmail.com>
This is free software; see the source for copying conditions.  There is NO
warranty; not even MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Database plugins:
 simple proxy upnp

Storage plugins:
 local smbclient udisks nfs curl

Neighbor plugins:
 smbclient upnp udisks

Decoders plugins:
 [mad] mp3 mp2
 [mpg123] mp3
 [vorbis] ogg oga
 [oggflac] ogg oga
 [flac] flac
 [opus] opus ogg oga
 [sndfile] wav aiff aif au snd paf iff svx sf voc w64 pvf xi htk caf sd2
 [dsdiff] dff
 [dsf] dsf
 [hybrid_dsd] m4a
 [faad] aac
 [mpcdec] mpc
 [wavpack] wv
 [openmpt] mptm mod s3m xm it 669 amf ams c67 dbm digi dmf dsm dtm far imf ice j2b m15 mdl med mms mt2 mtm nst okt plm psm pt36 ptm sfx sfx2 st26 stk stm stp ult wow gdm mo3 oxm umx xpk ppm mmcmp
 [modplug] 669 amf ams dbm dfm dsm far it med mdl mod mtm mt2 okt s3m stm ult umx xm
 [mikmod] amf dsm far gdm imf it med mod mtm s3m stm stx ult uni xm
 [sidplay] sid mus str prg P00
 [wildmidi] mid
 [fluidsynth] mid
 [adplug] amd d00 hsc laa rad raw sa2
 [gme] ay gbs gym hes kss nsf nsfe rsn sap spc vgm vgz
 [ffmpeg] 16sv 3g2 3gp 4xm 8svx aa3 aac ac3 adx afc aif aifc aiff al alaw amr anim apc ape asf atrac au aud avi avm2 avs bap bfi c93 cak cin cmv cpk daud dct divx dts dv dvd dxa eac3 film flac flc fli fll flx flv g726 gsm gxf iss m1v m2v m2t m2ts m4a m4b m4v mad mj2 mjpeg mjpg mka mkv mlp mm mmf mov mp+ mp1 mp2 mp3 mp4 mpc mpeg mpg mpga mpp mpu mve mvi mxf nc nsv nut nuv oga ogm ogv ogx oma ogg omg opus psp pva qcp qt r3d ra ram rl2 rm rmvb roq rpl rvc shn smk snd sol son spx str swf tak tgi tgq tgv thp ts tsp tta xa xvid uv uv2 vb vid vob voc vp6 vmd wav webm wma wmv wsaud wsvga wv wve rtp:// rtsp:// rtsps://
 [pcm]

Filters:
 libsamplerate soxr

Tag plugins:
 id3tag

Output plugins:
 shout null fifo sndio pipe alsa ao oss openal pipewire pulse jack httpd snapcast recorder

Encoder plugins:
 null vorbis opus lame wave flac

Archive plugins:
 [bz2] bz2
 [zzip] zip
 [iso] iso

Input plugins:
 file io_uring archive alsa curl ffmpeg smbclient nfs mms cdio_paranoia

Playlist plugins:
 extm3u m3u pls xspf asx rss soundcloud flac cue embcue

Protocols:
 file:// alsa:// cdda:// ftp:// ftps:// gopher:// hls+http:// hls+https:// http:// https:// mms:// mmsh:// mmst:// mmsu:// nfs:// rtmp:// rtmpe:// rtmps:// rtmpt:// rtmpte:// rtmpts:// rtp:// rtsp:// rtsps:// scp:// sftp:// smb:// srtp://

Other features:
 avahi dbus udisks epoll icu inotify ipv6 systemd tcp un

Configuration

mpd.24.conf

db_file                                 "/music/.share/mpd/database.24"
log_file                                "/var/log/mpd/mpd.24.log"
sticker_file                            "/music/.share/mpd/sticker.24.sql"
# If you use systemd, do not configure a pid_file.
pid_file                                "/music/.share/mpd/mpd.pid"
music_directory                         "/music"
playlist_directory                      "/music/mpd playlists"
state_file                              "/music/.share/mpd/state.24"
restore_paused                          "no"
user                                    "mpd"
#group                                  "media"
group                                   "audio"
port                                    "6624"
#Log levels:
#error:                                 errors
#warning:                               warnings
#notice:                                interesting informational messages
#info:                                  unimportant informational messages
#verbose:                               debug messages (for developers and for troubleshooting)
log_level                               "default"
follow_outside_symlinks                 "yes"
follow_inside_symlinks                  "yes"
zeroconf_enabled                        "no"
zeroconf_name                           "Music Player @ %h"
filesystem_charset                      "UTF-8"
save_absolute_paths_in_playlists        "no"
auto_update                             "no"

# Limit the depth of the directories being watched, 
# 0 means only watch the music directory itself. 
# There is no limit by default
#auto_update_depth                      "3"

include "/music/.share/mpd/mpd.24.outputs.conf"

mpd.24.outputs.conf:

audio_output {
	type "null"
	name "null output"
}

# An example of a shout output (for streaming to Icecast):
#
audio_output {
	enabled		"no"
	type            "shout"
	encoder         "vorbis"                # optional
	name            "My Shout Stream"
	host            "localhost"
	port            "8000"
	mount           "/mpd.ogg"
	password        "< - redacted - >"
#	quality         "5.0"			# Failed to configure output in line 8; 
	bitrate         "128"			# quality and bitrate are both defined
	format          "44100:16:1"		
##	protocol        "icecast2"              # optional
##	user            "source"                # optional
##	description     "My Stream Description" # optional
##	url             "http://example.com"    # optional
##      genre           "jazz"                  # optional
##      public          "no"                    # optional
##      timeout         "2"                     # optional
##      mixer_type      "software"              # optional
}

Log

# runuser -g audio -l mpd -c 'mpd.24 --stderr --no-daemon --verbose /library/music/.share/mpd/mpd.24.conf'
runuser: warning: cannot change directory to /home/mpd: No such file or directory
config_file: loading file /library/music/.share/mpd/mpd.24.conf
config_file: loading file /library/music/.share/mpd/mpd.24.outputs.conf
server_socket: bind to '0.0.0.0:6624' failed (continuing anyway, because binding to '[::]:6624' succeeded): Failed to bind socket: Address already in use
libsamplerate: libsamplerate converter 'Fastest Sinc Interpolator'
vorbis: Xiph.Org libVorbis 1.3.7
opus: libopus 1.4
sndfile: libsndfile-1.2.2
adplug: adplug 2.3.3
simple_db: reading DB
input: Input plugin 'qobuz' is not configured: No Qobuz app_id configured
curl: version 8.5.0
curl: with GnuTLS/3.8.3
event: RTIOThread could not get realtime scheduling, continuing anyway: sched_setscheduler failed: Operation not permitted
state_file: Loading state file /library/music/.share/mpd/state.24
playlist: queue song 5:"Grateful Dead/Grateful Dead -- 1966-05-19: Avalon Ballroom, San Francisco, CA (1966) (mp3)/Grateful Dead -- 06 - I Know You Rider.mp3"
decoder_thread: probing plugin mad
mad: detected LAME version 3.96 ("LAME3.96r")
mad: LAME peak found: 0
mad: encoder delay is 576, encoder padding is 1728
decoder: audio_format=44100:24:2, seekable=true
output: OutputThread could not get realtime scheduling, continuing anyway: sched_setscheduler failed: Operation not permitted
output: opened "null output" (null) audio_format=44100:24:2
client: [0] opened from [::1]:56188
client: [0] process command list
client: process command "status"
client: command returned 0
client: process command "currentsong"
client: command returned 0
client: [0] process command list returned 0
client: [0] closed
client: [1] opened from [::1]:56198
client: [1] process command "next"
playlist: play 5:"Grateful Dead/Grateful Dead -- 1966-05-19: Avalon Ballroom, San Francisco, CA (1966) (mp3)/Grateful Dead -- 06 - I Know You Rider.mp3"
decoder_thread: probing plugin mad
mad: detected LAME version 3.96 ("LAME3.96r")
mad: LAME peak found: 0
mad: encoder delay is 576, encoder padding is 1728
decoder: audio_format=44100:24:2, seekable=true
client: [1] command returned 0
playlist: queue song 6:"Grateful Dead/Grateful Dead -- 1966-05-19: Avalon Ballroom, San Francisco, CA (1966) (mp3)/Grateful Dead -- 07 - I Know It's a Sin.mp3"
client: [1] process command list
client: process command "status"
client: command returned 0
client: process command "currentsong"
client: command returned 0
client: [1] process command list returned 0
client: [1] closed
client: [2] opened from [::1]:56208
client: [2] process command "outputs"
client: [2] command returned 0
client: [2] closed
client: [3] opened from [::1]:56216
client: [3] process command list
client: process command "enableoutput "1""
output: OutputThread could not get realtime scheduling, continuing anyway: sched_setscheduler failed: Operation not permitted
client: command returned 0
client: [3] process command list returned 0
client: [3] process command "outputs"
client: [3] command returned 0
client: [3] closed
output: opened "My Shout Stream" (shout) audio_format=44100:f:1
output: converting in=44100:24:2 -> f=44100:24:2 -> out=44100:f:1
client: [4] opened from [::1]:35700
client: [4] process command list
client: process command "status"
client: command returned 0
client: process command "currentsong"
client: command returned 0
client: [4] process command list returned 0
client: [4] closed
client: [5] opened from [::1]:55978
client: [5] process command "next"
playlist: play 6:"Grateful Dead/Grateful Dead -- 1966-05-19: Avalon Ballroom, San Francisco, CA (1966) (mp3)/Grateful Dead -- 07 - I Know It's a Sin.mp3"
Killed

corresponding commands:

egt 10.12 am $ mpc24 status; mpc24 next; mpc24 outputs; mpc24 enable 2; mpc status; mpc next
Grateful Dead - Viola Lee Blues
[playing] #5/3471   0:59/7:44 (12%)
volume: n/a   repeat: off   random: off   single: off   consume: off
Grateful Dead - I Know You Rider
[playing] #6/3471   0:00/2:51 (0%)
volume: n/a   repeat: off   random: off   single: off   consume: off
Output 1 (null output) is enabled
Output 2 (My Shout Stream) is disabled
Output 1 (null output) is enabled
Output 2 (My Shout Stream) is enabled
Grateful Dead - I Know You Rider
[playing] #6/3471   0:00/2:51 (0%)
volume: n/a   repeat: off   random: off   single: off   consume: off
MPD error: Timeout
edt 10.13 am $
edt 10.23 am $ sudo pkill -9 mpd.24
edt 10.24 am $

I don't know that mpd has "crashed" per se, as I need to kill it. I can try to run a back trace if it is something that will help.

[MaxKellermann]

Post a backtrace of all threads when MPD is stuck.