mod_http_api explicitly required

[githubDante]

Hi,

Just upgraded a Debian (buster) server running ejabberd 21.01-2~bpo10+1 to bullseye, where the ejabberd package looks the same, according to its version (21.01-2). After the upgrade, however, it was unable to start due to the following:

2022-02-12 20:40:30.795881+00:00 [critical] <0.119.0>@ejabberd_app:start/2:68 Failed to start ejabberd application: {error, {shutdown, {failed_to_start_child, ejabberd_commands, {aborted, {bad_type, {ejabberd_commands, gen_html_doc_for_commands, [documentation], "Generates html documentation for ejabberd_commands", [],0,1,ejabberd_commands_doc, generate_html_output, [{file,binary}, {regexp,binary}, {examples,binary}], restricted,[], {res,rescode}, [], ["Path to file where generated documentation should be stored", "Regexp matching names of commands or modules that will be included inside generated document", "Comma separated list of languages (chosen from java, perl, xmlrpc, json)that will have example invocation include in markdown document"], "0 if command failed, 1 when succeeded", ["/home/me/docs/api.html", "mod_admin","java,json"], ok}}}}}}

The upgrade from bullseye-backports (21.12-1~bpo11+1) didn't help either. The node has started normally when mod_http_api: {} has been added to the modules section.
Is it possible this requirement to be included in the upgrade notes.

[badlop]

Let's see if I understood your setup:

• You were running ejabberd (21.01-2~bpo10+1), and it worked correctly
• Then you upgraded to ejabberd (21.01-2)
• It crashed at start reporting {error, {shutdown, {failed_to_start_child, ejabberd_commands,...
• Then you added mod_http_api to modules section in ejabberd.yml, and it starts correctly

You say that ejabberd.yml in your old configuration didn't have mod_http_api enabled, but... I see ejabberd.yml.example in ejabberd (21.01-2~bpo10+1) contains mod_http_api in the modules section.

I installed ejabberd (21.01-2), works correctly, then I remove mod_http_api in modules, and ejabberd starts correctly too.

And, of course, there's the doubt: what is the relation between ejabberd_commands and mod_http_api?

So, there's something weird here. Maybe your old configuration had some additional lines, that may be relevant to the problem?

If you remove the mod_http_api line in modules, does ejabberd crash at start again? I mean, is that reproducible?

[githubDante]

Yes, that's the setup, but the initial version of the config is from older custom installation. Probably from version 17.x (if I remember correctly), with several additions added over the years. The relation to mod_http_api was a wild guess, as the command "gen_html_doc_for_commands" can be found in the API Reference.

I just restarted the node with commented mod_http_api and no errors were reported, which is even weirder.

[badlop]

Some points:

A) gen_html_doc_for_commands is the first command defined in ejabberd_commands.erl, so probably it's the first one in the non-sorted list of commands. In that case, the problem was not about that specific command, but to the whole commands code.

B) The ejabberd 21.12-1 package in debian includes a postinst script that runs "ejabberdctl registered_users" and then, maybe, "register".

Maybe your problem was unrelated to mod_http_api, but quite simply that ejabberd needed some restarts after the version upgrade?

Did you restart ejabberd several times without success, and then tried enabling mod_http_api with success?

[githubDante]

Yes it was restarted several times with version 21.01-2 and 21.12-1~bpo11+1

root@vps1-dante:~# egrep "Failed.*Jabber" /var/log/syslog.1
Feb 12 20:24:13 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server
Feb 12 20:26:05 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:27:21 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:28:37 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:29:53 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:31:09 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:32:25 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:33:41 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:34:57 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:36:13 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:37:29 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:38:45 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:40:02 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:41:19 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:42:27 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:43:00 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:43:34 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:44:07 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:44:39 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:45:12 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:45:45 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:46:17 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:46:50 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:47:23 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:48:22 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:48:55 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:49:29 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
Feb 12 20:50:02 vps1-dante systemd[1]: Failed to start A distributed, fault-tolerant Jabber/XMPP server.
root@vps1-dante:~#
root@vps1-dante:/tmp# egrep -B1 "install ejabberd" /var/log/apt/history.log
Start-Date: 2022-02-12  20:41:51
Commandline: apt -t bullseye-backports install ejabberd
root@vps1-dante:/tmp#

The mod_http_api somehow fixed the crash, it was the only change in the config file.

[badlop]

Without further clues, I consider this a specific case, non-reproducible, and cannot suggest anything specific to the Debian packagers. If no more evidence surfaces, I propose to consider this unresolved (as in X-Files), and maybe someday somebody else kicks it again.