Allow ProxyReverseServers to call a script

[tomsommer]

I need to return a certain backend server per user, and I need to get the server from a remote API-call, so i can't make a static file per user, or an SQL call, I need to run something like

ProxyReverseServers exec:/var/ftp/apicall.sh %U

/var/ftp/apicall.sh %U would then return the list of backend servers in STDOUT, either as JSON or whatever.

[Castaglia]

Can some other script, outside of mod_proxy, call that external remote API to obtain the necessary data, and then insert/provision that data into a static file or SQL statement that mod_proxy can then use? That'd be a much cleaner design, with separation of concerns.

Otherwise, with the script approach, mod_proxy could be used as point of DoS attack, by some malicious client connecting many times, simply to cause the connect flood to that remote API, which might not otherwise be accessible to the connecting client.

[tomsommer]

Not really, I would be afraid of things getting out of sync between the API and the internal SQL/data.

If you had an 'exec'-functionality, the options for injecting data would be very powerful.

The .sh|.php script could simply implement some cache to prevent DoS against the remote API.

[Castaglia]

You might be able to do this sort of thing already, using ExecOnConnect...

[tomsommer]

How so?

[Castaglia]

Well, I guess ExecOnConnect would work if your remote API returned data based on the client's IP address. For a per-user thing, you'd use ExecBeforeCommand, _e.g.:

<IfModule mod_exec.c>
  ExecEngine on
  ExecLog ...

  # Call the remote API to get our per-user JSON file of backend servers
  ExecBeforeCommand USER /var/ftp/apicall.sh %U
</IfModule>

all within the <VirtualHost> section for your mod_proxy configuration. The apicall.sh script, then, would call the remote API, obtain the results, and then write out the results to a file (or SQL table) which is then read in by the ProxyReverseServers directive's current mechanisms.

I've not actually tried the above configuration, but...does that make sense?

[tomsommer]

It makes sense, but it feels like a bit of a hack.

Would it not make sense to have ProxyReverseServers support some kind of flexible backend? Like exec?

[Castaglia]

Why does it feel like a hack? An "exec" type functionality is precisely what the mod_exec module provides; why duplicate it (but differently) in mod_proxy?

[tomsommer]

Because it would depend on the exec not failing, and adding the data to either a .json or SQL entry, and then ProxyReverseServers reading this.

Would it be possible to expand ProxyReverseServers with exec: if mod_exec is enabled? Just like sql: works if mod_sql is enabled?

[Castaglia]

The issue of exec not failing, and failing to add to a file or SQL, are the same either way. Having mod_exec do this, or mod_proxy do this, does not change those potential errors. So that is not a convincing argument.

[tomsommer]

What about redis as a ProxyReverseServers-backend? or an entire .json mapped with username => backend (and not a .json per user) - this is why an exec wrapper would be nice, then the type of ProxyReverseServers-backend could be anything :)

[ModulaShop]

+1 for expanding ProxyReverseServers to more backends - even if not really necessary for exec as it imho barely makes a difference to using ExecBeforeCommand.

ldap would be great as this (as far as i figured out) would currently also only be possible with ExecBeforeCommand and there already is mod_ldap that can query directory services.
Also, having the possibility to give more than one server for the directory would be nice, like here:

http://www.proftpd.org/docs/contrib/mod_ldap.html#LDAPServer

[tomsommer]

In ExecBeforeCommand USER you don't get %U ? rendering the above option impossible.

[marius815]

I've been experimenting with this over the past few days, since I need to select a backend server based on supplied username. It is correct that ExecBeforeCommand USER does not have access to %U, so I've tried to do all the script based work using ExecOnCommand USER. I had some problems getting my script to be executed though, but I noticed that the process seem chrooted to etc/proftpd-proxy/empty, so I set up my environment there.

Based on the username, my shell script basically writes the backend server to /tmp/ftpmap/%U.json. This does not work when combining mod_exec and mod_proxy if the file does not already exist. mod_exec will not run the shell script prior to mod_proxy trying to lstat the file, and thus the following error appears: proftpd[1042] 7c7c7ed71d21 (172.23.0.1[172.23.0.1]): mod_proxy/0.5: unable to lstat ProxyReverseServers '/usr/local/etc/proftpd-proxy/empty/tmp/ftpmap/test-user.json': No such file or directory.

Any suggestions for further steps from here?

[marius815]

Just an update from my end. I had some issues using ProxyReverseServers file:/path/to/%U.json as the above error would crash the server. I actually ended up writing 95% of a duct tape filesystem using python-llfuse that would default to a fallback server in case of ENOENT. Then I realized that ProxyReverseServers can actually be used to define fallback servers (not sure how I overlooked that).

I set up a virtual server with an empty AuthUserFile that unauthorized usernames will fall back on. A caveat here is that since mod_exec with ExecOnCommand USER is unable to write to %U.json before mod_proxy falls back, the first time legitimate logins connect they will be refused. However, on consecutive connections, since %U.json has been written, they are correctly routed to their backend server and authenticated.

I wasn't able to figure out how to prevent mod_exec from executing the script from within mod_proxy's empty/ directory, so I was also forced to set up a chrooted environment there.

If not for these issues, an 'exec:' functionality would be redundant, but due to the workarounds I would +1 'exec:'

[tomsommer]

Having the first connect fail is a bad experience, @Castaglia any chance of getting the exec feature added?