Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pgbouncer didn't even try auth_query to verify user password #484

Closed
bagafoot opened this issue May 8, 2020 · 13 comments · Fixed by #1052
Closed

pgbouncer didn't even try auth_query to verify user password #484

bagafoot opened this issue May 8, 2020 · 13 comments · Fixed by #1052
Labels
bug something is broken

Comments

@bagafoot
Copy link

bagafoot commented May 8, 2020

Hi all, I'm stuck and don't know where I'm doing wrong.
I'm try to configure pgbouncer to auth_type = hba and auth_user with auth_query but pgbouncer didn't send request to database. Pgbouncer and Postgres on the same server. What I'm missing

pgbouncer --version
PgBouncer 1.13.0
libevent 2.0.21-stable
adns: libc-2.17
tls: OpenSSL 1.0.2k-fips  26 Jan 2017
PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit

pgbouncer.ini

[databases]
test = dbname=test host=127.0.0.1 port=5432
test = auth_user=pgbouncer
[users]
test_app = pool_mode=session max_user_connections=3
[pgbouncer]
logfile = /var/log/pgbouncer/pgbouncer.log
pidfile = /var/run/pgbouncer/pgbouncer.pid
listen_addr = 127.0.0.1
listen_port = 6432
auth_type = hba
auth_file = /etc/pgbouncer/userlist.txt
auth_hba_file = /etc/pgbouncer/pg_hba.conf
auth_query = SELECT p_user, p_password FROM public.lookup($1)
admin_users = pgbouncer
stats_users = stats
server_reset_query = DISCARD ALL
server_reset_query_always = 1
max_client_conn = 100
default_pool_size = 20
log_connections = 1
log_disconnections = 1
log_pooler_errors = 1
log_stats = 1
verbose = 1
tcp_keepalive = 1
job_name = pgbouncer

userlist file
"pgbouncer" "md5be5544d3807b54dd0637f2439ecb03b9"

pgbouncer pg_hba.conf file

host    all     pgbouncer       127.0.0.1/32    trust
host    test    test_app        127.0.0.1/32    md5

postgres pg_hba.conf file

 # TYPE  DATABASE        USER            ADDRESS                 METHOD
 
 # "local" is for Unix domain socket connections only
 local   all             all                                    trust
 # IPv4 local connections:
 host    all             all             127.0.0.1/32            md5

lookup function

test=# \df
                                 List of functions
 Schema |  Name  | Result data type |          Argument data types           | Type
--------+--------+------------------+----------------------------------------+------
 public | lookup | record           | INOUT p_user name, OUT p_password text | func
@bagafoot bagafoot changed the title pgbouncer even didn't try auth_query to verify user password pgbouncer didn't even try auth_query to verify user password May 8, 2020
@petere
Copy link
Member

petere commented May 15, 2020

You also need to tell us how you attempted to connect (which command and which options), and the log entries in the postgres and pgbouncer logs from around that time.

@bagafoot
Copy link
Author

bagafoot commented May 15, 2020

You also need to tell us how you attempted to connect (which command and which options), and the log entries in the postgres and pgbouncer logs from around that time.

psql commands
testpsql

and tail -f db log file

dblog

first connection attempt denied by pgbouncer hence nothing in pg log file

pgbouncer log

pgbouncerlog

@petere
Copy link
Member

petere commented May 15, 2020

This setup

[databases]
test = dbname=test host=127.0.0.1 port=5432
test = auth_user=pgbouncer

is not correct. You have two entries for the test database. You need to combine those into one.

@bagafoot
Copy link
Author

This setup

[databases]
test = dbname=test host=127.0.0.1 port=5432
test = auth_user=pgbouncer

is not correct. You have two entries for the test database. You need to combine those into one.

I've change configuration and remove test = auth_user=pgbouncer line and restart/reload pgbouncer but still the same error.

config

reload

errror

@eulerto
Copy link
Member

eulerto commented May 18, 2020

I tried your setup and any user into users section cannot login. Looking at the postgres logs, pgbouncer does not try to connect and execute auth_query when user is from users section. Other users (that is not in users section) work fine. It seems a bug to me.

@bagafoot
Copy link
Author

I tried your setup and any user into users section cannot login. Looking at the postgres logs, pgbouncer does not try to connect and execute auth_query when user is from users section. Other users (that is not in users section) work fine. It seems a bug to me.

You are right, I tried too after your comment and It works fine after remove users in [users] section.

@sribe-drw
Copy link

sribe-drw commented Jul 2, 2020

Yes, I just went through this today, and have some more details. In my specific case I set client_tls_sslmode = prefer in the [pgbouncer] section. What happens then is that any user that has settings specified in the [users] section ignores that, uses the default of no SSL, does not even attempt to fetch user info via the auth_query and reports that password auth failed.

I find that the statement in the docs "Only a few settings are available here." followed by just the two pool_mode and max_user_connections is misleading. In fact, adding client_tls_sslmode=prefer to the user's entry fixes the problem. (That is all that is necessary; I did not repeat the settings for cert/key locations and it worked.)

[EDIT] I also noticed that changes to this did not seem to take effect with a reload, but required actual quit & restart of pgbouncer.

@sribe-drw
Copy link

Addendum to above: reviewing logs from after out config changes, I see:

ERROR skipping user ... because of unknown parameter in settings: client_tls_sslmode

So in fact it "works" by ignoring the setting where we try to limit the pool size for that user, so there seems to be NO WAY to have TLS and per-user settings.

@jpuris
Copy link

jpuris commented Nov 15, 2020

Spent better half of the day on this and can confirm that with following config configuration will not let postgre user to connect to DB, unless it is listed in userlist.txt

[databases]

pgbench = host=127.0.0.1 port=5431 auth_user=pgbouncer pool_size=4
postgres = host=127.0.0.1 port=5431 auth_user=pgbouncer pool_size=0

[users]
postgres = max_user_connections=2

[pgbouncer]
listen_addr = *
listen_port = 5432

logfile = /var/log/pgbouncer/pgbouncer.log
pidfile = /var/run/pgbouncer/pgbouncer.pid

unix_socket_dir = /var/run/pgbouncer

auth_type = md5
auth_query = SELECT p_user, p_password FROM pgbouncer.auth($1)
auth_file = /etc/pgbouncer/userlist.txt

pool_mode = session
server_reset_query = DISCARD ALL

stats_users = pgbouncer_stats
admin_users = pgbouncer_admin

When [users] section is removed and pgbouncer is restarted (reload will not work for this), postgres user is able to log in via pgbouncer.

This pretty much hard blocks us to use [users] all together and there is no way for us to add connection limits to particular users.

@petere
Copy link
Member

petere commented Nov 17, 2020

It has become apparent, also through other reports, that the [users] section support is pretty buggy. It's good to have this reproducible report. The whole thing probably needs a deeper analysis and rewrite.

@Justin-Kwan
Copy link

@eulerto @petere Here is a PR to fix this issue: #706

@veshant
Copy link

veshant commented Oct 11, 2023

@eulerto @petere Here is a PR to fix this issue: #706

@petere can this PR be merged in? I have the same issue

@JelteF
Copy link
Member

JelteF commented Oct 12, 2023

@veshant The #706 PR is not in a state where it can be merged.

benchub pushed a commit to benchub/pgbouncer that referenced this issue Mar 8, 2024
As described in pgbouncer#484, if a user is defined in the [users] section of
the config file (perhaps to take advantage of per-user overrides)
but then that user is *not* defined in auth_file, pgBouncer currently
gets confused when trying to check their password. It sees the user
exists, but fails to notice a password was never defined, resulting
in no running of auth_query.
benchub pushed a commit to benchub/pgbouncer that referenced this issue Mar 12, 2024
As described in pgbouncer#484, if a user is defined in the [users] section of
the config file (perhaps to take advantage of per-user overrides)
but then that user is *not* defined in auth_file, pgBouncer currently
gets confused when trying to check their password. It sees the user
exists, but fails to notice a password was never defined, resulting
in no running of auth_query.

Add a test case to catch this. The case successfully fails before
this patchset and succeeds after it.
benchub pushed a commit to benchub/pgbouncer that referenced this issue Mar 13, 2024
As described in pgbouncer#484, if a user is defined in the [users] section of
the config file (perhaps to take advantage of per-user overrides)
but then that user is *not* defined in auth_file, pgBouncer currently
gets confused when trying to check their password. It sees the user
exists, but fails to notice a password was never defined, resulting
in no running of auth_query.

Add a test case to catch this. The case successfully fails before
this patchset and succeeds after it.
benchub pushed a commit to benchub/pgbouncer that referenced this issue Mar 13, 2024
As described in pgbouncer#484, if a user is defined in the [users] section of
the config file (perhaps to take advantage of per-user overrides)
but then that user is *not* defined in auth_file, pgBouncer currently
gets confused when trying to check their password. It sees the user
exists, but fails to notice a password was never defined, resulting
in no running of auth_query.t

Add a test case to catch this. The case successfully fails before
this patchset and succeeds after it.
@JelteF JelteF closed this as completed in b003f7b May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug something is broken
Projects
None yet
8 participants