Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[defect]: TACACS and Dynamic Clients #5270

Closed
levide opened this issue Jan 18, 2024 · 10 comments
Closed

[defect]: TACACS and Dynamic Clients #5270

levide opened this issue Jan 18, 2024 · 10 comments
Labels
defect category: a defect or misbehaviour

Comments

@levide
Copy link

levide commented Jan 18, 2024

What type of defect/bug is this?

Crash or memory corruption (segv, abort, etc...)

How can the issue be reproduced?

Configuration:

server tacacs {
	namespace = tacacs
	tacacs {
		Authentication {
			log {
				stripped_names = no
				auth = yes
				auth_goodpass = no
				auth_badpass = no
				msg_denied = "You are already logged in - access denied"
			}
			session {
				max_rounds = 4
			}
		}
	}
	listen {
		type = Authentication-Start
		type = Authentication-Continue
		type = Authorization-Request
		type = Accounting-Request
		transport = tcp
		tcp {
			port = 49
			ipaddr = *
			dynamic_clients = yes
			networks {
				allow = 0.0.0.0/0
				#deny = 127.1/16
			}
		}
		limit {
			max_clients = 0
			max_connections = 0
			idle_timeout = 60.0
		}
	}
    new client {
	&control += {
	    &FreeRADIUS-Client-IP-Address = "%{Net.Src.IP}"
	    &FreeRADIUS-Client-Require-MA = no
	    &FreeRADIUS-Client-Secret = "testing123"
	    &FreeRADIUS-Client-Shortname = "%{Net.Src.IP}"
	    &FreeRADIUS-Client-NAS-Type = "other"
	}
	ok
    }
    add client {
	ok
    }
    deny client {
	ok
    }
	recv Authentication-Start {
	}
	authenticate PAP {
		pap
	}
	authenticate CHAP {
		chap
	}
	authenticate MSCHAP {
		mschap
	}
	authenticate MSCHAPv2 {
		mschap
	}
	authenticate ASCII {
		pap
	}
	send Authentication-Pass {
		&reply.Server-Message := "Hello %{User-Name}"
	}
	send Authentication-Fail {
		&reply.Server-Message := "Failed login!"
	}
	send Authentication-GetUser {
		&reply.Server-Message := "Username:"
	}
	send Authentication-GetPass {
		&reply.Server-Message := "Password:"
	}
	recv Authentication-Continue {
		"%{Authentication-Continue-Flags}"
		"%{User-Message}"
		"%{Data}"
	}
	recv Authorization-Request {
		"%{Authentication-Method}"
		"%{Privilege-Level}"
		"%{Authentication-Type}"
		"%{Authentication-Service}"
		"%{User-Name}"
		"%{Client-Port}"
		"%{Remote-Address}"
		"%{Argument-List}"
	}
	send Authorization-Pass-Add {
		&reply.Authorization-Status := Pass-Add
		&reply.Server-Message := "authorization-response-server"
		&reply.Data := "authorization-response-data"
		&reply.Argument-List := "key1=var1"
	}
	recv Accounting-Request {
		#detail
	}
	accounting Start {
	}
	accounting Watchdog-Update {
	}
	accounting Watchdog {
	}
	accounting Stop {
	}
	send Accounting-Success {
		&reply.Server-Message := "Success"
	}
	send Accounting-Error {
		&reply.Server-Message := "Error"
	}
}

Log output from the FreeRADIUS daemon

Thu Jan 18 15:23:03 2024: #### Opening listener interfaces ####
Thu Jan 18 15:23:03 2024: Listening on tacacs_tcp server * port 49 bound to virtual server tacacs
Thu Jan 18 15:23:03 2024: Network - Using new socket tacacs_tcp server * port 49 with FD 19
Thu Jan 18 15:23:03 2024: Opened listener for tacacs
Thu Jan 18 15:23:03 2024: Ready to process requests
Thu Jan 18 15:23:03 2024: Main loop waking up in 0.999971357 seconds
Thu Jan 18 15:23:03 2024: Main loop waking up in 0.999484236 seconds
Thu Jan 18 15:23:03 2024: Worker - Channel open
Thu Jan 18 15:23:03 2024: Worker - Received channel 0x558e54cf0ec0 into array entry 0
Thu Jan 18 15:23:03 2024: Main loop waking up in 0.999051766 seconds
Thu Jan 18 15:23:04 2024: Main loop waking up in 0.999980703 seconds
Thu Jan 18 15:23:05 2024: Main loop waking up in 0.999981934 seconds
Thu Jan 18 15:23:06 2024: Main loop waking up in 0.999984315 seconds
Thu Jan 18 15:23:07 2024: Main loop waking up in 0.999985005 seconds
Thu Jan 18 15:23:08 2024: Main loop waking up in 0.999985052 seconds
Thu Jan 18 15:23:09 2024: Main loop waking up in 0.999980437 seconds
Thu Jan 18 15:23:10 2024: Main loop waking up in 0.999984531 seconds
Thu Jan 18 15:23:11 2024: Main loop waking up in 0.999985438 seconds
Thu Jan 18 15:23:12 2024: Main loop waking up in 0.999981376 seconds
Thu Jan 18 15:23:13 2024: Main loop waking up in 0.99998142 seconds
Thu Jan 18 15:23:13 2024: Network - Reading data from FD 19
Thu Jan 18 15:23:13 2024: proto_tacacs_tcp - starting connection tacacs_tcp from client 192.168.11.251 port 15135 to server * port 49
Thu Jan 18 15:23:13 2024: Listening on tacacs_tcp from client 192.168.11.251 port 15135 to server * port 49 bound to virtual server tacacs
Thu Jan 18 15:23:13 2024: Network - Using new socket tacacs_tcp from client 192.168.11.251 port 15135 to server * port 49 with FD 20
Thu Jan 18 15:23:13 2024: Main loop waking up in 0.71371635 seconds
Thu Jan 18 15:23:13 2024: Network - Reading data from FD 20
Thu Jan 18 15:23:13 2024: hex: -- tacacs_tcp_recv --
Thu Jan 18 15:23:13 2024: hex: 0000: c0 01 01 00 7b 83 c8 2d 00 00 00 19 6f 93 0c 0f 
Thu Jan 18 15:23:13 2024: hex: 0010: c2 c8 5a e2 f1 e4 3b c6 a3 07 68 88 3d 0e cc 94 
Thu Jan 18 15:23:13 2024: hex: 0020: b8 ad f9 f0 df 
Thu Jan 18 15:23:13 2024: proto_tacacs_tcp - Received Authentication seq_no 1 length 37 tacacs_tcp from client 192.168.11.251 port 15135 to server * port 49
CAUGHT SIGNAL: Segmentation fault
Backtrace of last 12 frames:
/home/deck/radius-core-v4/lib/libfreeradius-util.so(fr_fault+0x10a)[0x7faf23f8a6ef]
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7faf23ad7520]
/home/deck/radius-core-v4/lib/libfreeradius-util.so(_fr_event_filter_update+0x86)[0x7faf23fb37f7]
/home/deck/radius-core-v4/lib/libfreeradius-io.so(+0x1127c)[0x7faf23ce527c]
/home/deck/radius-core-v4/lib/libfreeradius-io.so(+0x1bac3)[0x7faf23cefac3]
/home/deck/radius-core-v4/lib/libfreeradius-util.so(fr_event_service+0x7ce)[0x7faf23fb753e]
/home/deck/radius-core-v4/lib/libfreeradius-util.so(fr_event_loop+0x67)[0x7faf23fb79d5]
/home/deck/radius-core-v4/lib/libfreeradius-server.so(main_loop_start+0x2e)[0x7faf23e45b56]
./radiusd(main+0x16ce)[0x558e523d5273]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7faf23abed90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7faf23abee40]
./radiusd(_start+0x25)[0x558e523d3485]
No panic action set
_EXIT(139) CALLED src/lib/util/debug.c[1058].  Last error was: SQL-User-Name (string) contains no 'children' extension
root@auth-tacacs:/home/deck/radius-core-v4/sbin#

Relevant log output from client utilities

No response

Backtrace from LLDB or GDB

Thu Jan 18 16:13:11 2024: Instantiating xlat "logical_or" node 0x5555555ec430, instance 0x5555566f1830, new thread instance 0x55555699f200
Thu Jan 18 16:13:11 2024: Instantiating xlat "logical_or" node 0x5555566fd790, instance 0x5555566fb3b0, new thread instance 0x55555699f2a0
Thu Jan 18 16:13:11 2024: Scheduler created in single-threaded mode
Thu Jan 18 16:13:11 2024: #### Opening listener interfaces ####
Thu Jan 18 16:13:11 2024: Listening on tacacs_tcp server * port 49 bound to virtual server tacacs
Thu Jan 18 16:13:11 2024: Network - Using new socket tacacs_tcp server * port 49 with FD 19
Thu Jan 18 16:13:11 2024: Opened listener for tacacs
Thu Jan 18 16:13:11 2024: Ready to process requests
Thu Jan 18 16:13:11 2024: Main loop waking up in 0.999948115 seconds
Thu Jan 18 16:13:11 2024: Main loop waking up in 0.999497495 seconds
Thu Jan 18 16:13:11 2024: Worker - Channel open
Thu Jan 18 16:13:11 2024: Worker - Received channel 0x5555564adeb0 into array entry 0
Thu Jan 18 16:13:11 2024: Main loop waking up in 0.999067803 seconds
Thu Jan 18 16:13:12 2024: Main loop waking up in 0.999982777 seconds
Thu Jan 18 16:13:13 2024: Main loop waking up in 0.999981963 seconds
Thu Jan 18 16:13:14 2024: Main loop waking up in 0.999985059 seconds
Thu Jan 18 16:13:15 2024: Main loop waking up in 0.999983157 seconds
Thu Jan 18 16:13:16 2024: Main loop waking up in 0.999981196 seconds
Thu Jan 18 16:13:17 2024: Main loop waking up in 0.999970873 seconds
Thu Jan 18 16:13:18 2024: Main loop waking up in 0.999982393 seconds
Thu Jan 18 16:13:19 2024: Main loop waking up in 0.999986589 seconds
Thu Jan 18 16:13:20 2024: Main loop waking up in 0.99998152 seconds
Thu Jan 18 16:13:21 2024: Main loop waking up in 0.999982293 seconds
Thu Jan 18 16:13:22 2024: Main loop waking up in 0.9999804 seconds
Thu Jan 18 16:13:23 2024: Main loop waking up in 0.999985442 seconds
Thu Jan 18 16:13:24 2024: Main loop waking up in 0.999983045 seconds
Thu Jan 18 16:13:24 2024: Network - Reading data from FD 19
Thu Jan 18 16:13:24 2024: proto_tacacs_tcp - starting connection tacacs_tcp from client 192.168.11.251 port 38329 to server * port 49
Thu Jan 18 16:13:24 2024: Listening on tacacs_tcp from client 192.168.11.251 port 38329 to server * port 49 bound to virtual server tacacs
Thu Jan 18 16:13:24 2024: Network - Using new socket tacacs_tcp from client 192.168.11.251 port 38329 to server * port 49 with FD 20
Thu Jan 18 16:13:24 2024: Main loop waking up in 0.485998682 seconds
Thu Jan 18 16:13:24 2024: Network - Reading data from FD 20
Thu Jan 18 16:13:24 2024: hex: -- tacacs_tcp_recv --
Thu Jan 18 16:13:24 2024: hex: 0000: c0 01 01 00 2c 00 28 f8 00 00 00 19 67 03 bd 5f 
Thu Jan 18 16:13:24 2024: hex: 0010: 42 26 9f aa 6e fe b5 08 48 30 c9 18 b9 39 b5 4b 
Thu Jan 18 16:13:24 2024: hex: 0020: 25 8c 34 3d 47 
Thu Jan 18 16:13:24 2024: proto_tacacs_tcp - Received Authentication seq_no 1 length 37 tacacs_tcp from client 192.168.11.251 port 38329 to server * port 49

Thread 1 "radiusd" received signal SIGSEGV, Segmentation fault.
0x00007ffff7ef27f7 in _fr_event_filter_update (file=0x7ffff7c40d2b "src/lib/io/master.c", line=1456, el=0x0, fd=20, filter=FR_EVENT_FILTER_IO, updates=0x7ffff7c50700 <pause_read>) at src/lib/util/event.c:1002
1002		ef = fr_rb_find(el->fds, &(fr_event_fd_t){ .fd = fd, .filter = filter });
(gdb) 
quit
@levide levide added the defect category: a defect or misbehaviour label Jan 18, 2024
@alandekok
Copy link
Member

do you have the full gdb backtrace? The output below isn't overly useful:

/home/deck/radius-core-v4/lib/libfreeradius-util.so(_fr_event_filter_update+0x86)[0x7faf23fb37f7]
/home/deck/radius-core-v4/lib/libfreeradius-io.so(+0x1127c)[0x7faf23ce527c]

you can run bt from the (gdb) prompt, and it will print out the full list of files, line numbers, and functions. That will help a lot.

@levide
Copy link
Author

levide commented Jan 18, 2024 

Listening on tacacs_tcp server * port 49 bound to virtual server tacacs
Ready to process requests
proto_tacacs_tcp - starting connection tacacs_tcp from client 192.168.11.251 port 29856 to server * port 49
Listening on tacacs_tcp from client 192.168.11.251 port 29856 to server * port 49 bound to virtual server tacacs
proto_tacacs_tcp - Received Authentication seq_no 1 length 37 tacacs_tcp from client 192.168.11.251 port 29856 to server * port 49

Thread 1 "radiusd" received signal SIGSEGV, Segmentation fault.
0x00007ffff7ef27f7 in _fr_event_filter_update (file=0x7ffff7c40d2b "src/lib/io/master.c", line=1456, el=0x0, fd=20, filter=FR_EVENT_FILTER_IO, updates=0x7ffff7c50700 <pause_read>) at src/lib/util/event.c:1002
1002		ef = fr_rb_find(el->fds, &(fr_event_fd_t){ .fd = fd, .filter = filter });
(gdb) bt
#0  0x00007ffff7ef27f7 in _fr_event_filter_update (file=0x7ffff7c40d2b "src/lib/io/master.c", line=1456, el=0x0, fd=20, filter=FR_EVENT_FILTER_IO, updates=0x7ffff7c50700 <pause_read>) at src/lib/util/event.c:1002
#1  0x00007ffff7c2427c in mod_read (li=0x555556d62fd0, packet_ctx=0x555556e67640, recv_time_p=0x555556e67620, buffer=0x555556d674d0 "\300\001\001", buffer_len=4096, leftover=0x555556d63148) at src/lib/io/master.c:1456
#2  0x00007ffff7c2eac3 in fr_network_read (el=0x555556358680, sockfd=20, flags=1, ctx=0x555556d630c0) at src/lib/io/network.c:898
#3  0x00007ffff7ef653e in event_callback (fflags=0x7fffffffe194, flags=1, filter=0x7fffffffe1c0, ef=0x555556e6f650, el=0x555556358680) at src/lib/util/event.c:2533
#4  fr_event_service (el=0x555556358680) at src/lib/util/event.c:2651
#5  0x00007ffff7ef69d5 in fr_event_loop (el=0x555556358680) at src/lib/util/event.c:2765
#6  0x00007ffff7d84b56 in main_loop_start () at src/lib/server/main_loop.c:214
#7  0x000055555555b273 in main (argc=2, argv=0x7fffffffe558) at src/bin/radiusd.c:986

@alandekok
Copy link
Member 

#1  0x00007ffff7c2427c in mod_read (li=0x555556d62fd0, packet_ctx=0x555556e67640, recv_time_p=0x555556e67620, buffer=0x555556d674d0 "\300\001\001", buffer_len=4096, leftover=0x555556d63148) at src/lib/io/master.c:1456

the call to fr_event_filter_update() is on line 1457 in the current code. Maybe you're running an older version without recent fixes?

I'll see if I can test it here.

@alandekok
Copy link
Member

maybe mod_event_list_set() in master.c?

	/*
	 *	No dynamic clients AND no packet cleanups?  We don't
	 *	need timers.
	 */
	if (inst->dynamic_clients && !fr_time_delta_ispos(inst->cleanup_delay)) return;

maybe if (!inst->dynamic_clients...) ???

@arr2036
Copy link
Member

arr2036 commented Jan 18, 2024

I've seen this too. It's easy to reproduce. Just haven't had time to look into it. Basically any time dynamic clients is used with tacacs you get this crash.

alandekok added a commit that referenced this issue Jan 18, 2024
@alandekok
always set event list. Should help with #5270
9d25b28
@alandekok
Copy link
Member

I believe that the commit above fixes it. If not, please re-open the bug report.

@levide
Copy link
Author

levide commented Jan 24, 2024 •
edited

Yes, this commit solved the problem. Then other errors occur.
But I understand that there is no point in disassembling them yet. Because dynamic clients for TACACS have not yet been implemented. I'm studying the code further.

@alandekok
Copy link
Member

Dynamic clients share common code for TACACS+, RADIUS/UDP, and RADIUS/TCP. If it works for one, it should work for all of them.

If there are other errors, then please open new issues for them. Or, post the errors to the freeradius-users mailing list.

@levide
Copy link
Author

levide commented Jan 24, 2024

Hmm...
But in:

I can't find section definitions for dynamic clients or I haven't understood all the source code yet.
I'll continue to experiment =)

@alandekok
Copy link
Member

@levide The RADIUS modules don't have dynamic clients listed there, either. It's somewhere else.

The TACACS+ dynamic clients should work. If they don't, debug mode && gdb back traces will help a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
defect category: a defect or misbehaviour
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@alandekok @arr2036 @levide