BIOCSRTIMEOUT: Invalid argument

[kencu]

Compiling release 0.4.2 using what appears to be very standard settings on MacPorts I am getting this error on 10.13 and also on 10.6:

$ sudo tracebox -s test_ftp.tbx 81.200.198.6
sh: ipfw: command not found
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Packet::SocketSendRecv() : Listening device en1: BIOCSRTIMEOUT: Invalid argument
Abort trap: 6

$ sudo tracebox localhost
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Packet::SocketSendRecv() : Listening device lo0: BIOCSRTIMEOUT: Invalid argument
Abort trap: 6

$ sudo tracebox -i en1 apple.com
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Packet::SocketSendRecv() : Listening device en1: BIOCSRTIMEOUT: Invalid argument
Abort trap: 6

Does this give you any ideas what might be up?

Process:               tracebox [758]
Path:                  /opt/local/bin/tracebox
Identifier:            tracebox
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        sudo [757]
Responsible:           tracebox [758]
User ID:               0

Date/Time:             2017-10-14 13:05:43.174 -0700
OS Version:            Mac OS X 10.13 (17A405)
Report Version:        12
Anonymous UUID:        F49C63B1-A38B-D268-55E1-E7224AAAFF41

Sleep/Wake UUID:       3FAC5710-8BF8-4297-A118-6F534E98FFD6

Time Awake Since Boot: 1500 seconds
Time Since Wake:       1000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
abort() called
terminating with uncaught exception of type std::runtime_error: Packet::SocketSendRecv() : Listening device en1: BIOCSRTIMEOUT: Invalid argument

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff720d6fce __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff72214150 pthread_kill + 333
2   libsystem_c.dylib             	0x00007fff7203332a abort + 127
3   libc++abi.dylib               	0x00007fff7001af8f abort_message + 245
4   libc++abi.dylib               	0x00007fff7001b113 default_terminate_handler() + 241
5   libobjc.A.dylib               	0x00007fff7139ca4f _objc_terminate() + 105
6   libc++abi.dylib               	0x00007fff700367c9 std::__terminate(void (*)()) + 8
7   libc++abi.dylib               	0x00007fff7003626d __cxa_throw + 121
8   tracebox                      	0x000000010d6ce3ee Crafter::Packet::SocketSendRecv(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, double, int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 3888
9   tracebox                      	0x000000010d6ab1c5 doTracebox(std::__1::shared_ptr<Crafter::Packet>, int (*)(void*, unsigned char, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, PacketModifications*), std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, void*) + 394
10  tracebox                      	0x000000010d6abdcd main + 2139
11  libdyld.dylib                 	0x00007fff71f87145 start + 1

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x00007fffab321340  rcx: 0x00007ffee2557f68  rdx: 0x0000000000000000
  rdi: 0x0000000000000307  rsi: 0x0000000000000006  rbp: 0x00007ffee2557fa0  rsp: 0x00007ffee2557f68
   r8: 0x00007ffee2557e30   r9: 0x0000000000000011  r10: 0x0000000000000000  r11: 0x0000000000000206
  r12: 0x0000000000000307  r13: 0x0000000000000030  r14: 0x0000000000000006  r15: 0x000000000000002d
  rip: 0x00007fff720d6fce  rfl: 0x0000000000000206  cr2: 0x00007fffab2ff148
  
Logical CPU:     0
Error Code:      0x02000148
Trap Number:     133


Binary Images:
       0x10d6a7000 -        0x10d715ff7 +tracebox (0) <989EB202-A16C-3639-8EA7-0F861122D429> /opt/local/bin/tracebox
       0x10d779000 -        0x10d79bffb +liblua.dylib (0) <AD6BE442-3EAF-35FD-8332-D52FECCFAEFA> /opt/local/lib/liblua.dylib
       0x10d7aa000 -        0x10d7afff7 +libjson-c.2.dylib (0) <C88A38DA-5065-3A34-852A-C63F02BC1DE8> /opt/local/lib/libjson-c.2.dylib
       0x10ec0e000 -        0x10ec5898f  dyld (519.2.1) <002B0442-3D59-3159-BA10-1C0A77859C6A> /usr/lib/dyld
    0x7fff6f803000 -     0x7fff6f836fff  libclosured.dylib (519.2.1) <31A6AC03-8F51-367E-9E00-FF0F1AD10F6F> /usr/lib/closure/libclosured.dylib
    0x7fff6fd14000 -     0x7fff6fd15ff3  libSystem.B.dylib (1252) <BBBE6ABC-FD84-30A3-B699-95F1D6EFB334> /usr/lib/libSystem.B.dylib
    0x7fff6ffc2000 -     0x7fff70018fff  libc++.1.dylib (400.9) <FCF5E1F6-2B04-3545-8004-F3AB32FED172> /usr/lib/libc++.1.dylib
    0x7fff70019000 -     0x7fff7003dff7  libc++abi.dylib (400.7) <217656D5-BC40-37FF-B322-91CB2AAD4F34> /usr/lib/libc++abi.dylib
    0x7fff71386000 -     0x7fff717773b7  libobjc.A.dylib (723) <37A7D77E-952C-3F5D-970B-3CDE349B2322> /usr/lib/libobjc.A.dylib
    0x7fff71791000 -     0x7fff717c5fff  libpcap.A.dylib (79) <D400B242-A56A-39CD-9445-740460F1A62F> /usr/lib/libpcap.A.dylib
    0x7fff71845000 -     0x7fff71861ffb  libresolv.9.dylib (65) <E8F3415B-4472-3202-8901-41FD87981DB2> /usr/lib/libresolv.9.dylib
    0x7fff71e20000 -     0x7fff71e24ff7  libcache.dylib (80) <354F3B7D-404E-3398-9EBF-65CA2CE65211> /usr/lib/system/libcache.dylib
    0x7fff71e25000 -     0x7fff71e2fff3  libcommonCrypto.dylib (60118.1.1) <6C502A55-3B54-3B48-BA7B-FA39F56C0B1E> /usr/lib/system/libcommonCrypto.dylib
    0x7fff71e30000 -     0x7fff71e37fff  libcompiler_rt.dylib (62) <4487CFBA-A5D7-3282-9E6B-94CAD7BE507E> /usr/lib/system/libcompiler_rt.dylib
    0x7fff71e38000 -     0x7fff71e40fff  libcopyfile.dylib (146) <7E4BD264-5617-339B-AC73-E8D07EF2C51D> /usr/lib/system/libcopyfile.dylib
    0x7fff71e41000 -     0x7fff71ec5ff7  libcorecrypto.dylib (562) <7974762C-DEF7-3056-9856-9821C27846DB> /usr/lib/system/libcorecrypto.dylib
    0x7fff71f4c000 -     0x7fff71f85ff7  libdispatch.dylib (913.1.6) <0DD78497-6A2A-350A-99EF-15BF41EA07DD> /usr/lib/system/libdispatch.dylib
    0x7fff71f86000 -     0x7fff71fa3ff7  libdyld.dylib (519.2.1) <2597D818-42D2-3375-BD9D-451D5942A6BA> /usr/lib/system/libdyld.dylib
    0x7fff71fa4000 -     0x7fff71fa4ffb  libkeymgr.dylib (28) <6D84A96F-C65B-38EC-BDB5-21FD2C97E7B2> /usr/lib/system/libkeymgr.dylib
    0x7fff71fb2000 -     0x7fff71fb2ff7  liblaunch.dylib (1205.1.10) <5AD77A68-BB4D-33AA-AA4F-DF51D0972FD9> /usr/lib/system/liblaunch.dylib
    0x7fff71fb3000 -     0x7fff71fb7ffb  libmacho.dylib (900.0.1) <756F2553-07B6-3B42-ACEA-2F0F1A5E8D0F> /usr/lib/system/libmacho.dylib
    0x7fff71fb8000 -     0x7fff71fbaff3  libquarantine.dylib (86) <6AC8773F-3817-3D82-99C2-01BABB9C3CBB> /usr/lib/system/libquarantine.dylib
    0x7fff71fbb000 -     0x7fff71fbcff3  libremovefile.dylib (45) <912FA211-DD8C-3C92-8424-21B89F8B10FD> /usr/lib/system/libremovefile.dylib
    0x7fff71fbd000 -     0x7fff71fd4fff  libsystem_asl.dylib (356.1.1) <94972913-9DF0-3C78-847C-43E58919E3DA> /usr/lib/system/libsystem_asl.dylib
    0x7fff71fd5000 -     0x7fff71fd5fff  libsystem_blocks.dylib (67) <F2493BB5-B1C6-3C4D-9F1F-1B402E0F1DB7> /usr/lib/system/libsystem_blocks.dylib
    0x7fff71fd6000 -     0x7fff7205ffff  libsystem_c.dylib (1244.1.7) <2D4C21C0-9938-3552-8DC1-2C1CFAA10D38> /usr/lib/system/libsystem_c.dylib
    0x7fff72060000 -     0x7fff72063ffb  libsystem_configuration.dylib (963) <D7EFEAE6-22A0-348E-BBBE-44FFD41934FA> /usr/lib/system/libsystem_configuration.dylib
    0x7fff72064000 -     0x7fff72067ffb  libsystem_coreservices.dylib (51) <21A488D0-2D07-344E-8631-CC8B2A246F35> /usr/lib/system/libsystem_coreservices.dylib
    0x7fff72068000 -     0x7fff72069fff  libsystem_darwin.dylib (1244.1.7) <552C05CA-D151-3F93-B570-413340BDA9C7> /usr/lib/system/libsystem_darwin.dylib
    0x7fff7206a000 -     0x7fff72070ff7  libsystem_dnssd.dylib (878.1.1) <6E28973E-A258-36F6-ACFB-259ED5885C7A> /usr/lib/system/libsystem_dnssd.dylib
    0x7fff72071000 -     0x7fff720baff7  libsystem_info.dylib (517) <483BE95B-62EB-3663-ACB3-9915A40C70F5> /usr/lib/system/libsystem_info.dylib
    0x7fff720bb000 -     0x7fff720e0ff7  libsystem_kernel.dylib (4570.1.46) <71BA15CB-3056-3CBD-A5F5-EE61566EEA0C> /usr/lib/system/libsystem_kernel.dylib
    0x7fff720e1000 -     0x7fff7212cfcb  libsystem_m.dylib (3146) <ABB1B85F-9FFE-31B8-AD4F-E39A30794A93> /usr/lib/system/libsystem_m.dylib
    0x7fff7212d000 -     0x7fff7214cfff  libsystem_malloc.dylib (140.1.1) <9F0745FF-B92F-330D-8812-BB74001D1D33> /usr/lib/system/libsystem_malloc.dylib
    0x7fff7214d000 -     0x7fff721f0ff3  libsystem_network.dylib (1229.1.5) <A51EB0A4-9AA5-3F84-8E25-DD8EABE0F97E> /usr/lib/system/libsystem_network.dylib
    0x7fff721f1000 -     0x7fff721fbffb  libsystem_networkextension.dylib (767.1.4) <E842569D-00BA-3DE7-AD5A-9EF33422CF3E> /usr/lib/system/libsystem_networkextension.dylib
    0x7fff721fc000 -     0x7fff72205ff3  libsystem_notify.dylib (172) <98EA3D62-7C86-30DE-8261-D020D2F1EFF3> /usr/lib/system/libsystem_notify.dylib
    0x7fff72206000 -     0x7fff7220dff7  libsystem_platform.dylib (161) <9F5A67F7-BC65-300F-BD74-07E7732D6372> /usr/lib/system/libsystem_platform.dylib
    0x7fff7220e000 -     0x7fff72219ff7  libsystem_pthread.dylib (301.1.6) <6D0B0110-2B44-3D3C-B672-BD08FE46378A> /usr/lib/system/libsystem_pthread.dylib
    0x7fff7221a000 -     0x7fff7221dffb  libsystem_sandbox.dylib (765.1.8) <EC909728-0365-3710-B00B-0BCFCE03DC71> /usr/lib/system/libsystem_sandbox.dylib
    0x7fff7221e000 -     0x7fff7221fff3  libsystem_secinit.dylib (30) <F06ADB8F-9E94-34A7-B3C9-2C22FDD14BAD> /usr/lib/system/libsystem_secinit.dylib
    0x7fff72220000 -     0x7fff72227ff7  libsystem_symptoms.dylib (820.1.4) <BF0566B2-9475-3B5C-8641-7910F313BA3C> /usr/lib/system/libsystem_symptoms.dylib
    0x7fff72228000 -     0x7fff7223bff7  libsystem_trace.dylib (829.1.2) <10955EBB-1AC8-3085-9A2D-F3088CA2DF71> /usr/lib/system/libsystem_trace.dylib
    0x7fff7223d000 -     0x7fff72242ff7  libunwind.dylib (35.3) <6D4FCD49-D2A9-3233-95C7-A7635CE265F2> /usr/lib/system/libunwind.dylib
    0x7fff72243000 -     0x7fff7226eff7  libxpc.dylib (1205.1.10) <E7C5DB12-6D0E-3D1E-A743-F750DF112F5F> /usr/lib/system/libxpc.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 911
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=197.0M resident=0K(0%) swapped_out_or_unallocated=197.0M(100%)
Writable regions: Total=20.5M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=20.5M(100%)
 
                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Activity Tracing                   256K        2 
Kernel Alloc Once                    8K        2 
MALLOC                            12.0M       11 
MALLOC guard page                   16K        4 
STACK GUARD                       56.0M        2 
Stack                             8192K        2 
__DATA                            2384K       47 
__LINKEDIT                       187.5M        6 
__TEXT                            9808K       46 
shared memory                       12K        4 
===========                     =======  ======= 
TOTAL                            275.7M      116 

[kencu]

Here is MacPorts build file I'm using -- looks very standard:

# -*- coding: utf-8; mode: tcl; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*- vim:fenc=utf-8:ft=tcl:et:sw=4:ts=4:sts=4
 
PortSystem          1.0
PortGroup           github 1.0
PortGroup           cxx11 1.1

github.setup        tracebox tracebox 0.4.2 v

fetch.type          git
post-extract {
    system -W ${worksrcpath} "git submodule update --init"
}

categories          net
platforms           darwin
license             GPL-2

maintainers         gmail.com:rmerpes openmaintainer

description         Tool similar to traceroute
long_description    Tracebox is a tool that allows to detect middleboxes on any paths, \
					i.e., between a source and any destination. \
  					Tracebox can be viewed as a tool similar to traceroute as \
  					it uses ICMP replies to identify changes in the packets.

checksums           rmd160  76ce79f512a53b84ff5250c17b02fb79ba412806 \
                    sha256  7b14482aacaafcdd7271e7f7b6a10c7d50a8969d491a1af6948ef8388f44bc3c

use_autoreconf      yes
autoreconf.args     --force --install --verbose

# this patch can be deleted once the version is bumped to 0.4.3 or greater
patch.pre_args      -p1
patchfiles-append   patch-remove-dirty.diff

depends_lib-append  port:lua \
                    port:json-c

configure.args      --disable-debug \
                    --disable-dependency-tracking \
                    --disable-silent-rules

livecheck.type      none

and the patchfile is very standard too - same one used on homebrew.

From 5ee627ca64aaea7ebcd52086834c4cca3be6d25b Mon Sep 17 00:00:00 2001
From: Olivier Tilmans <olivier.tilmans@uclouvain.be>
Date: Mon, 2 Oct 2017 09:36:30 +0200
Subject: [PATCH] Remove  from the displayed version string

Signed-off-by: Olivier Tilmans <olivier.tilmans@uclouvain.be>
---
 configure.ac | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index 3ffa41235c4769c0038d20f216de558830304723..3eef68962f0c987bad3ba391292ee558c52256d0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -266,7 +266,7 @@ fi
 AM_CONDITIONAL([HAVE_LDOC],[test -n "$LDOC"])
 
 AC_DEFINE_UNQUOTED([_REV_PARSE], 
-          ["$(git describe --abbrev=16 --dirty --always --tags)"],
+          ["$(git describe --abbrev=16 --always --tags)"],
           ["Version number: tag-commits-g revision-branch status"])
 
 AC_CONFIG_FILES([

[oliviertilmans]

Hi @kencu,

Thanks for the report! All 3 crashes are related to the way the libcrafter library which is used by tracebox sets the bpf filters on its raw socket and the associated timers (i.e., it was using an illegal value).
This should be solved in 4505af5 .

Additionally, your first test exhibited the issue reported in #49:
sh: ipfw: command not found Which I am unable to solve as I have no macOS at hand and no knowledge of pf (esp. the dynamic rule entries adding/removal process which is what we currently use iptables/ipfw for).

[kencu]

thank you - I'll test this out and see if it works as expected.

[kencu]

It builds correctly, and I no longer get the Abort Trap 6 error. Thanks!

At this moment, I'm not certain if the output I'm seeing is what is expected. For example

$ traceroute localhost
traceroute to localhost (127.0.0.1), 64 hops max, 52 byte packets
 1  localhost (127.0.0.1)  0.366 ms  0.232 ms  0.050 ms

$ sudo tracebox localhost
tracebox to 127.0.0.1 (localhost): 64 hops max
: *
: *
: *
: *
: *
: *
: *
: *
	: *

: *

: *

: *
: *
: *
: *
: *
: *
: *
: *
: *
: *
: *
: *
: *
: *
: *
 *
: *
: *
: *
: *
 : *
!: *
": *
#: *
$: *
%: *
&: *
': *
(: *
): *
*: *
+: *
,: *
-: *
.: *
/: *
0: *
1: *
2: *
3: *
4: *
5: *
6: *
7: *
8: *
9: *
:: *
;: *
<: *
=: *
>: *
?: *
@: * 

[oliviertilmans]

This output is indeed unexpected as instead of displaying the ttl distance it displays the ascii interpretation of the ttl ... (hence the whole set of characters ...).

098cd08 should fix that display issue (note that it was fine for i.e. the json output or the lua api).

As tracebox never stops while traceroute does, it likely means that tracebox fails to infer that it reached the target host, which could stem from 3 different reasons:

• (i) tracebox itself does not understand that it received a reply from the destination (i.e., the source address of the reply is not the destination address it was targeting)
• (ii) libcrafter fails to bind properly the socket to the loopback interface
• (iii) libcrafter installs an incorrect bpf filter on the raw socket, preventing it from actually capturing the generated reply

I cannot reproduce that here, could you record a trace when doing your test ? i.e.
sudo tcpdump -i lo -Uw ~/capture.pcap 'icmp or tcp and dst port 80'
and upload the resulting capture.pcap file from your home directory (this assumes you run sudo tracebox localhost, i.e. implicitely use IPv4 probes with a TCP header towards port 80).

Detecting (i) should be straightforward from the trace, (ii) should not matter in this case (i.e. tracebox currently listens on too many interfaces, not the reverse), (iii) can be inferred from the trace, i.e., there was an (ICMP) reply seen by tcpdump but libcrafter filtered it out.

Could you try the revision 2b3bb42 and run tracebox with a -D flag (uppercase D) when runing the test and report the output (i.e. the filter that is set on the interface as that would help to diagnose (iii) if that was the case) ?

In short,

1. Rebase and build against 2b3bb42
2. Run in a terminalsudo tcpdump -i lo -Uw ~/capture.pcap 'icmp or tcp and dst port 80'
3. Run in another terminal sudo tracebox -D -m 5 localhost and wait for it to finish
4. Upload the resulting capture.pcap file from your home directory.

Thanks!

[kencu]

OK. For some reason I had to manually bring up the loopback interface with this:
sudo ifconfig lo0 alias 127.0.0.1 up but once that was done, everything else seemed to work. I had to zip the capture.pcap file so that github would accept it.

capture.pcap.zip

[oliviertilmans]

Thanks for the trace. However, it looks like I wrote incorrect tcpdump filters (i.e. these should have been tcp and dst port 80 or icmp and not the opposite) as the trace does not contain any other packet than the one sent by tracebox.

For some reason I had to manually bring up the loopback interface with this:
sudo ifconfig lo0 alias 127.0.0.1 up but once that was done, everything else seemed to work.

Was traceroute working before you brought the interface up ? Having to bring it up in the first place sounds odd.

Could try to compare what you see in tcpdump when running a traceroute vs running tracebox ? (i.e. without filtering on the interface or the packet types). If tracebox does receive ICMP replies, i'd be interested to get another capture of these (i.e. to get the source IP as well as the values of the various ICMP fields and payload).

Did you try tracebox towards external hosts ?
e.g. sudo tracebox -p IP/TCP/MPCAPABLE multipath-tcp.org or sudo tracebox google.com