libpcap can't read pcapng files with more than one link-layer type

[daltonhamilton]

Version:
tcpdump version 4.9.3 -- Apple version 114.100.1.700.1
libpcap version 1.9.1
LibreSSL 3.3.6

When I transfer the resulting pcap to Ubuntu to use Zeek, Zeek gives error:

fatal error in line 1: failed to reed a packet from xxx.pcap: an interface has a type 0 different from the type of the first interface

Zeek version is 6.0.2.
The capture was on MacOS using -i any for interface paramater.

[infrastation]

I wonder if that's Apple libpcap too, not just Apple tcpdump. If you run file xxx.pcap, how does it identify?

[daltonhamilton]

Both macOS (ver 12.7.2) and Ubuntu return the same:

% file all-1-240121.pcap
all-1-240121.pcap: pcapng capture file - version 1.0

[daltonhamilton]

Just FYI, Wireshark on both Ubuntu and MacOS can read the pcap fine.

[infrastation]

Thank you for these details. So, this is a pcapng file, which means you are using Apple libpcap because the libpcap from tcpdump.org does not write pcapng files. Let's suppose that the pcapng file is correctly formed, in this case the error message from Zeek would mean it expects exactly one DLT in an input file, whether pcap or pcapng. Then at least some of the following would work around the problem:

• Use tcpdump and libpcap from tcpdump.org to make the capture, so the file is a pcap file.
• Use tshark -F pcap to make the capture, so the file is a pcap file.
• Use Apple tcpdump with -i <interface> other than any, so the file is a pcapng file with exactly one DLT.

[daltonhamilton]

Thanks for the help. I was hoping there would be a way to edit the current cap file as it is a 24hr capture and will take time to reproduce. Maybe I'll do a -i en2 and run it for a short period and validate that will work before I move do another coplete 24hr capture scenario again.

[guyharris]

Use Apple tcpdump with -i other than any, so the file is a pcapng file with exactly one DLT.

Or just a pcap file:

$ /usr/sbin/tcpdump --version
tcpdump version 4.99.1 -- Apple version 127.140.3
libpcap version 1.10.1
LibreSSL 3.3.6
$ /usr/sbin/tcpdump -i en0 -w /tmp/output
tcpdump: listening on en0, link-type EN10MB (Ethernet), snapshot length 524288 bytes
^C19 packets captured
23 packets received by filter
0 packets dropped by kernel
$ file /tmp/output
/tmp/output: pcap capture file, microsecond ts (little-endian) - version 2.4 (Ethernet, capture length 524288)

Apple's tcpdump writes pcapng files only if 1) told to do so with the -P or --apple-pcapng command-line flag, 2) run without a -i flag, or 3) run with -i any.

[guyharris]

I was hoping there would be a way to edit the current cap file as it is a 24hr capture and will take time to reproduce.

I'm not seeing an option to support writing only packets to particular interfaces, or writing only packets not to particular interfaces, with Wireshark's editcap, but such an option might be useful, to allow a capture such as yours to be split between a capture on one or more Ethernet interfaces and a capture on the loopback interface.

[guyharris]

Wireshark on both Ubuntu and MacOS can read the pcap fine.

Wireshark has its own separate library to read capture files, which fully supports pcap and pcapng, as well as a bunch of other capture file formats.

[guyharris]

in this case the error message from Zeek would mean it expects exactly one DLT in an input file, whether pcap or pcapng

In particular, it means it's using libpcap, which expects exactly one DLT in an input file, whether pcap or pcapng, either because 1) it's tcpdump.org's libpcap or 2) it's Apple's libpcap but the special undocumented APIs in Apple's libpcap aren't being used, so it's just using the standard libpcap APIs, which don't support multiple DLTs per file.

(Sadly, Apple's code to implement those undocumented APIs is licensed under the APSL, which has patent clauses, so I don't want to include that code in tcpdump.org's libpcap. It's also annoying because, to open pcapng files, you have to use a different call, which doesn't support opening pcap files, so using those APIs requires extra cruft.)

[guyharris]

But whatever APIs we add in libpcap to fully support pcapng, programs will have to be changed to use the new APIs, so, were we to do that, Zeek would have to be changed to handle your pcapng file. (So would tcpdump, but we'd change that.)

[mcr]

I begin to get really annoyed at Apple for shipping code that calls itself "tcpdump", but is not in fact our code.

[guyharris]

It's our code plus a bunch of Apple additions.

One addition we might want to pick up is the -g flag:

       -g
       --apple-oneline
              Do not insert line break after IP header in verbose mode for  
              easier parsing. This is an Apple addition.

"g" is for "grepable".

(And one thing about which I get annoyed at Apple on occasion is autocorrect. :-))

[infrastation]

As far as the original problem goes, is there a good reason to keep this issue open?

[daltonhamilton]

None. On Jan 26, 2024, at 12:51 PM, Denis Ovsienko ***@***.***> wrote: As far as the original problem goes, is there a good reason to keep this issue open?

[infrastation]

See also #1252.

[guyharris]

This requires a new API that fully supports pcapng.

[guyharris]

Duplicate of #1321

[guyharris]

The only way in which this is connected with macOS is that macOS's libpcap and tcpdump can write pcapng files in which not all the IDBs have the same link-layer type. The same is true of Wireshark on any platform.

The underlying problem is that libpcap doesn't have APIs that fully support pcapng. The issue for that is #1321.