Read directly from network device #13
base: master
Are you sure you want to change the base?
Conversation
Uses libpcap and hands each packet to the parser.
Grammars often depend on the knowledge of the length of the content. This is especially the case for Ethernet packets. Thus, the modified pac-driver needs to provide this information to Hilti. The caplen provided by libpcap as uint32_t is added at the very beginning to the input of Hilti. In a grammar, this is the first data to be interpreted. By default, a grammar expects big endian; thus, caplen is converted accordingly.
Although this could be derived from the test I wrote, I should provide the commands to test this feature:
Execute the first one and wait a little bit until |
Nice patch, thanks. Couple thoughts:
Regarding your questions:
|
Just a bit of context about EtherPacket and why it requires a length. Basically -- the Ethernet frame itself does not contain any length information about its payload. And the payload of the protocols it contains (like, e.g. IP) does not have to be equivalent to the actual payload of the packet (e.g. when someone adds frame-checksums or timestamp information like Arista can do to it). In those case, you would not read the entire packet if you don't have the outside information that tells you the packet length :) Note - currently I think that length is not really used to make sure that really all data has been read - but it can and should be used for that in the future. |
Chosen such that it should only take up one line, this keeps the output clearer.
Add a new command line option -f. The file is read using libpcap. This shares a lot of code with listening on a network interface. The grammar networkinterface.pac2 was renamed to libpcap.pac2 because it is now used for both reading from pcap files and network devices. There is now an ambiguity with parsers/libpcap.pac2 and parsers/pcap.pac2, there should be a way to resolve this.
Thanks for your feedback!
|
When using libpcap to read from a network interface or a pcap file, hand over the packet capture time and also the total length of the packet; not only the captured length. Currently, the timestamps are handled as being 64 bit. This has to be generalized to cover platforms where time_t and suseconds_t are 32 bit, as well.
I enhanced the patch so now the packet capture time and the actual length of the packet is given to the grammar as well. Tests and grammars still have to be adapted. |
I did some initial development to enable pac-driver to read directly from a network device using libpcap. I would like to propose this as a pull request. In the following, I summarize the work I did, and at the end I list some questions regarding this implementation that should probably be discussed before considering a merge.
The reason I implemented this was because I need to analyse Ethernet packets different from TCP, UDP and ICMP, which are the only protocols that can be analyzed with the Bro plugin. On the long run, it would surely be better to adapt Bro, but for now this seemed too much to dig in.
I introduce a new parameter
-n
that expects the network device as argument. If it is set, then instead of the normalparseSingleInput
call, libpcap is used with a callback functionprocessPacket
for every packet that arrives. The code of this function is inspired by the existing code ofpac-driver
.I did not implement support for
-e
and-i
because this had no immediate benefit for me, but I think it could be possible. I added this restriction to the help output.Because grammars usually need to know the length of the captured content, this is done by prepending this information as
uint32
. Thus, this leads us to some kind of a header before the beginning of the actual Ethernet packet, like inpcap.pac2
. I added an example grammar withnetworkinterface.pac2
. After the header it immediately refers to the pcap grammar for the handling of Ethernet packets.I added a test for this new functionality, which uses tcpreplay to replay
dns.trace
, which is already included in the source tree. The implementation of the test is kind of ugly, because the grammar does not compile withhilti-build
because of #3. As soon as this gets fixed, there is no need for this ugly timer-based approach.pac-driver seems to be able to process packets quite fast with this addition, although I have no statistics to show.
Questions:
processPacket
, there is the commented lineGC_DTOR(input, hlt_bytes, ctx);
. This line is also present inparseSingleInput
and that's why I copied it. The problem is that pac-driver segfaults at the processing of the second packet if this line is executed and that's why I commented it out. I suspect this of being a memory leak. What do you think?processPacket
? Do you see possible optimizations?