IPFIX & nano seconds timestamps

[jukrut]

Description
I have Problems to get timestamp_end, timestamp_start display the correct value.

I found a issue mailthread (https://www.mail-archive.com/pmacct-discussion@pmacct.net/msg03195.html) but that did not help so I open a ticket to ask for your help.

I run nfacctd like this:

nfacctd -P print -c src_host,dst_host,src_port,dst_port,proto,tos,timestamp_start,timestamp_end

and with pmacctd like this:

pmacct -i enp39s0 -P nfprobe

nfacct displays it perfectly fine.

but with my generator it always has 0 for timestamp_end and the timestamp_start has the time of end.

e.g.

10.0.0.35                                      8.8.8.31                                       64884     57373     tcp         0      2023-09-27 14:00:14.000000     0000-00-00 00:00:00.000000     10                    10

and wireshark decodes it like this:

Frame 11: 130 bytes on wire (1040 bits), 130 bytes captured (1040 bits) on interface -, id 0
Ethernet II, Src: 00:00:00_00:00:00 (00:00:00:00:00:00), Dst: 00:00:00_00:00:00 (00:00:00:00:00:00)
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
User Datagram Protocol, Src Port: 44280, Dst Port: 2100
Cisco NetFlow/IPFIX
    Version: 10
    Length: 88
    Timestamp: Sep 27, 2023 16:00:14.000000000 CEST
    FlowSequence: 10
    Observation Domain Id: 0
    Set 1 [id=2000] (1 flows)
        FlowSet Id: (Data) (2000)
        FlowSet Length: 72
        [Template Frame: 1]
        Flow 1
            DstAddr: 8.8.8.31
            SrcAddr: 10.0.0.35
            Post NAT Source IPv4 Address: 1.2.3.44
            Post NAPT Source Transport Port: 34630
            Protocol: TCP (6)
            mib Mobile IMSI: 123456810075469
            [Duration: 1000.000000745 seconds (nanoseconds)]
                StartTime: Sep 27, 2023 15:43:34.311081871 CEST
                EndTime: Sep 27, 2023 16:00:14.311082616 CEST
            SrcPort: 64884 (64884)
            DstPort: 57373 (57373)
            Nat Event: NAT44 session create (4)
            Octets: 10
            Packets: 10

the full ipfix messages are uploaded below:

ipfix.pcapng.zip

the main difference that I see is that I use nanoseconds and pmacct microseconds..

Cisco NetFlow/IPFIX
    Version: 10
    Length: 76
    Timestamp: Sep 27, 2023 16:00:14.000000000 CEST
    FlowSequence: 0
    Observation Domain Id: 0
    Set 1 [id=2] (Data Template): 2000
        FlowSet Id: Data Template (V10 [IPFIX]) (2)
        FlowSet Length: 60
        Template (Id = 2000, Count = 13)
            Template Id: 2000
            Field Count: 13
            Field (1/13): IP_DST_ADDR
                0... .... .... .... = Pen provided: No
                .000 0000 0000 1100 = Type: IP_DST_ADDR (12)
                Length: 4
            Field (2/13): IP_SRC_ADDR
                0... .... .... .... = Pen provided: No
                .000 0000 0000 1000 = Type: IP_SRC_ADDR (8)
                Length: 4
            Field (3/13): postNATSourceIPv4Address
                0... .... .... .... = Pen provided: No
                .000 0000 1110 0001 = Type: postNATSourceIPv4Address (225)
                Length: 4
            Field (4/13): postNAPTSourceTransportPort
                0... .... .... .... = Pen provided: No
                .000 0000 1110 0011 = Type: postNAPTSourceTransportPort (227)
                Length: 2
            Field (5/13): PROTOCOL
                0... .... .... .... = Pen provided: No
                .000 0000 0000 0100 = Type: PROTOCOL (4)
                Length: 1
            Field (6/13): mobileIMSI
                0... .... .... .... = Pen provided: No
                .000 0001 1100 0111 = Type: mobileIMSI (455)
                Length: 65535 [i.e.: "Variable Length"]
            Field (7/13): flowStartNanoseconds
                0... .... .... .... = Pen provided: No
                .000 0000 1001 1100 = Type: flowStartNanoseconds (156)
                Length: 8
            Field (8/13): flowEndNanoseconds
                0... .... .... .... = Pen provided: No
                .000 0000 1001 1101 = Type: flowEndNanoseconds (157)
                Length: 8
            Field (9/13): L4_SRC_PORT
                0... .... .... .... = Pen provided: No
                .000 0000 0000 0111 = Type: L4_SRC_PORT (7)
                Length: 2
            Field (10/13): L4_DST_PORT
                0... .... .... .... = Pen provided: No
                .000 0000 0000 1011 = Type: L4_DST_PORT (11)
                Length: 2
            Field (11/13): natEvent
                0... .... .... .... = Pen provided: No
                .000 0000 1110 0110 = Type: natEvent (230)
                Length: 1
            Field (12/13): BYTES
                0... .... .... .... = Pen provided: No
                .000 0000 0000 0001 = Type: BYTES (1)
                Length: 8
            Field (13/13): PKTS
                0... .... .... .... = Pen provided: No
                .000 0000 0000 0010 = Type: PKTS (2)
                Length: 8

Do you see any Problems with the IPFix Messages?

Version
I would say all.. but I just tested with 1.7.7, 1.7.8 and master

[paololucente]

Hi Jurgen ( @jukrut ),

Just to confirm that nano seconds counters are currently not supported. I guess this is not a biggie in terms of development and i may have something working maybe next week. Just please confirm me that this is a priority for you to have it working.

Paolo

[jukrut]

I can't easily change the tool that exports the ipfix, I just did a ipfix-generator that mimics the behaviour of the dataplane to easliy test the behaviour.

So yes I would be very happy If you could add support for nanoseconds.

And thank you very much for the great software and always so quick and helpful responses.

[jukrut]

Hi @paololucente

I managed to get around this by changing the tool that sends the ipfix packets.

If you don't plan to implement it,
a warning/error would still be nice, that nano seconds can't be handled.

[paololucente]

Hi Jurgen ( @jukrut ),

Thanks for this note & good to know you have a workaround. I could add a warning meanwhile although i'd still be planning to look into this and implement it. One thing that did catch my attention is that flowStartNanoseconds and flowEndNanoseconds are 8 bytes, it seems enough space only for an offset to some other base timestamp (and not an absolute value); like, flowStartMicroseconds and flowEndMicroseconds are 16 bytes long in the implementations that i did come across.

Paolo