Could this compiler support XDP_REDIRECT

[kwjjyn]

The XDP_TX return code only support to send the packets to the same NIC card . And XDP_REDIRECT can send the received packets to another NIC . However , this compiler seems not to implement XDP_REDIRECT ? Is there any other way to implement forwarding function from one nic like eth1 to another NIC like eth2 using P4C-XDP compiler?

I notice that the xdp_model.p4 has defined xdp_output:

struct xdp_output {
    xdp_action output_action;
    bit<32> output_port;  // output port for packet
}

As for the output_port , is it used to appoint the different NIC id?

[mihaibudiu]

The work in the compiler to support XDP_REDIRECT is very simple.
At the time we implemented this it wasn't supported by the kernel. Maybe this has changed.
@williamtu : do you know how one has to proceed to use XDP_REDIRECT?

[mihaibudiu]

In fact, no work in the compiler is needed. You can just add XDP_REDIRECT to the p4include/xdp_model.p4 file and it will produce the right C. I will submit a PR. However, I still don't know how you should use it.

[kwjjyn]

Thank you for your prompt reply.
After I add XDP_REDIRECT to the xdp_model.p4 , I can successfully compile the xdp program with XDP_REDIRECT .
I guess the xout.output_port is related to different NICs. For example , output_port set to 0 corresponds to eth1 and 1 corresponds to eth2. Then I still set output_port as 0 and output_action as XDP_REDIRECT to implement the same function of XDP_TX transmitting received packets to the same NIC eth0.

if(hd.ipv4.isValid())
        {
            dstmactable.apply();
            xout.output_port = 0;
            xout.output_action = xoutdrop ? xdp_action.XDP_DROP : xdp_action.XDP_REDIRECT;
        }

and the generated xdp5.c file :

    xout.output_port = 0;
    if (xoutdrop_0)
            tmp = XDP_DROP;
    else
            tmp = XDP_REDIRECT;
    xout.output_action = tmp;

However, I cannot capture any transmitted packets . And then I did another test to set the out_port as 1 and output_action as XDP_REDIRECT to see if I can capture packets in the eth1. Unfortunately still cannot.
So how to implement sending packets to different NIC using XDP_REDIRECT and how to specify the NIC id ? Is it the output_port to specify which NIC to redirect to ?
I see there is a map, called ebpf_outTable, only has one max_entries in /sys/fs/bpf/xdp/ . Is this map used to specify different NICs cooperating with XDP_REDIRECT ？

Thank you for your attention to this matter. It would be nice if you could provide a bit more information on this question.

[williamtu]

Hi,

XDP_REDIRECT requires using a map, ebpf_outTable, to lookup the output port.
And the now we send packet using TC, not XDP, by calling a bpf helper function called bpf_clone_redirect.
For example, if you want to send to eth8, eth8's ifindex should be programmed into the value of the map.

Are you able to see the contents in the ebpf_outTable? ex: by using the bpftool?
Thanks

[kwjjyn]

Thank you for your prompt reply.
Yes, I can see the contents of ebpf_outTable generated by xdp5.p4 with command :

bpftool map dump pinned /sys/fs/bpf/xdp/globals/ebpf_outTable

the output is

key:
00 00 00 00
value (CPU 00): 00 00 00 00
value (CPU 01): 01 00 00 00
value (CPU 02): 00 00 00 00
value (CPU 03): 00 00 00 00
value (CPU 04): 00 00 00 00
value (CPU 05): 00 00 00 00
value (CPU 06): 00 00 00 00
value (CPU 07): 00 00 00 00
Found 1 element

Do you have any simple examples that using TC cooperating with xdp program generated by P4C-XDP to send raw packets from one NIC to another . The xdp program just pass packets. Then TC receives the packets and send to another NIC.
Really appreciate your help!

[kwjjyn]

Actually the ebpf_outTable only has one entry in the xdp.c file. And how to specify the max_entries of this map in P4 program? Do you know how to specify which NIC to send packets in P4 program ?
I guess maybe the xout.output_port in P4 program is the ebpf_outTable's key and NIC's ifindex is the value ?
It would be nice if you could provide a bit more information on this question.

[williamtu]

So looking into more details, XDP_REDIRECT is not fully supported now.
The way p4c-xdp implemented redirect or send to another port is to use TC.
So idea is that the XDP program always return XDP_PASS, and the packet will
go to the TC layer.
At the TC layer, we have to insert another ebpf program to forward this packet,
based on the lookup result of ebpf_outTable.
So in your case, it doesn't work because you need another ebpf program attached to TC.

The reason we did this is because at the time we started the project, there is no XDP_REDIRECT support.
Now the right way to do XDP_REDIRECT is through BPF_MAP_TYPE_DEVMAP
see this slide for more details
http://people.netfilter.org/hawk/presentations/LLC2018/XDP_LLC2018_redirect.pdf

In this way, we don't need TC to send packet, the packet will be handled at XDP layer.

[williamtu]

Actually the ebpf_outTable only has one entry in the xdp.c file. And how to specify the max_entries of this map in P4 program? Do you know how to specify which NIC to send packets in P4 program ?

hash_table(64), example below:

    table dstmactable {
        key = { hd.ethernet.protocol : exact; }
        actions = {
            Fallback_action;
            Drop_action;
        }
        default_action = Fallback_action;
        implementation = hash_table(64);
    }

to send to a nic, get the nic's ifindex, and program the <key, value> to <port, ifindex> into the
ebpf_outTable.

I guess maybe the xout.output_port in P4 program is the ebpf_outTable's key and NIC's ifindex is the value ?
Yes, that's right

[kwjjyn]

Hi,
Thanks a lot for your help. Do you have any simple ebpf program attched to TC which uses ebpf_outTable to send raw packets from one NIC to another ?

[kwjjyn]

Yes, the hash_table(64) can appoint the dstmactable's max entries. Actually I want to specify the max entries of ebpf_outTable. However, it seems that there is no method to specify this in P4 program .
And as for the forwarding workflow , I guess it's maybe like this:

First, user knows which packet should be passed to the TC layer to be forwarded. For example , user wants the packet , which dst IP address is 10.0.0.1 , received from eth0 to be passed to TC and forwarded to eth8. So the P4 table might like this :

action Fallback_action {
        xout.output_port = 8;
        xout.output_action = XDP_PASS;
}

table dstmactable {
        key = { hd.ipv4.dstAddr : exact; }
        actions = {
            Fallback_action;
            Drop_action;
        }
        default_action = Drop_action;
        implementation = hash_table(64);
    }

the table entry is

key	actions
10.0.0.1	Fallback_action

It means that once a packet whose dst IP address is 10.0.0.1 from eth0 arrives at XDP layer, the XDP program will match the first entry of dstmactable and set output_port = 8 and pass the packet to TC layer. And all this actions don't have effect on the ebpf_outTable.

Second, user knows that once TC layer receives a packet , that packet must be the one which user wants to forward. In this case , only packet of which the dst ip address is equal to 10.0.0.1 will be passed to TC.
So , user updates the ebpf_outTable by adding <output_port , ifindex > to ebpf_outTable:

key	value
8	666

the 8 is the output port number which P4 program sets. And 666 is the eth8's ifindex.

Third, user writes another ebpf program to attach to TC layer. Once TC gets packets, it looks up the ebpf_outTable by using output port 8 as key which assigned in P4 program and gets the ifindex of NIC . After that , TC calls bpf_clone_redirect helper function to forward this packet to the eth8's egress.

And here is the question:
How would TC know the output port is 8 . I mean, in TC layer , TC only gets the raw packets and no other information. And how the P4 program transfer the output port information to TC . Or the user just knows it and sets the output port as a static number in TC's ebpf program ? But in this way, it seems the output_port field belongs to struct xout in P4 program is no use. User could assign any number as key because only the ifindex is valuable . And what if user wants to forward to a third interface, the method of setting the output port as a static number won't work well.
What I think is that if TC wants to know which interface egress should be selected to forward packets , TC must parse the packet and get the dst IP address. But that might seem like overkill.

[williamtu]

Hi,
Thanks a lot for your help. Do you have any simple ebpf program attched to TC which uses ebpf_outTable to send raw packets from one NIC to another ?

I don't have an exactly working example, but you can follow this sample code
https://elixir.bootlin.com/linux/latest/source/samples/bpf/tcbpf1_kern.c
which uses bpf_clone_redirect and specify ifindex for output packet

[williamtu]

And here is the question:
How would TC know the output port is 8 . I mean, in TC layer , TC only gets the raw packets and no other information. And how the P4 program transfer the output port information to TC . Or the user just knows it and sets the output port as a static number in TC's ebpf program ? But in this way, it seems the output_port field belongs to struct xout in P4 program is no use. User could assign any number as key because only the ifindex is valuable . And what if user wants to forward to a third interface, the method of setting the output port as a static number won't work well.
What I think is that if TC wants to know which interface egress should be selected to forward packets , TC must parse the packet and get the dst IP address. But that might seem like overkill.

TC doesn't need to know key is 8, all TC does is to get the first entry in the map and output the packet to the ifindex 888.
And every time before the packet pass XDP layer, the map entry will be updated for the coming packet arriving at TC layer. So we don't need to parse the packet again at the TC layer.

[maharishib]

Hi,
Is there any update on doing XDP_REDIRECT without using TC?

[mihaibudiu]

Technically I don't think any changes need to be made in the compiler; it's probably just the control-plane setup that needs to be adapted. I personally don't have a machine where I can test XDP; I don't know if @williamtu has time to build an example.

[maharishib]

Using code generated by p4c-xdp, I was not able to redirect.
Used redirect_map helper inside XDP code to redirect packets successfully.

[mihaibudiu]

Can you contribute an example?