Skip to content

Tutorial Part 2B: Add the new principal type

Jump to bottom
Dan Barrett edited this page Feb 8, 2017 · 1 revision

XIA GENI Tutorial

Add the new principal type: Load Balancing ID (LID)

Prerequisites: A running XIA network described in the previous part.

In this part we will show you how to add a new principal type, Load Balancing ID (LID), and a new forwarding semantic associated with it.

The goal of LID is to balance requests between service replicas. In this tutorial, we will just implement load balancing between two replicas of a simple UDP-based service.

We will go through the following steps

  • Start the service on both server0 and server1. Run the client on host0 to see the unicast output.
  • Add a new forwarding semantic on the forwarding engine. It randomly selects next hop out of two candidates
  • Add the new principal type: LID. Associate it with the new load balancing forwarding engine.
  • Manually add a route of that type on router1 as we have no routing protocol for it yet.
  • Run the client on host0. Contact the service via LID to see the load balancing behavior.
  1. Bring up two replicas of a simple UDP based service.
    • Start the simple GetTimeOfDay service in server0 window. 
      /xia-core/tutorial/lb_server &
    • executing this in host0 window. 
      /xia-core/tutorial/lb_client
      The client side will print the response of date and time:

      server0: Wed Oct 15 20:27:12 2014 GMT

      The server side will print the DAG (address) of the client:

      request from: DAG 0 - AD:1000000000000000000000000000000000000000 1 - HID:0000000000000000000000000000000000000000 2 - SID:30000ff0000000000000000000000000c8c1f026 (This SID is random)

    • Bring up another replica of that service on server1 in a new window 
      ssh -A server1
      Then 
      /xia-core/tutorial/lb_server &
      Now you can see from host0 the service replica on server1 responds all the request because it overwrite the service ID record on the name server.

      The name, lb.tutorial.xia, recorded on the name server is

    To this end, only on server is accessible if we use the fixed address with conventional principal types. Next, we will add the load balancing principal type to distributed load to both servers.
  2. Adding the new principal type LID.
    All the updates should be done in router1.
    • Login to router1
    • First we fix some permissions. 
      sudo chown -R `whoami` /xia-core
      Which XID uses which forwarding engine is defined in /xia-core/etc/click/xia_router_lib.click.
    • Open /xia-core/etc/click/xia_router_lib.click to upgrade to the new engine.
      • At line 44:

        c :: XIAXIDTypeClassifier(next AD, next HID, next SID, next CID, next IP, -);

        Change it by inserting next LID, to 
        
c :: XIAXIDTypeClassifier(next AD, next HID, next SID, next CID, next IP, next LID, -);
        This step basically tells XIA to identify the new XID type.
      • At line 70 and 71:

        rt_AD, rt_HID, rt_SID, rt_CID, rt_IP :: XIAXIDRouteTable($local_addr, $num_ports);
        c => rt_AD, rt_HID, rt_SID, rt_CID, rt_IP, [2]output;

        Change the code by inserting rt_LID :: XIANEWXIDRouteTable($local_addr, $num_ports); in the middle line and add rt_LID to the last line. 
        
rt_AD, rt_HID, rt_SID, rt_CID, rt_IP :: XIAXIDRouteTable($local_addr, $num_ports);
rt_LID :: XIANEWXIDRouteTable($local_addr, $num_ports);
c => rt_AD, rt_HID, rt_SID, rt_CID, rt_IP, rt_LID, [2]output;
        This code basically tell XIA to use a new routing,XIANEWXIDRouteTable, table for LID. We will define the table later.
      • At line 80 or 81

        rt_AD[0], rt_HID[0], rt_SID[0], rt_CID[0], rt_IP[0] -> GPRP;
        rt_AD[1], rt_HID[1], rt_CID[1], rt_IP[1] -> XIANextHop -> check_dest;
        rt_SID[1] -> XIANextHop -> XIAPaint($DESTINED_FOR_LOCALHOST) -> [1]output;
        rt_AD[2], rt_HID[2], rt_SID[2], rt_CID[2], rt_IP[2] -> consider_next_path;
        rt_AD[3], rt_HID[3], rt_CID[3], rt_IP[3] -> Discard;
        rt_SID[3] -> [3]output;
        rt_AD[4], rt_HID[4], rt_SID[4], rt_CID[4], rt_IP[4] -> x; // xcmp redirect message

        Added rt_LID[0,1,2,3,4] to the first, second, fourth, fifth, and seventh line. Change it to 
        
rt_AD[0], rt_HID[0], rt_SID[0], rt_CID[0], rt_IP[0], rt_LID[0] -> GPRP;         
rt_AD[1], rt_HID[1], rt_LID[1], rt_CID[1], rt_IP[1] -> XIANextHop -> check_dest;
                     rt_SID[1]                      -> XIANextHop -> XIAPaint($DESTINED_FOR_LOCALHOST) -> [1]output;
rt_AD[2], rt_HID[2], rt_SID[2], rt_CID[2], rt_IP[2], rt_LID[2] -> consider_next_path;
rt_AD[3], rt_HID[3], rt_LID[3], rt_CID[3], rt_IP[3] -> Discard;
                     rt_SID[3]                      -> [3]output;
rt_AD[4], rt_HID[4], rt_SID[4], rt_CID[4], rt_IP[4], rt_LID[4] -> x; // xcmp redirect message
      • At line 278, add a line of code 
        Script(write n/proc/rt_LID.add - $FALLBACK);
      Save this file. (If you have no time updating this file you could just cp /xia-core/tutorial/xia_router_lib.click /xia-core/click/conf/xia_router_lib.click to use the file we prepared for you.)
    • Finally, give the new type a unique type number by 
      echo "0x50 LID" > /xia-core/etc/xids
  3. Add a new forwarding engine: the binary load balancer
      We already prepared skeleton codes of a new forwarding engine for you. This source file is identical to the table based forwarding engine we've been using in XIA.
    • Design
      • The goal of the new engine is to balance the load between two service replicas.
      • The forwarding engine looks up the next hop port number for a XID. The port number is stored in a integer but only the several lowest bits are use because there are number of physical ports are usually less than 100. 
        
int port
               3               2               1               0
 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|C|                      Unused                   | HextHop port|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
C= Control bit
      • We could use the rest bits to store a second port number. We use the lowest 8 bits to store the port number of the service replica on server0 and the next lowest 8 bit for port number of server1. 
        
int port
               3               2               1               0
 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|C|                           |I| HextHop port 2| HextHop port 1|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
C= Control bit, I= Indicator bit
      • We randomly pick one port out of two when the engine looks up a LID from the table.
        for example, if we want to load balance between port 1 and port 2, the port number should be: 
        int port = 00000000 00000001  00000010  00000001 = 66049
                    two ports port2 = 2 port1 = 1
      Next we implement this idea.
    • We open the file /xia-core/click/elements/xia/xianewxidroutetable.cc for editing. Go to line 582 in that source file

      /* GENI TUTORIAL: EDIT BEGIN
      * route entry found
      * start forwarding
      */
      XIARouteData *xrd = (*it).second;
      int port = xrd->port;
      // check if outgoing packet
      if(xrd->port != DESTINED_FOR_LOCALHOST && xrd->port != FALLBACK && xrd->nexthop != NULL) {
          p->set_nexthop_neighbor_xid_anno(*(xrd->nexthop));
      }
      return port;
      /* GENI TUTORIAL: EDIT END
      */


    • We assume the hybrid port variable is set by some routing protocol and here we just try to interpret it.
    • The basic logic goes like this 
      IF port < (1<<16) THEN only portA is used, just select it
ELSE both portA and portB are used, randomly choose one from them
    • The C code of such logic is 
      
XIARouteData *xrd = (*it).second;
int port = xrd->port;
// check if outgoing packet
if(port < (1<<16) && xrd->port != DESTINED_FOR_LOCALHOST && xrd->port != FALLBACK && xrd->nexthop != NULL) {
    p->set_nexthop_neighbor_xid_anno(*(xrd->nexthop));
}
else if (port >= (1<<16)){
    click_chatter("Load balancing");
    int random = rand();
    port = port & 0xFFFF; //clean high 16 bits
    if (random%2 == 0){   //random selection
        port = port & 0xFF; //use lowest 8 bits
    }
    else{
        port = port >> 8; //use lowest 9-16 bits
    }
    p->set_nexthop_neighbor_xid_anno(_bcast_xid); // next hop could be any host
    click_chatter("go to port %d", port);
}
return port;
      That is all for the forwarding engine of the new load balancing LID. Save that file. (If you have no time updating this file you can just cp /xia-core/tutorial/xianewxidroutetable.cc /xia-core/click/elements/xia/xianewxidroutetable.cc to use the file we prepared for you.)
  4. Compile and restart the network
    On router1 
    cd /xia-core
make
tutorial/xnode restart
  5. Use this new engine: add route Now the new LID is deployed. We need to add routes telling router1 there are two service replicas.
    If we know server0 connects to port 0 of router1 and server1 connects to port 1. Then our hybrid port number should be 
    int port = 00000000 00000001  00000001  00000000 = 65792
           0s       two ports port2 = 1 port1 = 0
    Use bin/xroute to tell which two ports of server0 and server1 should use. 
    
router1:/xia-core$ bin/xroute
XIA Route version 0.9
router1
TYPE  XID                                           PORT          FLAGS    NEXT HOP
------------------------------------------------------------------------------------------------------------------------
...
AD    AD_NAMESERVER                                  0            0000ffff HID_ROUTER0
...
HID   HID_SERVER1                                    2            0000ffff HID_SERVER1
HID   HID_SERVER0                                    1            0000ffff HID_SERVER0
...
    In this case we should use port 1 and port 2. 
    port (1|2) = 66049
port (1|0) = 65792
port (2|0) = 66048
    • Use one of the three value above according to the ports you get. You can use any LID for the time being. We just pick a handy one for you. 
      
/xia-core/bin/xroute -a router1,LID,LID:0e00000000000000000000000000009876543210,65792
      Then we will see the route is added into router1 
      /xia-core/bin/xroute
      It should show 
      
router1
TYPE  XID                                           PORT          FLAGS    NEXT HOP
---------------------------------------------------------------------------------------
...
AD    AD_NAMESERVER                                  0            0000ffff HID_ROUTER0
...
LID   (default)                                     -7 (fallback) 00000000 
LID   lid:0e00000000000000000000000000009876543210  65792 (1|0)   00000000
      That (1|0) means your LID lid:0e00000000000000000000000000009876543210 is balancing the load between port 0 and 1. We already prepared the script to print both ports for you.
      If AD_NAMESERVER is not shown in the routing table. you need to login to router0 to restart that node 
      ssh router0
/xia-core/tutorial/xnode restart
  6. Using the LID to access your service
    • Now we login to host0.
    • Declear the new LID number. 
      cd /xia-core
echo "0x50 LID" > /xia-core/etc/xids
    • Restart the local network stack on host0 to make it effective. 
      tutorial/xnode restart
    • We want the client to reach the service replicas via the load balancing LID. So we update the name lb.tutorial.xia, associating it with a new DAG 
      
/xia-core/tutorial/addname lb.tutorial.xia "DAG 4 0 - AD:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb 4 2 - LID:0e00000000000000000000000000009876543210 4 - HID:6666666666666666666666666666666666666666 4 1 3 - HID:3333333333333333333333333333333333333333 4 - SID:0f00000000000000000000000000000123456789"
      This DAG basically means that to reach that Service ID (SID), one needs to go to router1 first (AD,HID), then use Load balancing ID (LID) to choose which service replica (SID) to go to and if LID is not supported, go to server0 as a fallback

      If the servers want clients to use the LID, they should use this DAG at the first place. But in this tutorial, we just update it for them.

    • Finally we verify the LID by running the lb_client multiple times 
      tutorial/lb_client
      You should see that about 50% of the requests are served by server0, the other 50% are served by server1.

Tutorial Part 3: Cleanup

XIA Research Home

Home

Prototype Overview

Running XIA

Development

Contact

Archive

Clone this wiki locally