Mirage OpenFlow Implementation

NB. This implementation is under development and is being extensively restructured. Recommend contacting richard.mortier@nottingham.ac.uk before attempting to make use of it.

Build notes

To regenerate oasis files:

$ oasis setup-clean
$ oasis setup

To build:

$ make configure ## needed to run `mirage configure`
$ make depend    ## ensure dependencies for mirage targets installed
$ make install   ## install the openflow libraries per findlib
$ make           ## build (if openflow libraries are already installed)

Usage

Current Openflow module contains an implementation of Openflow switch, and can be run under Mirage. The configuration of switch is stored in the file mirage/switch/config.ml, and main module is defined in mirage/switch/unikernel.ml. The file unikernel.ml contains a simple example of a running switch with two ports, and also remote controller's IP address and tcp port.

Building the switch generates the executable mirage/switch/mir-ofswitch. To run the switch:

$ sudo mirage/switch/mir-ofswitch

###Switch port definition

To add a switch port, modify both config.ml and unikernel.ml in the following way:

config.ml:

in main variable declaration and for every port to be added, add network to the arguments using combinator @-> and register the device.

unikernel.ml:

Add arguments to the Main module--for example (Nx: NETWORK) where x is the device (switch port) number--that matches main variable declaration in config.ml.
Add arguments to start in the Main module that again matches main variable declaration in config.ml.
Add network devices that appear in the arguments of start to the list netl.

###Controller

In unikernel.ml, set the ip address and tcp port of the remote controller using contaddr and contport.

Background

OpenFlow is a switching standard and open protocol enabling distributed control of the flow tables contained within Ethernet switches in a network. Each OpenFlow switch has three parts:

A datapath, containing a flow table, associating set of actions with each flow entry;
A secure channel, connecting to a controller; and
The OpenFlow protocol, used by the controller to talk to switches.

Following this standard model, the implementation comprises three parts:

Openflow library, contains a complete parsing library in pure Ocaml using an event-driven model.
Openflow.switch library, provides a skeleton OpenFlow switch supporting most elementary switch functionality.

N.B. _The implementation currently supports wire protocol 0x01.

Openflow.Ofpacket

The file begins with some utility functions, operators, types. The bulk of the code is organised following the v1.0.0 protocol specification. Each set of messages is contained within its own module, most of which contain a type t representing the entity named by the module, plus relevant parsers to convert a bitstring to a type (parse_*) and pretty printers for the type (string_of_*). At the end of the file, in the root Ofpacket module scope, are definitions for interacting with the protocol as a whole, e.g., error codes, OpenFlow message types and standard header, root OpenFlow parser, OpenFlow packet builders.

Queue, Port, Switch

The Queue module is really a placeholder currently. OpenFlow defines limited quality-of-service support via a simple queuing mechanism. Flows are mapped to queues attached to ports, and each queue is then configured as desired. The specification currently defines just a minimum rate, although specific implementations may provide more.

The Port module wraps several port related elements:

t, where that is either simply the index of the port in the switch, or the special indexes (> 0xff00) representing the controller, flooding, etc.
config, a specific port's configuration (up/down, STP supported, etc).
features, a port's feature set (rate, fiber/copper, etc).
state, a port's current state (up/down, STP learning mode, etc).
phy, a port's physical details (index, address, name, etc).
stats, current statistics of the port (packet and byte counters, collisions, etc).
reason and status, for reporting changes to a port's configuration; reason is one of ADD|DEL|MOD.

Finally, Switch wraps elements pertaining to a whole switch, that is a collection of ports, tables (including the group table), and the connection to the controller.

capabilities, the switch's capabilities in terms of supporting IP fragment reassembly, various statistics, etc.
action, the types of action the switch's ports support (setting various fields, etc).
features, the switch's id, number of buffers, tables, port list etc.
config, for masking against handling of IP fragments: no special handling, drop, reassemble.

Wildcards, Match, Flow

The Wildcards and Match modules both simply wrap types respectively representing the fields to wildcard in a flow match, and the flow match specification itself.

The Flow module then contains structures representing:

t, the flow itself (its age, activity, priority, etc); and
stats, extended statistics association with a flow identified by a 64 bit cookie.

Packet_in, Packet_out

These represent messages associated with receipt or transmission of a packet in response to a controller initiated action.

Packet_in is used where a packet arrives at the switch and is forwarded to the controller, either due to lack of matching entry, or an explicit action.

Packet_out contains the structure used by the controller to indicate to the switch that a packet it has been buffering must now have some actions performed on it, typically culminating in it being forward out of one or more ports.

Flow_mod, Port_mod

These represent modification messages to existing flow and port state in the switch.

Stats

Finally, the Stats module contains structures representing the different statistics messages available through OpenFlow, as well as the request and response messages that transport them.

Openflow.Ofsocket

A simple Make functor that accepts a TCP module as its argument to create an openflow channel abstraction over a series of different transport mechanisms. The protocol ensures to read from the socket full Openflow packets and transform them to appropriate Ofpacket structures.

Openflow.Ofswitch

An OpenFlow switch or datapath consists of a flow table, a group table (in later versions, not supported in v1.0.0), and a channel back to the controller. Communication over the channel is via the OpenFlow protocol, and is how the controller manages the switch.

In short, each table contains flow entries consisting of match fields, counters, and instructions to apply to packets. Starting with the first flow table, if an incoming packet matches an entry, the counters are updated and the instructions carried out. If no entry in the first table matches, (part of) the packet is forwarded to the controller, or it is dropped, or it proceeds to the next flow table.

At the current point the switch doesn't support any queue principles.

Skeleton code is as follows:

Entry

Represents a single flow table entry. Each entry consists of:

counters, to keep statistics per-table, -flow, -port, -queue (Entry.table_counter list, Entry.flow_counter list, Entry.port_counter list, Entry.queue_counter list); and
actions, to perform on packets matching the fields (Entry.action list).

Switch

Encapsulating the switch (or datapath) itself. It defines Table as a simple module representing a table of flow entries. Currently just an id (tid) , a hashtbl of entries ((OP.Match.t, Entry.t) Hashtbl.t), a list of exact match entries to reduce the lookup time for wildcard entries and a the table counter.

It currently defines a port as:

details, a physical port configuration (Ofpacket.Port.phy); and
device, some handle to the physical device (mocked out as a string).

The switch is then modelled as:

ports, a list of physical ports (Switch.port list);
table, the table of flow entries for this switch;
stats, a set of per-switch counters (Switch.stats); and
p_sflow, the probability in use when sFlow sampling.

Note that the vocabulary of a number of these changes with v1.1.0, in addition to the table structure becoming more complex (support for chains of tables, forwarding to tables, and the group table).

DEPRECATED

You can also find old release of Mirage OpenFlow in this repository. Following is the information about deprecated Mirage OpenFlow. The old release has the implementation of an standalone openflow controller.

Usage

API

The source code contains 3 main libraries: Openflow, Openflow.Switch and Openflow.Flv.

The Openflow module contains all the code to parse, print and generate openflow messages, as well as, a basic openflow control platform. The ofcontroller implements an openflow controller library. The library is event driven. The programmer can access any openflow message by registering event callback during the init phase of the code for every connected switch. The parsing uses cstruct.t objects.

The Openflow.Switch module implements an openflow switch. The module exposes a simple API through which a user can add and remove ports to the controller and run the default openflow processing switch functionality. In addition, the module contains an out-of-channel mechanism to modify the state of the switch using a json-rpc mechanism and insert, delete and view flow entries or enable or disable network ports. Finally, the switch provides a standalone mode, when the controller becomes unavailable, using a local learning switch logic, implemented in module Openflow.Switch.Ofswitch_standalone. Standalone functionality can be initiated through the Ofswitch.standalone_connect method.

Additionally the library contains a small number of helper functions that enhance the functionality of openflow application. Ofswitch_config is a daemon that exposes a json API through which other apps can have configure the ports of the switch and access the content of the datapath table, using a simple tcp socket. Ofswitch_standalone is a minimum learning switch implementation over openflow that can be enabled on the switch module when no controller is accessible.

The Openflow.Flv library re-implements the functionality provided by the flowvisor switch virtualisation software. FLowvisor is able to aggregate multiple switches and expose them to controller as a single switch, aggregating all the ports of the switches. The module provides elementary slicing functionality using wildcards. Additionally, the module implements a simple switch topology discovery mechanism using the lldp protocol. The functionality of this module is currently experimental and the library is not fully functional (e.g. OP.Port.Flood output actions are not supported by the module ).

Programs

The source code of the library contains a number of small appliances that provide simple examples over the functionality of the library.

lwt_switch

This is a unix backend implementation of an openflow switch. The application exposes both the json config web service and uses the standalone embedded controller. The application tries to connect to localhost in order to connect to controller and also run the json-based configuration daemon on port 6634.

lwt_controller

An openflow controller that implements a simple openflow-based learning switch. The program listens on port 6633.

ofswitch_ctr

This is a simple implementation of a configuration client for the switch code. The application has a lot of similarities with the syntax of the ovs-vsctl code. Users can access and modify the state of the switch with the following command line parameters:

dump-flows intf tupple: this command will match all flows on the forwarding table of the switch and return a dump of the matching flows to the provided tupple.
del-flows intf tupple: delete matching flows.
add-flows intf tupple: adding a tupple to the flow table.
add-port intf network_device : adding a port under the control of the openflow switch.
del-port intf network_device : remove a port from the control of the switch.

ofswitch.xen

A unikernel appliance of the lwt_switch for the xen backend.

ofcontroller.xen

A unikernel application of the lwt_controller for the xen backend.

Background

OpenFlow is a switching standard and open protocol enabling distributed control of the flow tables contained within Ethernet switches in a network. Each OpenFlow switch has three parts:

A datapath, containing a flow table, associating set of actions with each flow entry;
A secure channel, connecting to a controller; and
The OpenFlow protocol, used by the controller to talk to switches.

Following this standard model, the implementation comprises three parts:

Openflow library, contains a complete parsing library in pure Ocaml and a minimal controller library using an event-driven model.
Openflow.switch library, provides a skeleton OpenFlow switch supporting most elementary switch functionality.
Openflow.flv library, implements a basic FLowVisor re-implementation in OCaml.

N.B. There are two versions of the OpenFlow protocol: v1.0.0 (0x01 on the wire) and v1.1.0 (0x02 on the wire). The implementation supports wire protocol 0x01 as this is what is implemented in Open vSwitch, used for debugging.

Openflow.Ofpacket

The file begins with some utility functions, operators, types. The bulk of the code is organised following the v1.0.0 protocol specification, as implemented by Open vSwitch v1.2. Each set of messages is contained within its own module, most of which contain a type t representing the entity named by the module, plus relevant parsers to convert a bitstring to a type (parse_*) and pretty printers for the type (string_of_*). At the end of the file, in the root Ofpacket module scope, are definitions for interacting with the protocol as a whole, e.g., error codes, OpenFlow message types and standard header, root OpenFlow parser, OpenFlow packet builders.

Queue, Port, Switch

The Queue module is really a placeholder currently. OpenFlow defines limited quality-of-service support via a simple queuing mechanism. Flows are mapped to queues attached to ports, and each queue is then configured as desired. The specification currently defines just a minimum rate, although specific implementations may provide more.

The Port module wraps several port related elements:

t, where that is either simply the index of the port in the switch, or the special indexes (> 0xff00) representing the controller, flooding, etc.
config, a specific port's configuration (up/down, STP supported, etc).
features, a port's feature set (rate, fiber/copper, etc).
state, a port's current state (up/down, STP learning mode, etc).
phy, a port's physical details (index, address, name, etc).
stats, current statistics of the port (packet and byte counters, collisions, etc).
reason and status, for reporting changes to a port's configuration; reason is one of ADD|DEL|MOD.

Finally, Switch wraps elements pertaining to a whole switch, that is a collection of ports, tables (including the group table), and the connection to the controller.

capabilities, the switch's capabilities in terms of supporting IP fragment reassembly, various statistics, etc.
action, the types of action the switch's ports support (setting various fields, etc).
features, the switch's id, number of buffers, tables, port list etc.
config, for masking against handling of IP fragments: no special handling, drop, reassemble.

Wildcards, Match, Flow

The Wildcards and Match modules both simply wrap types respectively representing the fields to wildcard in a flow match, and the flow match specification itself.

The Flow module then contains structures representing:

t, the flow itself (its age, activity, priority, etc); and
stats, extended statistics association with a flow identified by a 64 bit cookie.

Packet_in, Packet_out

These represent messages associated with receipt or transmission of a packet in response to a controller initiated action.

Packet_in is used where a packet arrives at the switch and is forwarded to the controller, either due to lack of matching entry, or an explicit action.

Packet_out contains the structure used by the controller to indicate to the switch that a packet it has been buffering must now have some actions performed on it, typically culminating in it being forward out of one or more ports.

Flow_mod, Port_mod

These represent modification messages to existing flow and port state in the switch.

Stats

Finally, the Stats module contains structures representing the different statistics messages available through OpenFlow, as well as the request and response messages that transport them.

Openflow.Ofsocket

A simple module to create an openflow channel abstraction over a series of different transport mechanisms. At the moment the library contains support of Channel.t connections and Lwt_stream streams. The protocol ensures to read from the socket full Openflow packets and transform them to appropriate Ofpacket structures.

Openflow.Ofontroller

Initially modelled after NOX, this is a skeleton controller that provides a simple event based wrapper around the OpenFlow protocol. It currently provides the minimal set of events corresponding to basic switch operation:

DATAPATH_JOIN, representing the connection of a datapath to the controller, i.e., notification of the existence of a switch.
DATAPATH_LEAVE, representing the disconnection of a datapath from the controller, i.e., notification of the destruction of a switch.
PACKET_IN, representing the forwarding of a packet to the controller, whether through an explicit action corresponding to a flow match, or simply as the default when flow match is found.
FLOW_REMOVED, i.e., representing the switch notification regarding the removal of a flow from the flow table.
FLOW_STATS_REPLY, i.e., represents the replies transmitted by the switch after a flow_stats_req.
AGGR_FLOW_STATS_REPLY, i.e., representing the reply transmitted by the switch to an aggr_flow_stats_req.
DESC_STATS_REPLY, i.e., representing the reply of a switch to desc_stats request.
PORT_STATS_REPLY, i.e., representing the replt of a switch to a port_stats request providing port level counter and the state of the switch.
TABLE_STATS_REPLY, i.e., representing the reply of a switch to a table_stats request.
PORT_STATUS_REPLY, i.e., representing the notification send by the switch when the state of a port of the switch is changed.

The controller state is mutable and modelled as:

A list of callbacks per event, each taking the current state, the originating datapath, and the event;
Mappings from switch (datapath_id) to a Mirage communications channel (Channel.t); and

The main work of the controller is carried out in process_of_packet which processes each received packet within the context given by the current state of the switch: this is where the OpenFlow state machine is implemented.

The controller entry point is via the listen, local_connect or connect function which effectively creates a receiving channel to parse OpenFlow packets, and pass them to process_of_packet which handles a range of standard protocol-level interactions, e.g., ECHO_REQ, FEATURES_RESP, generating Mirage events as appropriate. Specifically, controller is passed as callback to the respective connection method, and recursively evaluates read_packet to read the incoming packet and pass it to process_of_packet.

Openflow.Switch.Ofswitch

An OpenFlow switch or datapath consists of a flow table, a group table (in later versions, not supported in v1.0.0), and a channel back to the controller. Communication over the channel is via the OpenFlow protocol, and is how the controller manages the switch.

In short, each table contains flow entries consisting of match fields, counters, and instructions to apply to packets. Starting with the first flow table, if an incoming packet matches an entry, the counters are updated and the instructions carried out. If no entry in the first table matches, (part of) the packet is forwarded to the controller, or it is dropped, or it proceeds to the next flow table.

At the current point the switch doesn't support any queue principles.

Skeleton code is as follows:

Entry

Represents a single flow table entry. Each entry consists of:

counters, to keep statistics per-table, -flow, -port, -queue (Entry.table_counter list, Entry.flow_counter list, Entry.port_counter list, Entry.queue_counter list); and
actions, to perform on packets matching the fields (Entry.action list).

Table

A simple module representing a table of flow entries. Currently just an id (tid) , a hashtbl of entries ((OP.Match.t, Entry.t) Hashtbl.t), a list of exact match entries to reduce the lookup time for wildcard entries and a the table counter.

Switch

Encapsulating the switch (or datapath) itself. Currently defines a port as:

details, a physical port configuration (Ofpacket.Port.phy); and
device, some handle to the physical device (mocked out as a string).

The switch is then modelled as:

ports, a list of physical ports (Switch.port list);
table, the table of flow entries for this switch;
stats, a set of per-switch counters (Switch.stats); and
p_sflow, the probability in use when sFlow sampling.

Note that the vocabulary of a number of these changes with v1.1.0, in addition to the table structure becoming more complex (support for chains of tables, forwarding to tables, and the group table).

Questions/Notes

What's the best way to structure the controller so that application code can introduce generation and consumption of new events? NOX permits this within a single event-handling framework -- is this simply out-of-scope here, or should we have a separate event-based programming framework available, or is there a straightforward Ocaml-ish way to incorporate this into the OpenFlow Controller?

What's the best way to expose parsing as a separate activity to reading data off the wire? Specifically, I'd really like to reuse functions from Net.Ethif, Net.Ipv4, etc to recover structure from the bitstring without need to have OfPacket.Match.parse_from_raw_packet. Previously I have found having parsers that return structured data and then wrapping up the packet structure as a nested type, e.g., PCAP(pcaph, ETH(ethh, IPv4(iph, payload))) or ...TCP(tcph, payload)))) worked well, permitting fairly natural pattern matching. The depth to which the packet was de-multiplexed was controlled by a parameter to the entry-point parser.

The Switch design is almost certainly very inefficient, and needs working on. This is waiting on implementation -- although sketched out, waiting on network driver model to actually be able to get hold of physical devices and frames. When we can, also need to consider how to control packet parsing, and de-multiplexing of frames for switching from frames comprising the TCP stream carrying the controller channel. Ideally, it would be transparent to have a Channel for the controller's OpenFlow messages and a per-device frame handler for everything else. That is, Mirage would do the necessary de-multiplexing -- but only what's necessary -- passing non-OpenFlow frames to the switch to be matched, but reassembling the TCP flow carrying the controller's OpenFlow traffic.

Testing

This setup describes using VirtualBox on OSX with Ubuntu images.

OSX Setup

Manually configure en3 on OSX to 172.16.0.1/255.255.255.0.
Setup bootpd on OSX: sudo /bin/launchctl load -w /System/Library/LaunchDaemons/bootps.plist

To unload: sudo /bin/launchctl unload -w /System/Library/LaunchDaemons/bootps.plist

Create /etc/bootpd.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Subnets</key>
    <array>
      <dict>
        <key>allocate</key>
        <true/>
        <key>lease_max</key>
        <integer>86400</integer>
        <key>lease_min</key>
        <integer>86400</integer>
        <key>name</key>
        <string>172.16.0</string>
        <key>net_address</key>
        <string>172.16.0.0</string>
        <key>net_mask</key>
        <string>255.255.255.0</string>
        <key>net_range</key>
        <array>
          <string>172.16.0.2</string>
          <string>172.16.0.254</string>
        </array>
      </dict>
    </array>
    <key>bootp_enabled</key>
    <false/>
    <key>detect_other_dhcp_server</key>
    <false/>
    <key>dhcp_enabled</key>
    <array>
      <string>en3</string>
    </array>
    <key>reply_threshold_seconds</key>
    <integer>0</integer>
  </dict>
</plist>

Create /etc/bootptab, eg.,

%%
# machine entries have the following format:
#
# hostname        hwtype  hwaddr            ipaddr     bootfile
greyjay-ubuntu-1  1       08:00:27:38:72:c6 172.16.0.11
greyjay-ubuntu-2  1       08:00:27:11:dd:a0 172.16.0.12

VirtualBox setup

Build two Ubuntu 10.04 LTS server (64 bit) image.
Set each VM to have two adaptors:
- eth0 bridged connected to en1 (or en0)
- eth1 bridged connected to en3

Ubuntu setup

Set ssh keys and adjust sshd_config setting to disallow passwords.

Install packages required to build Open vSwitch et al

apt-get install openssh-server git-core build-essential \
    autoconf libtool pkg-config libboost1.40-all-dev \
    libssl-dev swig

Pull and build Open vSwitch:

git clone git://openvswitch.org/openvswitch
cd openvswitch/
./boot.sh
./configure --with-linux=/lib/modules/`uname -r`/build
make -j6
make && sudo make install
cd ..

and NOX:

git clone git://noxrepo.org/nox
cd nox
./boot.sh
../configure
make -j5

Install the kernel module: sudo insmod ~/openvswitch/datapath/linux/openvswitch_mod.ko

Setup Open vSwitch:

sudo ovsdb-server ./openvswitch/ovsdb.conf --remote=punix:/var/run/ovsdb-server
ovsdb-tool create ovsdb.conf vswitchd/vswitch.ovsschema
sudo ovs-vswitchd unix:/var/run/ovsdb-server
sudo ovs-vsctl --db=unix:/var/run/ovsdb-server init
sudo ovs-vsctl --db=unix:/var/run/ovsdb-server add-br dp0
sudo ovs-vsctl --db=unix:/var/run/ovsdb-server set-fail-mode dp0 secure
sudo ovs-vsctl --db=unix:/var/run/ovsdb-server set-controller dp0 tcp:172.16.0.1:6633
sudo ovs-vsctl --db=unix:/var/run/ovsdb-server add-port dp0 eth0

Set IP addresses on the interfaces:

sudo ifconfig eth0 0.0.0.0
sudo ifconfig dp0 <whatever-eth0-was>

Name		Name	Last commit message	Last commit date
Latest commit History 255 Commits
controller		controller
lib		lib
mirage		mirage
switch		switch
.gitignore		.gitignore
.merlin		.merlin
.ocp-indent		.ocp-indent
CHANGES		CHANGES
Makefile		Makefile
Makefile.config		Makefile.config
Makefile.local		Makefile.local
README.md		README.md
_oasis		_oasis
_tags		_tags
configure		configure
myocamlbuild.ml		myocamlbuild.ml
setup.ml		setup.ml

NaaS/ocaml-openflow

Folders and files

Latest commit

History

Repository files navigation

Mirage OpenFlow Implementation

Build notes

Usage

Background

Openflow.Ofpacket

Queue, Port, Switch

Wildcards, Match, Flow

Packet_in, Packet_out

Flow_mod, Port_mod

Stats

Openflow.Ofsocket

Openflow.Ofswitch

Entry

Switch

DEPRECATED

Usage

API

Programs

lwt_switch

lwt_controller

ofswitch_ctr

ofswitch.xen

ofcontroller.xen

Background

Openflow.Ofpacket

Queue, Port, Switch

Wildcards, Match, Flow

Packet_in, Packet_out

Flow_mod, Port_mod

Stats

Openflow.Ofsocket

Openflow.Ofontroller

Openflow.Switch.Ofswitch

Entry

Table

Switch

Questions/Notes

Testing

OSX Setup

VirtualBox setup

Ubuntu setup

About

Resources

Stars

Watchers

Forks

Languages