Getting Started Guide
* !!! WORK IN PROGRESS !!! *
This guide provides a reference for getting started with the NetFPGA-1G-CML (NF1-CML) and its reference designs in the GitHub repository. It provides a step-by-step guide to building and running the Reference NIC project on the NF1-CML (reference_nic_nf1_cml). Building other projects use mostly the same steps, save for changing the names to reference the correct project.
Hardware
A host system with a PCI Express-compatible motherboard and a dual-port NIC are needed to run the reference designs. During development and testing of the NF1-CML reference designs, the following components were used:
Software
- Operating System: Fedora 20 64-bit (kernel version 3.14.7 at the time of writing)
- Xilinx Tools - ISE Design Suite 14.6
You will need to register to gain access to the GitHub repository by visiting the Registration page and following the instructions there.
There are a number of packages that should be installed in Fedora 20. These can be installed via the command line using Fedora's package manager, yum. The following command will install necessary packages:
$ sudo yum install make git minicom
For navigating the repository, we also recommend downloading qgit, a graphical git tool, and for debugging, we recommend WireShark, a network analyzer:
$ sudo yum install qgit wireshark wireshark-gnome
To setup the hardware for the reference_nic_nf1_cml, plug the NF1-CML into a PCIe slot on your host machine, plug a PCIe power connector from the host machine's power supply to the power connector on the NF1-CML, and set the power switch on the NF1-CML to the ON position.
To begin, clone the CML fork of the NetFPGA-10G-live repository. The repository is located here (if you receive a 404 page not found, make sure you are logged in to GitHub and revisit the site). Copy the HTTPS clone URL from the right side of the page. Then, clone the repository to a working directory:
$ git clone https://github.com/cmlab/NetFPGA-10G-live.git
You will be prompted for your GitHub username and password during this process. We recommend cloning the repository instead of downloading the .zip file of a branch, since it is easier to navigate between branches and pull updates if necessary.
First, add the Xilinx tools to your environment PATH by sourcing the settings64.sh script in the Xilinx installation location. By default, the command is:
$ source /opt/Xilinx/14.6/ISE_DS/settings64.sh
Update the path as necessary if your Xilinx tools are installed in a different location.
Next, set up your environment with the paths required by the NetFPGA-10G-live build system. In the root directory of the repository is a file bashrc_addon_NetFPGA_10G. Edit this in the following way to match your system:
- NF_ROOT should point to the base directory of the repository
- NF_DESIGN_DIR should point to the folder of the project you wish to run. For this guide, we will use the reference_nic_nf1_cml, so
NF_DESIGN_DIR=${NF_ROOT}/projects/reference_nic_nf1_cml
The remaining paths should stay the same. Save the file, then source it from the command line to update your environment variables:
$ source bashrc_addon_NetFPGA_10G
To verify it worked properly, do the command $ echo $NF_ROOT, and it should print the path to the root directory of the repository on your system.
To build the hardware test libraries for the hardware and simulation tests:
$ cd $NF_ROOT
$ make hwtestlib
To build the IP cores used by the NF1-CML projects:
$ make cml_cores
This may take a few minutes, and you should see a line that says //Xilinx PCIe core installed. If you encounter any problems with this process, please consult the NetFPGA-1G-CML FAQ and the NetFPGA-1G-CML issue tracker to see if there is a solution to your problem.
To build the project, change to the reference_nic_nf1_cml project directory and build it using make:
$ cd $NF_DESIGN_DIR
$ make
This command will run through synthesis, implementation, place and route, and bitfile generation using the Xilinx tools. It will take some time to complete. If successful, the last 2 lines of the make output will be:
mkdir -p bitfiles
cp hw/implementation/download.bit bitfiles/reference_nic_nf1_cml.bit
When the process is complete, the bitstream (now located in reference_nic_nf1_cml/bitfiles) can be downloaded to the board with another make command:
$ make download
This will configure the FPGA by sending the bitstream data through the USB_PROG port J12. If successful, the last 4 lines of the make download output will be:
INFO:iMPACT - '1': Checking done pin....done.
'1: Programmed successfully.
Elapsed time = 62 sec.
rm -f bitfiles/download.bit
Now, it is time to tell the system what to do with the NF1-CML after it has been configured. First, reboot your machine so it will see the NF1-CML as a PCIe device during bootup. Do not power off your machine, or the NF1-CML will lose its FPGA configuration and it will need to be re-downloaded. To verify the machine sees the NF1-CML, after it has rebooted, perform the following command and verify the output is similar:
$ lspci | grep Xilinx
01:00.0 Ethernet controller: Xilinx Corporation Device 4244
If all looks good, we're ready to install the NF1-CML driver. Navigate to your repository location, re-source the Xilinx tools and bashrc_addon_NetFPGA_10G, rebuild the hardware test library (make hwtestlib), then navigate to the driver software location and build it:
$ cd $NF_DESIGN_DIR/sw/host/driver
$ make
Once the build is complete, the driver can be installed with:
$ sudo insmod nf10.ko
Initialize the Ethernet interfaces on the NF1-CML with:
$ sudo ifconfig nf0 up
$ sudo ifconfig nf1 up
$ sudo ifconfig nf2 up
$ sudo ifconfig nf3 up
To verify this process happened successfully, run:
$ ifconfig -a
and you should see nf0-3 as network devices in the output.
Each reference design in the repository has a set of tests that verify the design both on hardware or simulation with Xilinx's Isim. For now, we will focus on the hardware tests.
The tests are run using a Python script located at $NF_ROOT/tools/bin/nf_test.py. This script uses parameters and environment variables to run the test, as well as the hardware test library, so before starting ensure you have sourced bashrc_addon_NetFPGA_10G and built the hardware test library with make hwtestlib. In order to run a test from the reference design directory, the user must specify whether it is a hardware test or simulation (hw or sim), and the "major" and "minor" names of the test. For example, a hardware test would be run with:
su -c "./nf_test.py hw --major <major_name> --minor <minor_name>"
The "major" and "minor" test names are dependent on the reference design being targeted (as specified by the $NF_DESIGN_DIR environment variable). Note also that the su -c in the command will run the command as the root user, and will require entering the root password (NOT the user's sudo password, if applicable).
In this example, the both_loopback_maxsize test will be used, located in $NF_DESIGN_DIR/test. Before running the test, the PHYs/Ethernet ports must first be connected the way the test expects. Each test specifies a connections file from the $NF_DESIGN_DIR/test/connections directory. Which file is used can be found in the run.py file for each test, e.g. $NF_DESIGN_DIR/test/both_loopback_maxsize/run.py and looking for an instance of ('../connections/<filename>', []). The file is formatted as source:destination, so for the both_loopback_maxsizetest, NF0 will connect to NF3, and NF1 will connect to NF2. Then, navigate to thenf_test.py` script location, and run the test:
$ cd $NF_ROOT/tools/bin
$ su -c "./nf_test.py hw --major loopback --minor maxsize"
If the connections are correct, and the design is working properly, the final 3 lines of the test output will be:
Running global setup... PASS
Running test both_loopback_maxsize... PASS
Running global teardown... PASS