Skip to content

Matrix multiplication example performed with OpenMP, OpenACC, BLAS, cuBLABS, and CUDA

License

Notifications You must be signed in to change notification settings

mnicely/computeWorks_examples

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

54 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

computeWorks_examples

Matrix multiplication example performed with OpenMP, OpenACC, BLAS, cuBLAS, and CUDA

Getting Started

This example requires the following packages:

  • CUDA Toolkit 10.1
  • PGI CE Compiler 19.10

Optional:

  • Eclipse IDE C/C++
  • Docker CE + NVIDIA-Docker v2
    • PGI Docker image
  • Jupyter Notebook

The following installation instructions have been tested on Ubuntu 18.04 and CUDA 10.0+.

OpenACC profiling with NVIDIA driver 418.67 and above requires elevated permissions. See here.

You can achieve this one of two ways.

  1. Run command with sudo
sudo LD_LIBRARY_PATH=/usr/local/cuda/extra/CUPTI/lib64:$LD_LIBRARY_PATH ./computeWorks_mm
  1. Following Administration instructions.
sudo systemctl isolate multi-user # Stop the window manager
sudo su # Switch to root
modprobe -r nvidia_uvm nvidia_drm nvidia_modeset nvidia-vgpu-vfio nvidia # Unload dependent modules
cd /etc/modprobe.d/
touch nvidia.conf # Create file named nvidia.conf
echo -e "options nvidia "NVreg_RestrictProfilingToAdminUsers=0"" > nvidia.conf
reboot

Installation

CUDA -> more details

  1. Download CUDA Toolkit
  2. Install (assuming file is in ~/Downloads)
sudo dpkg -i ~/Downloads/cuda-repo-ubuntu1804-10-1-local-10.1.168-418.67_1.0-1_amd64.deb
sudo apt-key add /var/cuda-repo-<version>/7fa2af80.pub
sudo apt-get update
sudo apt-get install cuda
  1. Add paths to ~/.bashrc
echo -e "\n# CUDA paths" >> ~/.bashrc
echo -e "export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}" >> ~/.bashrc
echo -e "export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}" >> ~/.bashrc
echo -e "export LD_LIBRARY_PATH=/usr/local/cuda/extras/CUPTI/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}" >> ~/.bashrc
  • The CUPTI directory is required for OpenACC profiling

PGI Community Edition (Bare Metal) -> more details

Skip this step if you prefer to run utilize the PGI compiler in a Docker container.

  1. Download PGI CE Compiler
  2. Install (assuming file is in ~/Downloads)
export PGI_SILENT=true
export PGI_ACCEPT_EULA=accept
export PGI_INSTALL_DIR=/opt/pgi
export PGI_INSTALL_TYPE=single
export PGI_INSTALL_NVIDIA=true
export PGI_INSTALL_JAVA=true
export PGI_INSTALL_MPI=false
export PGI_MPI_GPU_SUPPORT=false
mkdir -p ~/Downloads/tmp
tar xpfz ~/Downloads/pgilinux-2019-1910-x86-64.tar.gz -C ~/Downloads/tmp
sudo -E ~/Downloads/tmp/install
rm -rf ~/Downloads/tmp
  1. Add paths to ~/.bashrc
echo -e "\n# PGI paths" >> ~/.bashrc
echo -e "export PGI=/opt/pgi" >> ~/.bashrc
echo -e "export PATH=/opt/pgi/linux86-64/19.10/bin:$PATH" >> ~/.bashrc
echo -e "export MANPATH=$MANPATH:/opt/pgi/linux86-64/19.10/man" >> ~/.bashrc
echo -e "export LM_LICENSE_FILE=$LM_LICENSE_FILE:/opt/pgi/license.dat" >> ~/.bashrc

Eclipse

  1. Download Eclipse IDE C/C++
  2. Install (assuming file is in ~/Downloads)
sudo tar xpfz ~/Downloads/eclipse-cpp-2019-03-R-linux-gtk-x86_64.tar.gz -C /opt
sudo ln -s /opt/eclipse/eclipse /usr/local/bin/eclipse
  1. Install Nsight Eclipse Plugin -> more details
bash /usr/local/cuda/bin/nsight_ee_plugins_manage.sh install /opt/eclipse

Docker -> more details

  1. Remove older Docker versions
sudo apt remove docker docker-engine docker.io containerd runc -y
  1. Install Docker CE -> more details
sudo apt update
sudo apt install apt-transport-https ca-certificates curl software-properties-common -y
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt update
sudo apt install docker-ce -y
  1. Enable Docker commands without sudo
sudo usermod -aG docker $USER
  1. Log out and back in
  • Confirm $USER is in the docker group
groups

mnicely adm cdrom sudo dip plugdev lpadmin sambashare docker

  1. Verify docker runs without sudo
docker container run hello-world

Docker_Hello_World

NVIDIA Docker v2 -> more details

  1. Remove nvidia-docker v1, if installed.
docker volume ls -q -f driver=nvidia-docker | xargs -r -I{} -n1 docker ps -q -a -f volume={} | xargs -r docker rm -f
sudo apt-get purge nvidia-docker
  1. Add nvidia-docker v2 repository
curl -sL https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -sL https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
  1. Install nvidia-docker v2
sudo apt-get install nvidia-docker2 -y
  1. Reload Docker daemon
sudo pkill -SIGHUP dockerd
  1. (Optional) Modify Docker daemon to storage images in /home versus /var. Usually /home has more space.
sudo nano /etc/docker/daemon.json
{
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    },
    "experimental": true,
    "graph": "/home/<whoami>/.docker",
    "storage-driver": "overlay2"
}
sudo service docker restart
  1. Verify you can launch docker container with access to GPU
docker run --runtime=nvidia --rm nvcr.io/nvidia/cuda:latest nvidia-smi

PGI Community Edition (Docker Image)

This create a Docker image containing the PGI CE Compiler.

  1. Download git project
git clone https://github.com/mnicely/computeWorks_examples.git
cd computeWorks_examples/computeWorks_mm/pgi_build
  1. Download PGI CE Compiler
  2. Move PGI tar to Dockerfile directory (assuming file is in ~/Downloads)
cp ~/Downloads/pgilinux-2019-194-x86-64.tar.gz .
  1. Build PGI Docker image
docker build -t cuda-10.1_ubuntu-18.04_pgi-19.10 -f Dockerfile.cuda-10.1_ubuntu-18.04_pgi-19.10 .

Jupyter Notebook

  1. Install PIP package manager
sudo apt install python-pip3
  1. Install JupyterLab
sudo -H pip3 install jupyter

Usage

Bare Metal

  • This approach requires PGI Community Edition (Bare Metal)
  1. Download git project
git clone https://github.com/mnicely/computeWorks_examples.git
cd computeWorks_examples/computeWorks_mm
  1. Build computeWorks_mm binary
make
  1. Run computeWorks_mm <matrixSize | default=1024>
./computeWorks_mm 128

Docker

  • This approach requires PGI Community Edition (Docker Image)
  1. Download git project
git clone https://github.com/mnicely/computeWorks_examples.git
cd computeWorks_examples/computeWorks_mm
  1. Build computeWorks_mm binary
docker run --runtime=nvidia --rm -v $(pwd):/workspace -w /workspace cuda-10.1_ubuntu-18.04_pgi-19.10:latest make
  1. Run computeWorks_mm <matrixSize | default=1024>
docker run --runtime=nvidia --rm -v $(pwd):/workspace -w /workspace cuda-10.1_ubuntu-18.04_pgi-19.10:latest ./computeWorks_mm 128

Eclipse IDE C/C++

Eclipse, with Nsight Eclipse Plugins offers full-featured IDE that provides an all-in-one integrated environment to edit, build, debug and profile CUDA-C applications.

  1. Open Eclipse
eclipse &
  1. Import Project
  • File -> Import...
  • Select -> Git -> Projects from Git -> [Next >]
  • Select Repository Store -> Clone URI -> [Next >]
  • Source Git Repository -> URL -> https://github.com/mnicely/computeWorks_examples -> [Next >]
  • Branch Selection -> [Next >]
  • Local Destination -> [Next >]
  • Select a wizard to use for importing projects -> Import existing Eclipse project -> [Next >]
  • Import Projects -> [Finish]

Baremetal

  • This approach requires PGI Community Edition (Bare Metal)
  1. Build Project
  • Right click computeWorks_mm -> Build Project or Press Ctrl + B
  1. Run Project
  • Right click computeWorks_mm -> Run As -> Local C/C++ Application

Docker container

  • This approach requires PGI Community Edition (Docker Image)
  1. Point to PGI Docker container
  • Right click computeWorks_mm -> Properties
  • C/C++ Build -> Settings
  • Settings -> Container Settings -> select Build inside Docker Image
  • Image -> cuda-10.1_ubuntu-18.04_pgi-19.10:latest
  1. Run Project
  • Right click computeWorks_mm -> Run As -> Local C/C++ Application

JupyterLab

  1. Open computeWorks_mm.ipynb
cd computeWorks_examples/computeWorks_mm/jupyter
jupyter-notebook computeWorks_mm.ipynb