Follow these instructions to install Steam Headless on Ubuntu Server.
Note
This assumes that your Ubuntu Server has not be configured to run any desktop environment!
This will not work with Ubuntu Desktop.
Install docker-ce to your Ubuntu server following the official instructions.
Ensure you install the docker-compose-plugin mentioned within these instructions
The easiest way to get running with NVIDIA GPUs is to install the NVIDIA Container Toolkit.
Follow the official instructions for installing the container toolkit to your Ubuntu server with apt.
If you do this, ensure that when you configure Docker Compose in the next step you choose the nvidia runtime.
Alternately, it is possible to run the container without the NVIDIA runtime by uncommenting the /dev/nvidia devices in the Compose file.
Once you have installed docker, follow the Compose Files section and select the right configuration file for your hardware.