Utilize the Docker container

CUDA Docker Container Setup and Usage Guide This tutorial covers how to build, run, attach, and detach a CUDA-enabled Docker container supporting three NVIDIA A6000 GPUs.

1. Install NVIDIA Docker Support

1.1 Install Docker (if not installed)

curl -fsSL https://get.docker.com | sh
sudo systemctl start docker
sudo systemctl enable docker

1.2 Install NVIDIA Container Toolkit

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) && \
sudo curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg && \
echo "deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://nvidia.github.io/libnvidia-container/$distribution stable" | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list && \
sudo apt-get update

sudo apt-get install -y nvidia-container-toolkit

Restart Docker:

sudo systemctl restart docker

Verify GPU support:

docker run --rm --gpus all nvidia/cuda:12.2.0-base nvidia-smi

If it shows available GPUs, the setup is correct.

2. Build a CUDA-Enabled Docker Image

Create a file named Dockerfile:

# Base image with CUDA 12.2 and cuDNN 8
FROM nvidia/cuda:12.2.0-devel-ubuntu22.04

# Set environment variables
ENV DEBIAN_FRONTEND=noninteractive
ENV PATH="/usr/local/cuda/bin:${PATH}"

# Install necessary packages
RUN apt-get update && apt-get install -y \
    git wget curl build-essential \
    python3 python3-pip python3-dev && \
    rm -rf /var/lib/apt/lists/*

# Install PyTorch with CUDA support
RUN pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

# Set working directory
WORKDIR /workspace

CMD ["/bin/bash"]

2.1 Build the Docker Image

docker build -t cuda-container .

3. Run the Container with GPU Support

3.1 Run the Container Using 3 NVIDIA A6000 GPUs

docker run --rm --gpus '"device=0,1,2"' -it cuda-container bash

• --gpus '"device=0,1,2"' → Allocates three A6000 GPUs.
• -it → Runs interactively.
• --rm → Deletes the container when exited.

3.2 Run a Persistent Container

If you want a reusable container:

docker run --gpus '"device=0,1,2"' --name cuda_env -it cuda-container bash

To restart later:

docker start -ai cuda_env

4. Attach & Detach from Running Containers

4.1 Detach from a Running Container

Inside the container, press:

Ctrl + P, Ctrl + Q

This keeps the container running in the background.

4.2 List Running Containers

docker ps

4.3 Reattach to a Running Container

Find the container ID using docker ps, then run:

docker attach <container_id>

Or if you named your container:

docker attach cuda_env

5. Stop & Remove Containers

5.1 Stop a Running Container

docker stop <container_id>

or

docker stop cuda_env

5.2 Remove a Stopped Container

docker rm <container_id>

This tutorial provides a complete workflow for building, running, detaching, and reattaching CUDA containers.

🚀 It is also possible to store the things created in the workspace in the disk space

By default, files created inside a Docker container are lost when the container is stopped unless you use volumes or bind mounts to store them on the host machine. Here’s how to ensure file persistence.

Option 1: Use a Volume (Recommended)

Docker volumes store data persistently across container restarts.

1.1 Create a Volume

docker volume create my_data

1.2 Run a Container with the Volume Mounted

docker run --gpus '"device=0,1,2"' -it --mount source=my_data,target=/workspace cuda-container bash

• Any file created inside /workspace will be saved in my_data and accessible even if the container is stopped or removed.

1.3 Reuse the Volume in a New Container

docker run --gpus '"device=0,1,2"' -it --mount source=my_data,target=/workspace cuda-container bash

The files remain intact.

Option 2: Use a Bind Mount (Maps a Folder from Host)

This method links a host directory to a container directory.

2.1 Create a Directory on the Host

mkdir -p ~/cuda_workspace

2.2 Run a Container with a Bind Mount

docker run --gpus '"device=0,1,2"' -it -v ~/cuda_workspace:/workspace cuda-container bash

• Files inside /workspace in the container are actually stored in ~/cuda_workspace on the host.
• Even if the container is removed, files remain in ~/cuda_workspace.

Option 3: Commit Changes to a New Image

If you want to save the state of a container, you can create a new Docker image from it.

3.1 Find the Running Container ID

docker ps

3.2 Commit the Container to a New Image

docker commit <container_id> cuda-container-saved

3.3 Start a New Container from the Saved Image

docker run --gpus '"device=0,1,2"' -it cuda-container-saved bash

Which Method Should You Use?

| Use Case | Method | |———-|——–| | Persist files across container runs | Volumes (--mount) ✅ | | Access files directly from the host | Bind mounts (-v) ✅ | | Save entire container state | Commit to a new image |

Recommended: Use Volumes (--mount) for best performance and flexibility.