Step-by-Step Nvidia CUDA Toolkit Installation Guide: Ubuntu 24.04

“The NVIDIA CUDA Toolkit is a software development kit that helps users create GPU-accelerated applications. It includes libraries, compilers, debuggers, and optimization tools”. Since we have an Nvidia GPU we will install it as shown below.

Note: If you installed the nvidia-cuda-toolkit from default ubuntu noble repo, uninstall it first as that we do not want conflicts. Uninstall with “sudo apt purge nvidia-cuda* -y“. More info on ubuntu repo install vs nvidia repo install later in post.

# wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-ubuntu2404.pin

First we wget the Nvidia Cuda Repository pin as shown above. Then we move it to /etc/apt/preferences.d/ (shown below).

This is a configuration file that is used to prioritize packages from the NVIDIA CUDA repository when installing CUDA on a Linux system. Basically it tells apt where to get the Cuda Toolkit.

$ sudo mv cuda-ubuntu2404.pin /etc/apt/preferences.d/cuda-repository-pin-600

Now we wget the repo. Note that this is a pretty hefty .deb and may take a few minutes.

$ wget https://developer.download.nvidia.com/compute/cuda/12.8.0/local_installers/cuda-repo-ubuntu2404-12-8-local_12.8.0-570.86.10-1_amd64.deb

And install it.

$ sudo dpkg -i cuda-repo-ubuntu2404-12-8-local_12.8.0-570.86.10-1_amd64.deb
$ sudo cp /var/cuda-repo-ubuntu2404-12-8-local/cuda-*-keyring.gpg /usr/share/keyrings/

Finally we install the toolkit.

$ sudo apt-get update
$ sudo apt-get -y install cuda-toolkit-12-8
$ sudo apt install cuda-runtime-12-8

---

Update $Path

Add the the following to your path

/usr/local/cuda-12.8/bin

---

Installing the Nvidia Cuda toolkit from Ubuntu Vs Nvidia

The key differences between installing the NVIDIA CUDA Toolkit from the Ubuntu repositories vs. NVIDIA’s official website come down to version availability, ease of installation, and compatibility. Here’s are a few differences:

1. Installing from Ubuntu Repositories (apt install nvidia-cuda-toolkit)

Pros:

• Easier installation – The package is well-integrated into Ubuntu’s package management (apt), meaning dependency resolution is generally smoother.
• More stable – The versions in the repositories are tested against the Ubuntu ecosystem to ensure compatibility.
• Automatic updates – Managed through apt, making it easier to keep up with system updates.

Cons:

• Often outdated – Ubuntu’s repositories may not have the latest CUDA version, which can be an issue if you need newer features or performance improvements.
• Limited driver version control – The version of CUDA available in the Ubuntu repos may not always match the latest NVIDIA drivers, which can lead to compatibility issues.

---

2. Installing from NVIDIA’s Official Website

Pros:

• Latest version – You get the newest CUDA Toolkit, which can include performance improvements, bug fixes, and new features.
• Better control over versions – You can install specific versions to match requirements for AI/ML frameworks like TensorFlow or PyTorch.
• More driver compatibility – NVIDIA ensures that the toolkit and drivers are aligned, reducing the risk of version mismatches.

Cons:

• More manual setup – You have to install dependencies manually, such as the correct NVIDIA driver, and configure paths properly.
• Updates are not automatic – Unlike apt, updates need to be managed manually.