Install CUDA is really annoying. So I made this guide to guide myself.
Basically we need to first install the correct Driver, then install the CUDA Toolkit and CuDNN.

Update: Installing CUDA in 2022

If you just want to use PyTorch with GPU, you do not need to manually install CUDA anymore.

You only need to install the correct driver. Then when installing PyTorch using conda, it automatically ships CUDA and cuDNN.

Reference: https://discuss.pytorch.org/t/is-it-required-to-set-up-cuda-on-pc-before-installing-cuda-enabled-pytorch/60181

1
2
You would only need to install the NVIDIA driver to run GPU workloads using the PyTorch binaries with the appropriately specified cudatoolkit version.
One limitation to this is that you would still need a locally installed CUDA toolkit to build custom CUDA extensions or PyTorch from source.

Do this if you already have CUDA and you want to downgrade it

1
2
3
4
5
6
7
8
# Remove existing CuDA versions
sudo apt --purge remove "cublas*" "cuda*"
sudo apt --purge remove "nvidia*"
sudo rm -rf /usr/local/cuda*
sudo apt-get autoremove && sudo apt-get autoclean

# Reboot to remove cached files 
reboot

Then after reboot

1
sudo apt-get clean

Now we can go to next section.

Install CUDA Toolkit

This example is installing CUDA 11.2 on Ubuntu 18.04 .

First config the latest driver.

Then reboot

1
reboot

Then do this command:

1
nvidia-smi

Check available CUDA version.

Go here https://developer.nvidia.com/cuda-toolkit-archive

Install the matching CUDA version

If you are downgrading your CUDA, do sudo apt-get install cuda-XXX (where XXX is the version you want)
Instead of sudo apt-get install cuda.

Do the Environment Setup part

First locate /usr/local/cuda-xxx where xxx is your CUDA installed version.

https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#post-installation-actions

You may need to add the Environment Setup into bottom part of ~/.bashrc as well.

Then do this command:

1
nvcc -V

Now you should see some info about your CUDA version (should match with your CUDA version in nvidia-smi)

If you see the versions not match, probably you done something wrong in the Environment Setup part

Install CuDNN

Go here https://developer.nvidia.com/cudnn

Click Download CuDNN

Just Login with Google.

Install the correct packages you needed (Correct Platform and Correct CUDA version)

Then refer to the Installation Guide.

Here I downloaded the Deb files for Ubuntu 18.04. (cuDNN Runtime Library + cuDNN Developer Library)

If you choose to install Tar file, you also need to follow the Installation Guide

Now you should have both CUDA and CuDNN.

Common Misconceptions: