Unable to remove nvidia and cuda drivers

Question

Regardless of how I got here, I really just want to completely remove nvidia and cuda drivers and reeinstall them. But, no matter what I do, I can’t get rid of them.

Background

I’m on Ubuntu 22.04 and I was using a working nvidia-535 driver and successfully installed cuda 11-8 (using instructions here).

Now, for some (to me) unknown reason (possibly some update) cuda stopped working (i.e. gpu not utilized in e.g. stable diffusion) and nvidia-smi returned

Failed to initialize NVML: Driver/library version mismatch
NVML library version: 535.161

Hence wanted to try and remove everything cuda and nvidia related and start over, since I know it was working fine before.

I have tried

  • sudo apt remove --purge nvidia* returns
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'nvidia-driver-550-server' for glob 'nvidia*'
Note, selecting 'nvidia-firmware-535-535.154.05' for glob 'nvidia*'
Note, selecting 'nvidia-docker2' for glob 'nvidia*'

... [ca. 300 similar rows] ...

Package 'nvidia-egl-wayland-common' is not installed, so not removed
Note, selecting 'nvidia-settings' instead of 'nvidia-settings-binary'
Package 'nvidia-390' is not installed, so not removed
Note, selecting 'libnvtt-bin' instead of 'nvidia-texture-tools'
Package 'nvidia-libopencl1-dev' is not installed, so not removed
Package 'nvidia-current' is not installed, so not removed
Package 'nvidia-current-updates' is not installed, so not removed
Package 'nvidia-libopencl1' is not installed, so not removed
Package 'nvidia' is not installed, so not removed

... [ca. 500 similar rows] ...

Package 'nvidia-container-toolkit-base' is not installed, so not removed
Package 'nvidia-firmware-545-545.29.02' is not installed, so not removed
Package 'nvidia-firmware-550-550.40.07' is not installed, so not removed
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 cuda-drivers-535 : Depends: nvidia-compute-utils-535 (>= 535.129.03) but it is not going to be installed
                    Depends: nvidia-dkms-535 (>= 535.129.03)
                    Depends: nvidia-driver-535 (>= 535.129.03) but it is not going to be installed
                    Depends: nvidia-kernel-common-535 (>= 535.129.03) but it is not going to be installed
                    Depends: nvidia-kernel-source-535 (>= 535.129.03) but it is not going to be installed or
                             nvidia-kernel-open-535 (>= 535.129.03) but it is not going to be installed
                    Depends: nvidia-utils-535 (>= 535.129.03) but it is not going to be installed
                    Depends: nvidia-modprobe (>= 535.129.03) but it is not going to be installed
                    Depends: nvidia-settings (>= 535.129.03) but it is not going to be installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).
  • As suggested I tried:
    sudo apt --fix-broken install which returned:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  libsbc1 nvidia-firmware-535-535.146.02 nvidia-firmware-535-535.161.07
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  nvidia-kernel-common-535
The following packages will be upgraded:
  nvidia-kernel-common-535
1 upgraded, 0 newly installed, 0 to remove and 28 not upgraded.
2 not fully installed or removed.
Need to get 0 B/38,3 MB of archives.
After this operation, 45,1 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
(Reading database ... 256847 files and directories currently installed.)
Preparing to unpack .../nvidia-kernel-common-535_535.161.07-0ubuntu1_amd64.deb ...
Unpacking nvidia-kernel-common-535 (535.161.07-0ubuntu1) over (535.154.05-0ubuntu1) ...
dpkg: error processing archive /var/cache/apt/archives/nvidia-kernel-common-535_535.161.07-0ubuntu1_amd64.deb (--unpack)
:
 trying to overwrite '/lib/firmware/nvidia/535.161.07/gsp_ga10x.bin', which is also in package nvidia-firmware-535-535.1
61.07 535.161.07-0ubuntu0.22.04.1
dpkg-deb: error: paste subprocess was killed by signal (Broken pipe)
Errors were encountered while processing:
 /var/cache/apt/archives/nvidia-kernel-common-535_535.161.07-0ubuntu1_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
  • sudo apt autoremove gives:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 nvidia-dkms-535 : Depends: nvidia-kernel-common-535 (= 535.161.07-0ubuntu1) but 535.154.05-0ubuntu1 is installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).
  • sudo apt remove --purge "*cuda*" returns:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'libcuda-10.2-1' for glob '*cuda*'
Note, selecting 'libcudart.so.12' for glob '*cuda*'

... [ ca. 800 rows ] ...

Package 'libcudss0-cuda-12' is not installed, so not removed
Package 'libcudss0-dev-cuda-12' is not installed, so not removed
Package 'libcudss0-static-cuda-12' is not installed, so not removed
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 libcublas-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 libcufft-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                          cuda-cudart-11-2 but it is not installable
                 Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                          cuda-cudart-11-2 but it is not installable
                 Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 libcufile-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 libcurand-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 libcusolver-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                             cuda-cudart-11-2 but it is not installable
                    Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                             cuda-cudart-11-2 but it is not installable
                    Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 libcusparse-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                             cuda-cudart-11-2 but it is not installable
                    Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                             cuda-cudart-11-2 but it is not installable
                    Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 libnpp-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                        cuda-cudart-11-2 but it is not installable
               Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                        cuda-cudart-11-2 but it is not installable
               Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 libnvjpeg-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 nvidia-dkms-535 : Depends: nvidia-kernel-common-535 (= 535.161.07-0ubuntu1) but 535.154.05-0ubuntu1 is to be installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).
  • As described in another question, I tried to locate the uninstaller:
    sudo /usr/local/cuda-11.4/bin/cuda-uninstaller but there is no cuda or nvidia directory under /usr/local/.

  • I checked /etc/apt/sources.list, there is no instance of ‘nvidia’ or ‘cuda’, (i.e. cat /etc/apt/sources.list | grep nvidia returns nothing.

  • Following instructions in an additional question, sudo apt-get --purge remove "*cublas*" "cuda*" "nsight*" returned:

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'cublasmp-cuda-11' for glob '*cublas*'
Note, selecting 'cublasmp-cuda-12' for glob '*cublas*'

... [ >100 similar rows ] ...

Package 'libcublas-dev-12-4' is not installed, so not removed
Package 'nsight-compute-2024.1.0' is not installed, so not removed
Package 'nsight-systems-2023.4.4' is not installed, so not removed
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 libcufft-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                          cuda-cudart-11-2 but it is not installable
                 Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                          cuda-cudart-11-2 but it is not installable
                 Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 libcufile-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 libcurand-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 libcusolver-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                             cuda-cudart-11-2 but it is not installable
                    Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                             cuda-cudart-11-2 but it is not installable
                    Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 libcusparse-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                             cuda-cudart-11-2 but it is not installable
                    Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                             cuda-cudart-11-2 but it is not installable
                    Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 libnpp-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                        cuda-cudart-11-2 but it is not installable
               Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                        cuda-cudart-11-2 but it is not installable
               Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 libnvjpeg-11-8 : Depends: cuda-toolkit-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-config-common but it is not going to be installed or
                           cuda-cudart-11-2 but it is not installable
                  Depends: cuda-toolkit-11-8-config-common but it is not going to be installed
 nvidia-dkms-535 : Depends: nvidia-kernel-common-535 (= 535.161.07-0ubuntu1) but 535.154.05-0ubuntu1 is to be installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

In conclusion, no matter what I try to remove, it won’t let me because of unmet dependencies. I have tried starting to remove the last packages above (e.g. libcufft-11-8) one by one, but I just get a very similar output again listing other related packages.

Also,
sudo apt update returns:

Hit:1 https://download.docker.com/linux/ubuntu focal InRelease
Hit:2 http://se.archive.ubuntu.com/ubuntu jammy InRelease                                                              
Hit:3 http://se.archive.ubuntu.com/ubuntu jammy-updates InRelease                                                      
Hit:4 http://se.archive.ubuntu.com/ubuntu jammy-backports InRelease                                                    
Hit:5 https://dl.google.com/linux/chrome/deb stable InRelease                                                          
Hit:6 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64  InRelease                            
Get:7 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB]                                              
Hit:8 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu jammy InRelease                            
Hit:9 https://ppa.launchpadcontent.net/qbittorrent-team/qbittorrent-stable/ubuntu jammy InRelease
Get:10 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB]
Get:11 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [597 kB]
Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB]
Fetched 1 716 kB in 2s (1 033 kB/s)                               
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
29 packages can be upgraded. Run 'apt list --upgradable' to see them.
W: https://download.docker.com/linux/ubuntu/dists/focal/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
W: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.

and sudo apt upgrade doesn’t work:

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 nvidia-dkms-535 : Depends: nvidia-kernel-common-535 (= 535.161.07-0ubuntu1) but 535.154.05-0ubuntu1 is installed
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

System details

Ubuntu 22.04.4 LTS

Kernel: 6.5.0-21-generic

Asked By: ciru_4011

||

I just had this issue when running sudo apt --fix-broken install

Try running something like this to fix it

sudo dpkg --force-all -i /var/cache/apt/archives/nvidia-kernel-common-535_535.161.07-0ubuntu1_amd64.deb

sudo apt --fix-broken install

Then reboot the machine.

Answered By: PsyFer

Step by step:

sudo apt-get purge nvidia*

sudo apt remove nvidia-*

sudo rm /etc/apt/sources.list.d/cuda*

sudo apt-get autoremove && sudo apt-get autoclean

sudo rm -rf /usr/local/cuda*

If the commands above won’t help, boot via Grab menu and try them again.

P.S. Worked for me on multiple Ubuntu versions

Answered By: Siarhei

After simply manually setting the driver to Nouveau in "Software and Updates" and rebooting, sudo apt remove --purge nvidia*, sudo apt remove --purge "*cuda*", etc (as listed in my question and suggested by Siarhei) worked as expected and gave no errors.

Simple, I know, but previously (and on other systems) I have been able to do this without manually activating the Nouveau drivers and restarting, so I guess it just didn’t occur to me at the time.

(Thank you all for reopening and for your answers. Also, apologies for the typo.)

Answered By: ciru_4011