Using GPU passthrough with Proxmox VE allows you to dedicate a GPU to a virtual machine (VM), providing a clean and efficient way to run GPU-accelerated applications in a virtualized enviroment.

For me, I use GPU passthrough to run ollama in a Debian VM. Ollama is a self-hosted LLM (Large Language Model) server. I can now run LLMs with my own data without sharing it with a third-party server (like OpenAI).

Anyway, this is a note on how I set up GPU passthrough for Proxmox VE.

Prerequisites

My configuration:

• MB: Gigabyte Z690
• CPU: Intel i9-12900
• GPU: Nvidia RTX 3060 Ti
• OS: Proxmox VE 8.2

Enable VT-d

First, you need to make sure enable virtualization in your BIOS. Be sure to turn on "VT-d" or something. Check out this video if you're not sure how to do it.

Activating IOMMU

Then you need to enable IOMMU in the kernel.

If you're using GRUB as your bootloader, you can edit /etc/default/grub, and check the line GRUB_CMDLINE_LINUX_DEFAULT. Add the following arguments to your bootfile:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"

Then update GRUB:

update-grub

If you're using systemd-boot as your bootloader, then check your /etc/kernel/cmdline:

root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet intel_iommu=on iommu=pt

Then update systemd-boot:

pve-efiboot-tool refresh

Add VFIO Modules

Edit /etc/modules and add the following modules by running the following command:

echo "vfio" >> /etc/modules
echo "vfio_iommu_type1" >> /etc/modules
echo "vfio_pci" >> /etc/modules

Edit your interrupt settings by running:

echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/vfio.conf
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf

Then update your initramfs:

update-initramfs -u -k all

Then reboot your system.

Check if IOMMU, VFIO Modules are Enabled

After rebooting, you can check dmesg to see if IOMMU and VFIO modules are enabled.

First, by running the following command, you can check if IOMMU is enabled:

dmesg | grep -e DMAR -e IOMMU

You should see something like:

ACPI: DMAR 0x000000000349FC401 000033 (V02 INTEL EDK2   00002 0100013)
DMAR: IOMMU enabled

Then, you can check if VFIO modules are enabled by running:

dmesg | grep -i vfio

If you sucessfully enabled VFIO modules, you can see some messages like:

VFIO - User Level meta-driver version: 0.3

Finally, check if remapping is enabled by running:

dmesg | grep 'remapping'

The output must be one of AMD-Vi: Interrupt remapping enabled or DMAR-IR: Enabled IRQ remapping in x2apic mode. Then remapping is also supported.

Once you see these messages, you're good to go. Otherwise, you need to check your settings again.

If you sucessfully enabled IOMMU and VFIO modules while failed to enable remapping, please check your BIOS again if you have enabled "VT-d".

Disable GPU Drivers

Before you can passthrough your GPU to a VM, you need to disable the GPU drivers on your host system.

You can run the following command to disable the GPU drivers:

echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidiafb" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia-gpu" >> /etc/modprobe.d/blacklist.conf
echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf
echo "blacklist amdgpu" >> /etc/modprobe.d/blacklist.conf
echo "blacklist snd_hda_intel" >> /etc/modprobe.d/blacklist.conf
echo "blacklist snd_hda_codec_hdmi" >> /etc/modprobe.d/blacklist.conf
echo "blacklist i915" >> /etc/modprobe.d/blacklist.conf

The nouveau, nvidia, nvidiafb, nvidia-gpu are for NVIDIA GPUs. The radeon, amdgpu are for AMD GPUs. The snd_hda_intel, snd_hda_codec_hdmi are for audio devices. The i915 is for Intel GPUs.

Passthrough GPU to VM

Check your GPU's PCI address by running:

lspci | grep VGA

You should see something like:

01:00.0 VGA compatible controller: NVIDIA Corporation GA104 [GeForce RTX 3060 Ti] (rev a1)

Find the PCI address of your GPU. In this case, it's 01:00.0, and then run:

lspci -n -s 01:00.0

You should see something like:

01:00.0 0300: 10de:24c9 (rev a1)
01:00.1 0403: 10de:228b (rev a1)

Edit vfio configuration file by running:

echo "options vfio-pci ids=10de:24c9,10de:228b" > /etc/modprobe.d/vfio.conf

Then update your initramfs:

update-initramfs -u -k all

Reboot your system.

Add GPU to your VM

Finnaly, add your GPU to your VM. Go to your Proxmox VE web interface, select your desired VM, and click "Hardware".

Add PCI Device, select Raw Device, and choose your GPU there. Be sure you check the "All Functions" checkbox.

Now you can start your VM and enjoy GPU passthrough! Don't forget to install the GPU drivers on your VM. (not on Host machine)

References

• PCI Passthrough
• PCI/GPU Passthrough on Proxmox VE 8 : Installation and configuration