KoaVM
Guide pages

Guide · For administrators

GPU

KoaVM passes the nodes' graphics cards to VPSs: for artificial intelligence, compute, graphics and remote desktops. There are three ways, the same ones Incus offers for virtual machines:

  • Whole GPU (passthrough): the whole card goes to one VPS. One card, one VPS. It works with any GPU (NVIDIA, AMD, Intel).
  • vGPU: the card is split into equal slices, each with a profile (for example NVIDIA L4-4Q). The vendor driver is needed on the node: for NVIDIA the NVIDIA vGPU host driver and its licence, for Intel graphics GVT-g.
  • SR-IOV: the card offers virtual functions (VFs), one per VPS (AMD MxGPU, Intel Data Center Flex, some NVIDIA cards).

GPUs are never overcommitted: a slice of a card belongs to one VPS only.

Preparing the node

To pass a GPU to a VPS the server needs the IOMMU turned on:

  1. in the server BIOS turn on VT-d (Intel) or AMD-Vi / IOMMU (AMD);
  2. on the node system add intel_iommu=on iommu=pt (Intel) or amd_iommu=on iommu=pt (AMD) to GRUB_CMDLINE_LINUX in /etc/default/grub, then run update-grub and reboot the node (move its VPSs to another node first).

The vfio-pci module is already in the Debian and Ubuntu kernel. Nothing else is needed for a whole GPU: Incus detaches the card from the node driver when the VPS starts.

Enabling GPUs

In Nodes, under the node, the GPU section shows the cards Incus sees: model, PCI address, driver, vGPU profiles and SR-IOV VFs, who uses them.

  • For VPSs: a card goes to VPSs only once you enable it. The server video card (BMC, ASPEED or Matrox) shows up in the list too: it is marked “server card” and cannot be enabled.
  • Check the node: checks whether the IOMMU is on, whether vfio-pci is there and which IOMMU group each card is in, and tells you what to do if something is missing. If a card shares a group with other devices, they go to the VPS together with it: usually it is the HDMI audio of the same card, which is fine.

When you disable a card the VPSs using it keep it; new VPSs no longer get it.

Plans with a GPU

In Clients and plans, in the plan, the GPU section:

  • GPUs per VPS: how many (0 = none, up to 8);
  • Type: whole GPU, vGPU or SR-IOV;
  • Model: a specific model (e.g. NVIDIA L4) or any enabled GPU;
  • vGPU profile: only for vGPUs, the profile each VPS gets.

When you create a VPS with this plan the panel picks a node with the same GPUs free and assigns them to the VPS. If no node has them, creation stops and says why. The client sees on their VPS which GPU it has.

The RAM of VPSs with a GPU stays fully pinned (passing the card locks the VPS memory): for these VPSs the panel checks that their RAM fits in 85% of the node's real RAM, on top of the room with overcommit.

What changes for VPSs with a GPU

  • Moving between nodes: cold only, to a node with the same GPU free (the panel takes it as soon as you queue the move). The state of a GPU cannot be moved live.
  • High availability: the reserve node must have the same GPU free. The GPU is taken when the VPS restarts: if none is free at that moment, the VPS does not restart and the High availability page says why.
  • Plan change: the new plan must have the same GPU as the VPS. For another GPU create a new VPS.
  • Changing a plan's GPU applies to new VPSs: existing ones keep theirs.
  • Rescue, snapshots and backups work as usual.
  • Provider zones (Hetzner Cloud, Contabo, Scaleway) have no GPUs from the panel.