Migrating from Proxmox to QEMU/KVM/Libvirt on Debian

30 Jun 2024

This guide walks you through migrating your virtual machines (VMs) from Proxmox VE to a self-hosted QEMU/KVM/Libvirt setup on Debian, utilizing ZFS for flexible storage management

Ensure your system is up to date:

sudo apt update && sudo apt upgrade

Installing QEMU, KVM, and Libvirt

Install the required packages without extraneous graphical packages:

sudo apt install --no-install-recommends qemu-system libvirt-clients libvirt-daemon-system bridge-utils virtinst dnsmasq

Verify the libvirtd service status: systemctl status libvirtd

Network Configuration

Edit the network interfaces file:

sudo nano /etc/network/interfaces

Configure your network adapters as needed. Here’s an example configuration for two interfaces:

auto lo
iface lo inet loopback

iface enp3s0 inet manual

auto vmbr0
iface vmbr0 inet static
    address 192.168.88.3/24
    gateway 192.168.88.1
    bridge-ports enp3s0
    bridge-stp off
    bridge-fd 0
    pre-up /usr/sbin/ethtool -s vmbr0 speed 10000 duplex full autoneg off
    post-up echo 1 > /proc/sys/net/ipv4/conf/enp3s0/proxy_arp  
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward

auto vmbr1
iface vmbr1 inet static
    address 10.0.0.1/24
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    pre-up /usr/sbin/ethtool -s vmbr1 speed 10000 duplex full autoneg off
    post-up echo 1 > /proc/sys/net/ipv4/ip_forward
    post-up iptables -t nat -A POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE
    post-down iptables -t nat -D POSTROUTING -s '10.0.0.0/24' -o vmbr0 -j MASQUERADE
    post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
    post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1

Replace enp3s0 with your actual network interface name. This configuration creates two bridges: vmbr0 for connecting the host to the external network and vmbr1 for the internal VM network. Adjust IP addresses, subnet masks, and gateway settings to match your network environment.

Verify KVM network settings: sudo virsh net-list --all

Configuring ZFS

Enable ZFS:

sudo sed -r -i'.BAK' 's/^deb(.*)$/deb\1 contrib/g' /etc/apt/sources.list
sudo apt update
sudo apt install linux-headers-amd64 zfsutils-linux zfs-dkms zfs-zed
sudo modprobe zfs

Create ZFS storage:

zpool create -f -d -m none -o ashift=12 -O atime=off -O compression=lz4 storage mirror /dev/sdX /dev/sdY

Installing the First VM using ZVOL

Download Debian ISO and prepare the VM disk:

mkdir /images
wget https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.5.0-amd64-netinst.iso
cp debian-12.5.0-amd64-netinst.iso
sudo zfs create -o compression=off -o dedup=off -o volblocksize=32K -V 25G storage/debian12-disk0

And initiate the VM:

virt-install --name debian12 --memory=1024 --vcpus=1 --disk path=/dev/zvol/storage/debian12-disk0,size=25 --os-type linux --os-variant debian9 --network bridge=vmbr1 --noautoconsole --location /images/debian-12.5.0-amd64-netinst.iso

Creating and Deploying Templates

Create a template snapshot:

sudo zfs snapshot storage/debian12-disk0@template

Create new storage/template from the snapshot

sudo zfs send -R storage/debian12-disk0@template | sudo zfs receive -Fv storage/debian12-template-disk0

Resize zvol

zfs set volsize=50G storage/dataset

Deploy the template:

virt-install --name 1stmachine --memory=1024 --vcpus=1 --disk path=/dev/zvol/storage/debian12-template-disk0,size=25 --os-type linux --os-variant debian9 --network bridge=vmbr1 --graphics vnc,listen=0.0.0.0 --noautoconsole --import

Adding Cloud-Init

Create a cloud-init configuration file (cloud-init.yaml):

#cloud-config
users:
  - name: mop
    sudo: ALL=(ALL) NOPASSWD:ALL
    shell: /bin/bash
    ssh_authorized_keys:
      - ssh-rsa AAAA[...]

Install VM with cloud-init:

virt-install --name 1sttemplate --memory=1024 --vcpus=1 --disk path=/dev/zvol/storage/debian12-template-disk0,size=25 --os-type linux --os-variant debian9 --network bridge=vmbr1 --graphics vnc,listen=0.0.0.0 --noautoconsole --import --cloud-init user-data=/root/cloud-init.yaml

Migrating from Proxmox

List and configure Proxmox VMs:

qm list
qm config 500
pvesm path local-zfs:vm-500-cloudinit
qemu-img convert -O qcow2 /dev/zvol/rpool/data/vm-100-cloudinit debian-template.qcow2

Convert and import the image to ZFS:

sudo zfs create -o compression=off -o dedup=off -o volblocksize=32K -V 10G storage/debian-template-disk0
qemu-img convert -O raw debian-template.qcow2 /dev/zvol/storage/debian-template-disk0

Install the imported VM:

virt-install --name debian-template --memory=1024 --vcpus=1 --disk path=/dev/zvol/storage/debian-template-disk0,size=25 --os-type linux --os-variant debian9 --network bridge=vmbr1 --graphics vnc,listen=0.0.0.0 --noautoconsole --import

Convert Vm to .qcow2 file

qemu-img convert -f raw /dev/zvol/storage/vm-test-disk0 -O qcow2 vm-test-disk0.qcow2

Exporting and Importing ZVOL to Raw File

Export ZVOL to raw file:

dd if=your_raw_file.raw of=/dev/zvol/<pool>/<volume>

Import raw file to ZVOL:

dd if=/dev/zvol/<pool>/<volume> of=file.raw

By following this guide, you can successfully migrate your virtual machines from Proxmox to a QEMU/KVM/Libvirt setup on Debian, leveraging advanced storage options like ZFS.

Installing Cockpit for Management

Enable the backports repository:

. /etc/os-release
echo "deb http://deb.debian.org/debian ${VERSION_CODENAME}-backports main" | sudo tee /etc/apt/sources.list.d/backports.list

sudo apt update

Install Cockpit:

sudo apt install -t ${VERSION_CODENAME}-backports cockpit
sudo apt install cockpit-machines guestfs-tools cockpit-podman

Verify Cockpit is running: netstat -pnltu | grep 9090

Configure hugepages:

grep Huge /proc/meminfo

sudo nano /etc/default/grub

Add the following line to the GRUB_CMDLINE_LINUX_DEFAULT:

default_hugepagesz=1G hugepagesz=1G hugepages=10

Update GRUB and reboot:

sudo update-grub
sudo reboot

Install essential utilities:

sudo apt install screen lm-sensors syslog-ng openbsd-netcat

Davide Staffieri