Rebuilding My Home Network

ABSTRACT

I have had my ESXi box for YEARS and recently decided to take a dive into the world of KVM (QEMU) on Ubuntu Linux.  It’s a popular and completely open hypervisor that has become a staple in many datacenter environments.  I had been loathing the change because it meant I had to rebuild a few of the VMs I still had.  Though I made the plunge recently into docker containers and finding apps I could containerize, I still have a handfull of VMs that perform various functions on my home network and “home lab”.  I’ll preface this by saying that the ESXi box did us well over the years and I hardly ever had to touch it.  The one thing I did not have with ESXi was other hosts to use for moving VMs.  The fact that the license isn’t free and ESXi is very persnickety about hardware requirements just was a put-off in trying to implement V-Motion (VMWare’s method of migrating/moving VMs around in a cluster).  With KVM, my options are more open and I have a handful of hosts that are compatible with KVM so if I ever had to move VMs around, I can in a pinch if ever I have a problem with hardware.

SOLUTION

To create a small cluster of physical hosts to run my new VMs built on KVM, I simply carried out these steps (I’ll go into greater detail on each one):

• Install Ubuntu Server 20.04 OS on each physical machine
• Configure a netplan for each physical KVM (pkvm) host that achieves:
  • bonded (teamed) interfaces
  • LACP (802.3ad) attributes to bring up the channel-group session
  • vlan tagged traffic over the bond
  • bridge interfaces to allow kvm guest VMs to attach to the desired vlan
• Install necessary KVM packages
• Configured a port channel interface on the main switch & define physical switch ports that will participate in the LACP channel-group
• Configure a common NFS mount on the NAS to hold the KVM guest images on the network
• Moving new kvm VMs into my newly rebuilt KVM host
• Consume donuts that my wife and kids made while the “internet was out”

Setting a Netplan

Ubuntu 20.04 uses netplan to configure the operation of network interfaces.  This method uses a simple YAML formatted file that is easy to write and backup.  If you screw up, you can always revert back to a previous file.  (always make a backup!)  My netplan file looks like this:

network:
  bonds:
    bond0:
      interfaces:
      - enp1s0f2
      - enp1s0f3
      parameters:
        mode: 802.3ad
        lacp-rate: fast
        mii-monitor-interval: 100
  ethernets:
#    enp1s0f0: {}
#    enp1s0f1: {}
    enp1s0f2: {}
    enp1s0f3: {}
    enp3s0:
      dhcp4: no
    enp4s0:
      dhcp4: no
  vlans:
    vlan.2:
      id: 2
      link: bond0
      dhcp4: no
    vlan.5:
      id: 5
      link: bond0
      dhcp4: no
    vlan.11:
      id: 11
      link: bond0
      dhcp4: no
    vlan.10:
      id: 10
      link: bond0
      dhcp4: no
    vlan.50:
      id: 50
      link: bond0
      dhcp4: no
    vlan.73:
      id: 73
      link: bond0
      dhcp4: no
    vlan.300:
      id: 300
      link: bond0
      dhcp4: no
  bridges:
    br73:
      interfaces:
      - vlan.73
    br2:
      interfaces:
      - vlan.2
    br5:
      interfaces:
      - vlan.5
    br10:
      interfaces:
      - vlan.10
    br11:
      interfaces:
      - vlan.11
      addresses: [10.0.1.3/24]
      gateway4: 10.0.1.1
      nameservers:
        addresses: [10.0.1.10]
    br50:
      interfaces:
      - vlan.50
  version: 2

Installing KVM packages

apt install -y qemu qemu-kvm libvirt-daemon bridge-utils virt-manager virtinst

Configuring a port channel interface

With the netplan configuration saved and in place, I just executed sudo netplan apply and then proceeded to setup the switch-side of the bonded connection.  The first thing I needed to do was configure a new port channel interface on the switch that would be suitable for carrying tagged traffic:

interface Port-channel2
 description KVMBOX
 switchport trunk encapsulation dot1q
 switchport mode trunk
 spanning-tree portfast
end

After setting up the port channel interface, I now had to add the physical interfaces that are cabled from the switch to the big KVM box:

interface GigabitEthernet1/0/44
 description TRUNK_TO_VBOX1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 lacp port-priority 500
 channel-protocol lacp
 channel-group 2 mode active
interface GigabitEthernet1/0/45
 description TRUNK_TO_VBOX1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 lacp port-priority 500
 channel-protocol lacp
 channel-group 2 mode active
end

wr mem

Once both ports were setup, I could then test the channel-group status:

CORE-SW#sh etherchannel 2 summary
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      f - failed to allocate aggregator

        M - not in use, minimum links not met
        u - unsuitable for bundling
        w - waiting to be aggregated
        d - default port


Number of channel-groups in use: 2
Number of aggregators:           2

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
2      Po2(SU)         LACP      Gi1/0/44(P) Gi1/0/45(P)

It’s time to now setup an NFS share for holding KVM images

This part is easy:

• apt install nfs-client

Add a line to /etc/fstab:

10.9.9.20:/volume1/vol1 /vol1   nfs     _netdev,nfsvers=3,noatime,bg    0       0

Then mount: sudo mount /vol1.   I then created a symbolic link from the /var/lib/libvirt/images directory to point to /vol1/kvms (where the other KVM servers put their images)

Now time to move my newly built KVM VMs into my new KVM host (built on a temporary KVM host)