Argthtjtr

I am trying to set up a virtualized firewall for my home network. Architecture looks like: Internet -> virtualized pfSense -> lan. There are plenty of examples of how to do this using the old network configurator, but no examples for using netplan, Ubuntu 18.04's network manager.

I have two NIC cards on the host and freshly installed ubuntu 18.04 server edition installed.

Here is the contents of the /etc/netplan/50-cloud-init.yaml file

username@scarif:~$ cat /etc/netplan/50-cloud-init.yaml

network:

    version: 2

    renderer: networkd

    ethernets:

        eno1:

            dhcp4: no

            dhcp6: no

        enp5s0:

            dhcp4: no

            dhcp6: no



    bridges:

        br_wan:

            interfaces: [enp5s0]

            dhcp4: no

            dhcp6: no

            addresses: [10.0.0.1/24]

        br_lan:

            interfaces: [eno1]

            dhcp4: no

            dhcp6: no

            addresses: [192.168.1.29/24]

            gateway4: 192.168.1.1

            nameservers:

              addresses: [192.168.1.1,8.8.8.8]

Note that I believe I need to have a static IP assigned to the LAN bridge, and I want the virtualized pfSense to solicit a DHCP address from the ISP, thus I have assigned a throw away IP to the WAN bridge, and have purposely not assigned a gateway or nameserver to the WAN bridge. (Happy to be corrected on this netplan configuration if this is wrong)

I thought that i needed to create a second virtual bridge, so i have done that as well. I am showing both the default (aka the LAN) and the one i created which i named wan_bridge

username@scarif:~$ sudo virsh net-dumpxml default

<network>

  <name>default</name>

  <uuid>5e5d35c8-c46a-43ed-9fc4-13dcb3853b34</uuid>

  <forward mode='nat'>

    <nat>

      <port start='1024' end='65535'/>

    </nat>

  </forward>

  <bridge name='virbr0' stp='on' delay='0'/>

  <mac address='52:54:00:dc:57:8f'/>

  <ip address='192.168.122.1' netmask='255.255.255.0'>

    <dhcp>

      <range start='192.168.122.2' end='192.168.122.254'/>

    </dhcp>

  </ip>

</network>



username@scarif:~$ sudo virsh net-dumpxml wan_bridge

<network>

  <name>wan_bridge</name>

  <uuid>37a0163b-ae70-445d-a25f-c62cbe7d5b51</uuid>

  <forward mode='nat'>

    <nat>

      <port start='1024' end='65535'/>

    </nat>

  </forward>

  <bridge name='virbr1' stp='on' delay='0'/>

  <mac address='52:54:00:dc:57:8d'/>

  <ip address='192.168.133.1' netmask='255.255.255.0'>

  </ip>

</network>

Both show up just fine in virsh net-list

username@scarif:~$ sudo virsh net-list

 Name                 State      Autostart     Persistent

----------------------------------------------------------

 default              active     yes           yes

 wan_bridge           active     yes           yes

I also see both in brctl

username@scarif:~$ sudo brctl show

bridge name     bridge id               STP enabled     interfaces

br_lan          8000.aefa52b9a49b       no              eno1

br_wan          8000.22c9e0f24ba3       no              enp5s0

virbr0          8000.525400dc578f       yes             virbr0-nic

virbr1          8000.525400dc578d       yes             virbr1-nic

Running ifconfig after all this results in:

username@scarif:~$ ifconfig

br_lan: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet 192.168.1.29  netmask 255.255.255.0  broadcast 192.168.1.255

        inet6 fe80::acfa:52ff:feb9:a49b  prefixlen 64  scopeid 0x20<link>

        ether ae:fa:52:b9:a4:9b  txqueuelen 1000  (Ethernet)

        RX packets 20623  bytes 3453527 (3.4 MB)

        RX errors 0  dropped 36  overruns 0  frame 0

        TX packets 4756  bytes 5192472 (5.1 MB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0



br_wan: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500

        inet 10.0.0.1  netmask 255.255.255.0  broadcast 10.0.0.255

        inet6 fe80::20c9:e0ff:fef2:4ba3  prefixlen 64  scopeid 0x20<link>

        ether 22:c9:e0:f2:4b:a3  txqueuelen 1000  (Ethernet)

        RX packets 13  bytes 650 (650.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 26  bytes 1996 (1.9 KB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0



eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        ether 98:90:96:be:eb:02  txqueuelen 1000  (Ethernet)

        RX packets 22673  bytes 4163339 (4.1 MB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 9632  bytes 5540146 (5.5 MB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

        device interrupt 20  memory 0xf7d00000-f7d20000



enp5s0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500

        ether 68:1c:a2:12:f8:e9  txqueuelen 1000  (Ethernet)

        RX packets 849  bytes 128644 (128.6 KB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 26  bytes 1996 (1.9 KB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0



lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

        inet 127.0.0.1  netmask 255.0.0.0

        inet6 ::1  prefixlen 128  scopeid 0x10<host>

        loop  txqueuelen 1000  (Local Loopback)

        RX packets 347  bytes 26756 (26.7 KB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 347  bytes 26756 (26.7 KB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0



virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500

        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255

        ether 52:54:00:dc:57:8f  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 0  bytes 0 (0.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0



virbr1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500

        inet 192.168.133.1  netmask 255.255.255.0  broadcast 192.168.133.255

        ether 52:54:00:dc:57:8d  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 0  bytes 0 (0.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

But when i got to install pfSense with this command string:

sudo virt-install 

--name pfsense 

--memory 2048

--graphics vnc,listen=0.0.0.0 

--disk /var/lib/libvirt/images/pfsense.qcow2,size=8,format=qcow2 

--autostart 

--os-type linux

--cdrom /home/dooguls/pfSense-CE-2.4.4-RELEASE-amd64.iso 

--debug

The install goes fine, but pfsense only sees one interface, the one provided by the default network, which is my LAN interface/bridge.

After typing all the question out, I realized the problem was that the VM wasn't configured to 'see' another interface. So to fix the problem, I did the following:

sudo virsh dumpxml pfsense > 20181201-pfsense.txt

which of course gives the full xml definition of my VM. I then copied the existing interface entry:

<interface type='bridge'>

    <mac address='52:54:00:3a:37:7d'/>

    <source bridge='br_lan'/>

    <model type='rtl8139'/>

    <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

</interface>

and copied it to make a new interface by running this command:
sudo virsh edit pfsense

and adding these lines: (note that i changed the mac address by one bit, and assigned the virtual nic to slot 0x06)

<interface type='bridge'>

    <mac address='52:54:00:3a:37:7e'/>

    <source bridge='br_wan'/>

    <target dev='vnet1'/>

    <model type='rtl8139'/>

    <alias name='net1'/>

    <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>

</interface>

Then all i had to do was start up the pfsense VM and assign my interfaces to WAN and LAN. WAN grabbed a DHCP from the ISP, and I set a static IP on the LAN interface.

So in the end I think the three key steps on the hypervisor/host were:

1. Set up two bridges in netplan, one for each NIC card and assign the
   IP addresses to the bridges, not the physical NIC cards.


2. Create a new virtual network using virsh net-define not virsh net-create. (the former makes it persistent)


3. Add the second interface to the XML definition of the VM and point this interface at the WAN bridge.