VLAN for virtual machines: Unterschied zwischen den Versionen
Ingo (Diskussion | Beiträge) (revise descriptions, add discussion) |
Ingo (Diskussion | Beiträge) K (linkfix) |
||
Zeile 161: | Zeile 161: | ||
=== Discussion === | === Discussion === | ||
− | This works because of three conditions. | + | This works because of [[#References|three conditions]]. |
# ageing time is 0: ageing time specifies the number of seconds a MAC Address will be kept in the forwarding database after having a packet received from this MAC Address. Setting it to 0 means there is never a MAC Address stored in the FDB. | # ageing time is 0: ageing time specifies the number of seconds a MAC Address will be kept in the forwarding database after having a packet received from this MAC Address. Setting it to 0 means there is never a MAC Address stored in the FDB. | ||
# unicast flood on interfaces is on: this controls whether the bridge should flood traffic for which an FDB entry is missing and the destination is unknown through this port. Defaults to on. | # unicast flood on interfaces is on: this controls whether the bridge should flood traffic for which an FDB entry is missing and the destination is unknown through this port. Defaults to on. |
Version vom 24. September 2017, 13:01 Uhr
Inhaltsverzeichnis
Introduction
I wanted to update VLAN connections for virtual machines to newer technologies and put a question on unix.stackexchange. But I do not get any answer. It seems there is very little knowledge for this out there. So I decided to work on it by myself and document it here.
In gerneral I will look at four methods:
- oldstyle linux bridge as hub
- linux bridge as hub
- linux bridge with libvirt hook scripts
- Open vSwitch
Preparation
I have Debian GNU/Linux 9.1 (stretch) on the host and on virtual machines for testing. Setup is described here: Setup KVM with console. I'm sitting on harley as host, my all day workstation. Now I start the virtual machine, login and show its interface setting:
harley$ virsh start --console deb9-test
login
deb9-test$ cat /etc/systemd/network/08-vlan10.netdev [NetDev] Name=vlan10 Kind=vlan [VLAN] Id=10 deb9-test$ cat /etc/systemd/network/12-vlan10_attach-to-if.network [Match] Name=ens2 [Network] VLAN=vlan10 deb9-test$ cat /etc/systemd/network/16-vlan10_up.network [Match] Name=vlan10 [Network] DHCP=ipv4 IPv6AcceptRA=no LinkLocalAddressing=no
To test if the virtual machine has connection I use:
deb9-test$ journalctl -b --no-hostname -u systemd-networkd.service -- Logs begin at Fri 2017-09-15 17:09:51 CEST, end at Sat 2017-09-23 20:34:20 CEST. -- Sep 23 20:34:05 systemd-networkd[204]: Enumeration completed Sep 23 20:34:05 systemd[1]: Started Network Service. Sep 23 20:34:05 systemd-networkd[204]: vlan10: netdev ready Sep 23 20:34:05 systemd-networkd[204]: ens2: IPv6 enabled for interface: Success Sep 23 20:34:05 systemd-networkd[204]: ens2: Gained carrier Sep 23 20:34:05 systemd-networkd[204]: vlan10: Gained carrier Sep 23 20:34:06 systemd-networkd[204]: ens2: Gained IPv6LL Sep 23 20:34:06 systemd-networkd[204]: vlan10: Gained IPv6LL Sep 23 20:34:09 systemd-networkd[204]: vlan10: DHCPv4 address 192.168.10.89/24 via 192.168.10.1 Sep 23 20:34:09 systemd-networkd[204]: vlan10: Configured Sep 23 20:34:19 systemd-networkd[204]: ens2: Configured deb9-test$
4 sec after Started Network Service it gets an IP-Address and 14 sec later interface ens2 was Configured. If ens2 is Configured and the guest hasn't got an IP-Address the connection failed. It looks like this:
deb9-test$ journalctl -b --no-hostname -u systemd-networkd.service -- Logs begin at Fri 2017-09-15 17:09:51 CEST, end at Sat 2017-09-23 20:45:13 CEST. -- Sep 23 20:44:59 systemd-networkd[197]: Enumeration completed Sep 23 20:44:59 systemd[1]: Started Network Service. Sep 23 20:44:59 systemd-networkd[197]: vlan10: netdev ready Sep 23 20:44:59 systemd-networkd[197]: ens2: IPv6 enabled for interface: Success Sep 23 20:44:59 systemd-networkd[197]: ens2: Gained carrier Sep 23 20:44:59 systemd-networkd[197]: vlan10: Gained carrier Sep 23 20:45:00 systemd-networkd[197]: ens2: Gained IPv6LL Sep 23 20:45:00 systemd-networkd[197]: vlan10: Gained IPv6LL Sep 23 20:45:13 systemd-networkd[197]: ens2: Configured deb9-test$
Because I have to start the test virtual machine many times I setup autologin. It's no problem. There is nothing on the guest.
deb9-test$ grep ^ExecStart= /lib/systemd/system/serial-getty@.service ExecStart=-/sbin/agetty --keep-baud 115200,38400,9600 %I $TERM
modify to
ExecStart=-/sbin/agetty --autologin yourloginname --keep-baud 115200,38400,9600 %I $TERM
To list all settings of the bridge you can use:
harley$ find /sys/class/net/br0/bridge/ -type f -readable -printf '%f = ' -exec cat {} \; | sort
oldstyle linux bridge as hub
This works always with the old linux bridge that do not know anything about VLAN. The trick is to set it to a complete transparent state for all connected interfaces like a hub. But you have to know that the bridge will then forward all packets to all interfaces simultanously. You can do it by setting the ageing time to 0.
Disable systemd-networkd and start networking with ifupdown:
harley$ sudo systemctl stop systemd-networkd Warning: Stopping systemd-networkd.service, but it can still be activated by: systemd-networkd.socket harley$ sudo systemctl disable systemd-networkd Removed /etc/systemd/system/multi-user.target.wants/systemd-networkd.service. Removed /etc/systemd/system/sockets.target.wants/systemd-networkd.socket. harley$ sudo ip link set dev br0 down && sudo ip link del dev br0 harley$ sudo systemctl enable networking.service Synchronizing state of networking.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable networking harley$ sudo systemctl start networking.service harley$
Setup the bridge and start it:
harley$ cat /etc/network/interfaces auto br0 iface br0 inet manual bridge_ports enp1s0 bridge_ageing 0 bridge_stp off harley$ sudo ifup br0 Waiting for br0 to get ready (MAXWAIT is 32 seconds). harley$
It's all in place now:
harley$ cat /sys/class/net/br0/bridge/ageing_time 0 harley$ cat /sys/class/net/br0/bridge/stp_state 0 harley$ cat /sys/class/net/br0/bridge/vlan_filtering 0
Yes, there is no VLAN filtering, means VLAN on the bridge is disabled but the guest sees the VLAN-tagged packets.
References
linux bridge as hub
Now I try to setup #oldstyle linux bridge as hub just with systemd-networkd.
Disable networking with ifupdown and start systemd-networkd:
harley$ sudo systemctl stop networking.service harley$ sudo systemctl disable networking.service Synchronizing state of networking.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install disable networking harley$ sudo ip link set dev br0 down && sudo ip link del dev br0 harley$ sudo systemctl enable systemd-networkd Created symlink /etc/systemd/system/multi-user.target.wants/systemd-networkd.service → /lib/systemd/system/systemd-networkd.service. Created symlink /etc/systemd/system/sockets.target.wants/systemd-networkd.socket → /lib/systemd/system/systemd-networkd.socket. harley$ sudo systemctl start systemd-networkd harley$
Setup the bridge and start it:
harley$ cat /etc/systemd/network/08-br0.netdev [NetDev] Name=br0 Kind=bridge [Bridge] AgeingTimeSec=0 STP=false harley$ cat /etc/systemd/network/12-br0_add-enp1s0.network [Match] Name=enp1s0 [Network] Bridge=br0 harley$ cat /etc/systemd/network/16-br0_up.network [Match] Name=br0 harley$ sudo ip link set dev br0 down && sudo ip link del dev br0 harley$ sudo systemctl restart systemd-networkd harley$
AgeingTimeSec=0 is not acepted but should:
harley$ cat /sys/class/net/br0/bridge/ageing_time 30000 (means 300 sec) harley$
But I've found a workaround. Useing a number between .01 and .000001 (there are dots) will set ageing_time to 0. So set AgeingTimeSec=.000001 in /etc/systemd/network/08-br0.netdev. I suppose it's a bug. Then we will get:
harley$ cat /sys/class/net/br0/bridge/ageing_time 0 harley$ cat /sys/class/net/br0/bridge/stp_state 0 harley$ cat /sys/class/net/br0/bridge/vlan_filtering 0 harley$
The guest gets now an IP-Address on boot and is connected to VLAN 10.
Discussion
This works because of three conditions.
- ageing time is 0: ageing time specifies the number of seconds a MAC Address will be kept in the forwarding database after having a packet received from this MAC Address. Setting it to 0 means there is never a MAC Address stored in the FDB.
- unicast flood on interfaces is on: this controls whether the bridge should flood traffic for which an FDB entry is missing and the destination is unknown through this port. Defaults to on.
- spanning tree protocol (stp) is disabled: we don't have a forward_delay at startup for the learning phase of spanning tree.
I have a running and connected virtual machine:
harley$ sudo bridge vlan show port vlan ids enp1s0 1 PVID Egress Untagged br0 1 PVID Egress Untagged vnet0 1 PVID Egress Untagged harley$ cat /sys/class/net/br0/bridge/ageing_time 0 harley$ cat /sys/class/net/br0/bridge/forward_delay 1500 harley$ cat /sys/class/net/br0/bridge/stp_state 0
Indeed we have forward_delay 1500 (means 15 sec) but it doesn't matter. stp_state is 0 (disabled), no spanning tree. Flood (means unicast flood) is on as I can see:
harley$ sudo bridge -d link show 3: enp1s0 state UP : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 4 hairpin off guard off root_block off fastleave off learning on flood on mcast_flood on 95: vnet0 state UNKNOWN : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100 hairpin off guard off root_block off fastleave off learning on flood on mcast_flood on harley$
Let's have a look at flooding on the interfaces. I disable it on the physical interface enp1s0 of the bridge and reboot the guest:
harley$ sudo bridge link set dev enp1s0 flood off harley$
The guest gets an IP-Address from the DHCP-Server but then can't ping its gateway. DHCP-REQUEST is broadcast and goes thru enp1s0. DHCP-ANSWER comes back thru it to any other (here only vnet0) interface which has flood on. Ping is unicast and isn't forwareded on enp1s0. If I set enp1s0 flood on and vnet0 flood off and deb9-test$ sudo systemctl restart systemd-networkd
, I get no IP-Address from DHCP-Server and can't ping the interface. Incoming DHCP-ANSWER isn't broadcast and vnet0 doesn't forward it to the guest.
Btw. this method has bad performance as we can see with monitor. We insert MAC-Addresses into FDB for just deleting it immediately, all for nothing.
harley$ sudo bridge monitor fdb 52:54:00:01:76:20 dev enp1s0 master br0 52:54:00:b0:ca:63 dev vnet0 master br0 f4:f2:6d:2c:87:f7 dev enp1s0 master br0 00:80:3f:2a:31:1a dev enp1s0 master br0 Deleted 52:54:00:01:76:20 dev enp1s0 master br0 stale Deleted 52:54:00:b0:ca:63 dev vnet0 master br0 stale Deleted 00:80:3f:2a:31:1a dev enp1s0 master br0 stale Deleted f4:f2:6d:2c:87:f7 dev enp1s0 master br0 stale ...
References
- https://www.freedesktop.org/software/systemd/man/systemd.netdev.html
- https://www.freedesktop.org/software/systemd/man/systemd.network.html
linux bridge with libvirt hook scripts
References
- https://www.libvirt.org/hooks.html
- https://serverfault.com/questions/696011/libvirt-hook-qemu-suse12