Ygtjki

I am trying to set up a number of qemu quests on the same private network. I want them to not be accessible outside the host but they should be able to access each other and resources from the internet. For context this is going to be a thrash kubernetes setup.

What I have done:

1. Set up a bridge interface with a private ip

ip link add name br0 type bridge
ip addr add 10.24.0.1/24 dev br0
ip link set dev br0 up

2. Allowed access for the qemu-bridge-helper by adding the following line to /etc/qemu/bridge.conf
   

allow br0

3. Started a DHCP server bound to the bridge interface

dnsmasq --interface=br0 --bind-interfaces --dhcp-range=10.24.0.2,10.24.0.254

4. Allowed guest communication to skip iptables

sysctl -w net.bridge.bridge-nf-call-iptables=0

At this point, I can:

• from the host, access (ping/ssh) the guests


• from the guests, access (ping/ssh) other guests


• from the guests, access (ping/ssh) the host


• from the guests, ping the host's other IPs


• from the guests, perform hostname lookups

What I can not do is access anything beyond the host, e.g. internet resources.

I have docker installed and docker networking works, e.g. docker run -t alpine ping google.com works perfectly.

It seems to me that I need to allow packets to be forwarded from the 10.24.0.0 network, but I don't know how to do that. I have tried

iptables -I FORWARD -i br0 -j ACCEPT
iptables -I FORWARD -o br0 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -t nat -A POSTROUTING -o '!br0' --source 10.24.0.0/24 -j MASQUERADE

but that does not help.

And to clear up things, IP forwarding is enabled:

$ cat /proc/sys/net/ipv4/ip_forward
1

What can I change to allow internet access from my QEMU guests?

It should work if you correct the incorrect rule:

iptables -t nat -A POSTROUTING -o '!br0' --source 10.24.0.0/24 -j MASQUERADE
 ^^^^^^^^^

(which will never match because there is no interface named !br0)

into:

iptables -t nat -A POSTROUTING ! -o br0 --source 10.24.0.0/24 -j MASQUERADE

Simple test to show iptables doesn't consider '!br0' as a negation, using (easier) a dummy interface:

ip link add '!test' type dummy
ip address add dev '!test' 10.11.12.1/24
ip link set '!test' up
iptables -I OUTPUT -o '!test'

Now you can try pinging through this interface: ping 10.11.12.2 and will see with for example with iptables-save -c |grep test that the counter increases when the interface is used, and doesn't if not.

# iptables-save -c|grep test
[1:84] -A OUTPUT -o !test