Author: Siro Mugabi

Category: Development Setups

Summary:

Setting up IP forwarding and NAT on a 802.11 network interface for a non-WiFi LAN. For illustration, this configuration is performed on a laptop hosting QEMU guests attached to TAP network interfaces and connected to the Internet via IP masquerading on the laptop's Wireless NIC (WNIC). Ubuntu AMD64 (12.04/13.04/14.04) used for the test platform.

Tags: qemu linux development setups

Scenario

Internet access is available only via an Access Point (AP). A Linux PC/latop/tablet/etc with an 802.11 Wi-Fi radio is to be used to connect a (virtual) Local Area Network (LAN) of other machines that have no Wi-Fi capability to the Internet.

System Setup

There are a number of ways that a machine connected to an external network via Wi-Fi could use to provide connectivity for other nodes on a non-WiFi LAN. Two common configurations are Ethernet bridging and IP forwarding with NAT1. The different configurations require that the WNIC device - and the corresponding Linux device driver stack - support a certain 802.11 mode.

802.11 Modes

The normal mode of operation for a PC/laptop/tablet/etc WNIC is managed or station infrastructure mode. Almost any commodity client (i.e. non-AP) device/driver stack supports this mode. Some clients also include support for other 802.11 modes such as monitor, Wireless Distribution System (WDS), Ad-Hoc (IBSS), and Mesh. There also exists the AP infrastructure mode, but this 802.11 setting is used to turn a wireless device into a master (i.e. an AP) of a managed Wi-Fi network2. Check this out for more info on 802.11 modes and Linux.

WDS (4-address mode) Bridging vs. IP Forwarding and NAT

With respect to Ethernet bridging and the scenario described above, WDS is the most relevant mode3. Citing this, WDS mode is a non-standard extension to the IEEE 802.11 standard that allows transparent Ethernet bridging on the station4. More precisely, in order for a client WNIC operating in WDS mode to bridge nodes on the non-WiFi LAN to the wireless network, then both the client and AP need to transmit 4-address frames, containing both source and destination MAC addresses5. Note that 4-address mode is a WDS variant that is incompatible with other WDS implementations.

4-address mode lends to WDS bridging the advantage of Layer 2 transparency: the bridged nodes on the non-WiFi LAN are on the same broadcast domain as the external wireless network. This means that the bridged nodes on the non-WiFi LAN can get assigned IP addresses of the wireless subnet. For example, 4-address mode is how OpenWrt supports the AP-to-Sta WDS mode for mac80211 drivers.

However, due to lack of standardization, WDS is often implemented differently in wireless drivers and vendor firmwares making them incompatible with each other. In order to achieve WDS compatibility, the same hardware and software needs to be used for all participating wireless devices6. Now, not all commodity APs that are configurable for 4-address mode operation run Linux compatible WDS implementations. On the other hand, a significant number of vendor/commercial APs running Linux do not export interfaces that facilitate user configuration for 4-address mode. Furthermore, the AP could be a public facility (operating in standard 802.11 AP infrastructure mode) of which you have no priviledged access, i.e. to allow changing 802.11 modes. Therefore, using 4-address mode bridged setups is not always an option.

An alternative approach to connecting a non-WiFi LAN to a Wi-Fi network is by way of a gateway machine with a Wi-Fi radio. The advantage with this approach lies in the fact that, once IP forwarding and NAT have been enabled in the gateway's kernel, its WNIC now needs no special 802.11 mode of operation, i.e. continues to work in the default managed mode. This makes the setup simple and generic enough to work with any standard Wi-Fi connection.

Nevertheless, unlike a WDS bridged network setup, where the bridged nodes (on the non-WiFi LAN) lie in the same broadcast domain as the AP, nodes in this configuration reside in a different subnet behind the gateway. Broadcast traffic cannot be relayed between the non-WiFi LAN and the external wireless network. Therefore, the nodes on the non-WiFi LAN will instead receive (or statically configure) their IP address assignments from (or according to the subnet of) the gateway: The static routing table info on the node machines will point to the gateway's IP as the default. IP forwarding and SNAT/Masquerading on the gateway will then allow packets to reach the external network.

Network Interfaces

Now, depending on whether the nodes on the non-WIFI network are physical computers or virtual machines, the gateway/host configuration for the physical/virtual node interfaces will differ slightly:

  • If the gateway only provides connectivity for locally hosted (QEMU) VM guests on a virtual LAN, then no extra physical network interface is required. However, a virtual Linux bridge will have to be configured for the guest TAP interfaces.

  • If the gateway (also) provides connectivity for other physical machines, then it will require another physical network interface, e.g. an ethernet port, for the physical non-WiFi LAN.

In either case, IP forwarding and SNAT/Masquerading will be configured in the same way for the gateway's WNIC.

Example

The following case study was performed on a laptop hosting a QEMU VM:

Wi-Fi connectivity with the AP was first established. Use whichever utility/tool most familiar to you e.g. wpa_supplicant(8), iw(8), iwconfig(8), etc. In this particular instance, the nm-applet(1) (the front-end to GNOME's Network Manager) was used yeilding:

$ /sbin/iwconfig wlan0
wlan0     IEEE 802.11bgn  ESSID:"demo"  
                Mode:Managed  Frequency:2.437 GHz  Access Point: 99:11:88:22:77:66
        [...]

$ ip addr show dev wlan0
4: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
        link/ether 33:44:55:66:31:43 brd ff:ff:ff:ff:ff:ff
        inet 192.168.43.148/24 brd 192.168.43.255 scope global wlan0

$ cat /etc/resolv.conf 
nameserver 192.168.43.1

The following interfaces(5) configuration was then included in the host to automatically bring up a virtual bridge brW0:

$ cat /etc/network/interfaces
auto ... brW0
...
iface brW0 inet static
  pre-up brctl addbr brW0
  address 192.168.5.1
  netmask 255.255.255.0
  bridge_maxwait 0
  bridge_fd 0
  post-down brctl delbr brW0

Consult brctl(8) for information on ethernet bridge adminstration, bridge-utils-interfaces(5) for details on the bridge_* options, and interfaces(5) for info on settings such as pre-up and post-down.

Note that since this bridge configuration was meant to serve only (QEMU) VM guests, adding some physical network interface to it was not necessary. On the other hand, to enable a bridged connection between VM guests and other nodes on a physical non-WiFi LAN (if present), then the gateway's interface for the physical non-WiFi LAN will have to be added to this bridge. For example, check out the network interface configurations in A QEMU TAP Networking Setup and A QEMU VLAN Setup.

Moving on:

$ sudo service networking restart

$ ip addr show dev brW0
10: brW0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 8e:4f:47:11:ca:c5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.1/24 brd 192.168.5.255 scope global brW0
    inet6 fe80::8c4f:47ff:fe11:cac5/64 scope link 
        valid_lft forever preferred_lft forever

$ sudo brctl show brW0
bridge name  bridge id           STP enabled  interfaces
brW0         8000.000000000000   no

$ /sbin/route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.43.1    0.0.0.0         UG    0      0        0 wlan0
...
192.168.5.0     0.0.0.0         255.255.255.0   U     0      0        0 brW0
192.168.43.0    0.0.0.0         255.255.255.0   U     2      0        0 wlan0

A DHCP configuration for the QEMU guests on the VLAN was then added to /etc/dhcp/dhcpd.conf:

$ sudo apt-get install isc-dhcp-server

$ cat /etc/dhcp/dhcpd.conf
# Attention: If /etc/ltsp/dhcpd.conf exists, that will be used as
# configuration file instead of this file.
ddns-update-style none;
default-lease-time 600;
max-lease-time 7200;
max-lease-time 7200;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
#authoritative;

# Use this to send dhcp log messages to a different log file (you also
# have to hack syslog.conf to complete the redirection).
log-facility local7;
allow booting;
allow bootp;

# VLAN on brW0
subnet 192.168.5.0 netmask 255.255.255.0 {
    range dynamic-bootp 192.168.5.20 192.168.5.50;
    option broadcast-address 192.168.5.255;
    option routers 192.168.5.1;
    option domain-name-servers 192.168.43.1; ## as given by the AP 
}

$ sudo service isc-dhcp-server restart

$ sudo netstat -aup | grep dhcpd
udp        0      0 *:bootps        *:*     14722/dhcpd

... followed by the IP forwarding and SNAT setup on wlan0:

$ sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

$ sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE

$ sudo iptables -L -t nat
...
Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  all  --  anywhere             anywhere

Now, before launching the QEMU VM, the following set of executable scripts were prepared for the guest's TAP interfaces:

$ cat qemu_brW0_ifup.sh 
#!/bin/sh

switch=brW0
echo "$0: adding tap interface \"$1\" to bridge \"$switch\""
ifconfig $1 0.0.0.0 up
brctl addif ${switch} $1
exit 0

$ cat qemu_brW0_ifdown.sh 
#!/bin/sh

switch=brW0
echo "$0: deleting tap device \"$1\" from bridge \"$switch\""
brctl delif $switch $1
ifconfig $1 0.0.0.0 down
exit 0

$ chmod u+x qemu_brW0*

The following command line only shows the relevant QEMU networking command line options (root priviledges may be required to bring up the TAP interfaces):

$ sudo qemu-system-x86_64 ... -net nic,[model=$MODEL] \
    -net tap,ifname=tap0,script=qemu_brW0_ifup.sh,downscript=qemu_brW0_ifdown.sh

Once the VM was up and running, the following command on the host verified that tap0 was indeed attached to brW0:

$ sudo brctl show brW0
bridge name  bridge id        STP enabled  interfaces
brW0         8000.8a106b834eca no           tap0

$ ip link show dev tap0
15: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master brW0 state UNKNOWN qlen 500
        link/ether 8a:10:6b:83:4e:ca brd ff:ff:ff:ff:ff:ff

$ sudo brctl showmacs brW0
port no mac addr           is local?  ageing timer
1       52:54:00:12:34:56  no         29.34
1       8a:10:6b:83:4e:ca  yes        0.00

where mac address 52:54:00:12:34:56 on the remote bridge port is the ethernet address for the QEMU VM's virtual NIC (see below).

The network interface configuration on the QEMU VM was:

biko@vm:~$ cat /etc/network/interfaces
auto lo eth0 
iface lo inet loopback
iface eth0 inet dhcp

which automatically resulted in:

biko@vm:~$ ip addr show dev eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:12:34:56 brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.20/24 brd 192.168.5.255 scope global eth0

biko@vm:~$ /sbin/route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.5.1     0.0.0.0         UG    0      0        0 eth0
192.168.5.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0

biko@vm:~$ cat /etc/resolv.conf 
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 192.168.43.1

biko@vm:~$ ping -c 4 wiki.qemu.org
PING qemu.org (140.211.15.109) 56(84) bytes of data.
64 bytes from qemu.osuosl.org (140.211.15.109): icmp_seq=1 ttl=48 time=883 ms
64 bytes from qemu.osuosl.org (140.211.15.109): icmp_seq=2 ttl=48 time=1264 ms
64 bytes from qemu.osuosl.org (140.211.15.109): icmp_seq=3 ttl=48 time=1045 ms
64 bytes from qemu.osuosl.org (140.211.15.109): icmp_seq=4 ttl=48 time=855 ms

On the other hand, a quick & dirty manual and static IP configuration for the guest network interface will resemble:

biko@vm:~$ sudo ifconfig eth0 192.168.5.60  ## guest IP

biko@vm:~$ sudo route add default gw 192.168.5.1 dev eth0 ## gateway

biko@vm:~$ sudo sh -c "echo 'nameserver 192.168.43.1' >> /etc/resolv.conf"  ## DNS info according to AP

Just beware that if networkmanager(8) is "participating", then it will quite militantly impose its own policies, unceremoniously resetting your manual config. In this case, it is encouraged to silently conform and configure interfaces(5) with the static IP config and restart networking or reboot.

Resources

Footnotes

1. Specifically, Source NAT (SNAT) or Masquerading. With masquerading, the IP address of the gateway interface connected to the external network is dynamically assigned. With SNAT, this IP address is fixed. [go back].

2. To use AP mode in Linux hostapd is required - at least a current 0.6 release, and preferably from git. [go back].

3. Modes that support ethernet bridging with WNIC interfaces are WDS and Mesh. [go back].

4. WDS also enables seamless hand-over for wireless clients roaming between different APs [go back].

5. An illustration can be found here [go back].

6. See link [go back]