Network setup of proxmox VE 2.x on hetzner server

Submitted by admin on Fri, 11/30/2012 - 10:27

In the second part of the series I'm going to setup routed networking on a root server of Hetzner, for KVM based virtualization and with Ubuntu 12.04 LTS guests. These guests will be webservers and database-servers, connected by private LAN with internal bridge NAT-ed to eth0 of the host. Shorewall and Fail2ban will help to get the proper networking in place, next to security. Finally, a Pound load balancer (reverse proxy) with one additional public IP, will be setup, as well.

In this article you can learn how to setup the host, with Proxmox 2.x VE on Hetzner EX4, with Debian Squeeze 64 bit.

Kernel settings

First we need a few changes in kernel parameters.

/etc/sysctl.conf

The target setup requires some deviation from the default kernel settings of Debian: IP forwarding and Proxy ARP should be set. Change /etc/sysctl.conf as follows:

### Hetzner Online AG installimage
# sysctl config
net.ipv4.ip_forward=1  
net.ipv4.conf.all.rp_filter=1  
net.ipv4.icmp_echo_ignore_broadcasts=1  
net.ipv4.conf.all.proxy_arp=1  
net.ipv4.conf.default.proxy_arp=1

/etc/rc.local

We reload the kernel variables from /etc/sysctl.d/, because there are hints about recent issues, nevertheless the Proxmox staff do not bother with it too much.

#!/bin/sh -e
#
# rc.local
# Load kernel variables from /etc/sysctl.d
# see: http://wiki.debian.org/BridgeNetworkConnections
/etc/init.d/procps restart
exit 0

Network settings

Change the following four files accordingly.

/etc/hosts

Add here the public IP of your server and the hostname.

127.0.0.1 localhost  
PUBLIC_SERVER_IP myhost.hu myhost

/etc/hostname

Once again just the plain hostname myhost goes into here.

/etc/resolv.conf

Here you should list the nameservers of Hetzner, optionally of others, as well.

### Hetzner Online AG installimage
# nameserver config
nameserver 213.133.99.99  
nameserver 213.133.100.100  
nameserver 213.133.98.98

/etc/network/interfaces

First of all, get these three IP addresses from the original /etc/network/interface file, and write it down: PUBLICSERVERIP, SERVERGATEWAY and BROADCASTADDRESS. ADDITIONALSERVERIP is the second public IP address, what I ordered additionally.

I setup three network bridges on the host:

  • vmbr0 will handle the second public IP (ADDITIONALSERVERIP), and the guest on this bridge will appear on on the Internet with this public IP. I'll use it for the guest running the Pound load balancer.
  • vmbr1 will be a private LAN for all the guest, but through vmbr1 the guests will be able to go out to the Internet, it will be NAT-et through the host interface.
  • vmbr2 is also a private LANs to connect the guests with each other, but completely isolated from Internet.

For the final working setup Shorewall must be also up and running. Without Shorewall, you can still test this setup, if you comment out the line below the comment lines "# use only if Shorewall is down:", in both vmbr0 and vmbr1 brigde configs.

### Hetzner Online AG - installimage
# Loopback device:
auto lo  
iface lo inet loopback  
#
# external interface of the host
auto eth0  
iface eth0 inet static  
  address PUBLIC_SERVER_IP
  netmask 255.255.255.255
  gateway SERVER_GATEWAY
  broadcast BROADCAST_ADDRESS
  pointopoint SERVER_GATEWAY
#
# bridge for VMs with public IPs (DMZ)
auto vmbr0  
iface vmbr0 inet static  
  address PUBLIC_SERVER_IP
  netmask 255.255.255.255
  broadcast BROADCAST_ADDRESS
  bridge_ports none
  bridge_stp off
  bridge_fd 0
  # use only if Shorewall is down:
  # up ip route add ADDITIONAL_SERVER_IP/32 dev vmbr0
#
# bridge for internal LAN with private IPs
auto vmbr1  
iface vmbr1 inet static  
  address 192.168.0.1
  netmask 255.255.255.0
  bridge_ports none
  bridge_stp off
  bridge_fd 0
  # use only if Shorewall is down
  # post-up iptables -t nat -A POSTROUTING -s '192.168.0.0/24' -o eth0 -j MASQUERADE
  # post-down iptables -t nat -D POSTROUTING -s '192.168.0.0/24' -o eth0 -j MASQUERADE
#
# bridge for second internal LAN with private IPs
auto vmbr2  
iface vmbr2 inet static  
  address 10.10.10.1
  netmask 255.255.255.0
  bridge_ports none
  bridge_stp off
  bridge_fd 0

Shorewall firewall settings

Shorewall is a firewall configuration tool, and has very handy solutions to configure complex network setup on iptables, like proxy ARP and masquerading for NATs. Simple change the following Shorewall config files accordingly.

Shorewall install and initial config

First of all, install Shorewall:

apt-get install shorewall

You probably noticed a warning message at the end of the Shorewall installation, telling you the program will not start unless you change the /etc/default/shorewall file. You can do this by changing startup = 0 to startup = 1.

Next, edit /etc/shorewall/shorewall.conf and change the following values:

IP_FORWARDING=Keep  
DISABLE_IPV6=No

to:

IP_FORWARDING=On  
DISABLE_IPV6=Yes

And two simple tricks for quality. First, always check the Shorewall config files for correct syntax by shorewall check. To switch temporary to the new config, but return to the old ones after 60 seconds use shorewall try /etc/shorewall 60

Network config in Shorewall

The following config files implement then the required firewall functionality, as well as the target network setup. For more details please refer to Shorewall docs.

/etc/shorewall/zones

# http://linux.die.net/man/5/shorewall-zones
#ZONE   TYPE   OPTIONS   IN       OUT
#                        OPTIONS  OPTIONS
fw      firewall  
net     ipv4  
loc     ipv4  
dmz     ipv4

/etc/shorewall/interfaces

# http://linux.die.net/man/5/shorewall-interfaces
#ZONE INTERFACE BROADCAST OPTIONS
net   eth0      detect logmartians,tcpflags,nosmurfs  
dmz   vmbr0     detect logmartians,bridge,routefilter,tcpflags,nosmurfs  
dmz   vmbr1     detect logmartians,bridge,routefilter  
loc   vmbr2     detect logmartians,bridge,routefilter

/etc/shorewall/policy

# http://linux.die.net/man/5/shorewall-policy
#SOURCE DEST POLICY LOG LEVEL LIMIT:BURST
# 1. fw - loc
fw      loc  ACCEPT  
loc     fw   ACCEPT  
# 2. fw - dmz
fw      dmz  ACCEPT  
dmz     fw   DROP   info  
# 3. fw - net
fw      net  ACCEPT  
net     fw   DROP   info  
# 4. dmz - net
dmz     net  ACCEPT  
net     dmz  DROP   info  
# 5. loc - dmz
loc     dmz  ACCEPT  
dmz     loc  DROP   info  
# 6. loc - net
loc     net  ACCEPT  
net     loc  DROP   info  
# THE FOLLOWING POLICY MUST BE LAST
all     all  REJECT info

/etc/shorewall/rules

# http://linux.die.net/man/5/shorewall-rules
#ACTION SOURCE DEST PROTO DEST SOURCE ORIGINAL RATE
#
# Accept particular connections from Internet
#
# Permit access to SSH
SSH/ACCEPT net fw   -     -    -      -        6/min:5  
#
# Permit access to Proxmox Manager and Console
ACCEPT     net fw  tcp    443,5900:5999,8006  
#
# PING Rules
Ping/ACCEPT all all

# Permit traffic to - certain - VMs in DMZ
HTTP/ACCEPT net  dmz:$ADDITIONAL_SERVER_IP  
SSH/ACCEPT  net  dmz:$ADDITIONAL_SERVER_IP  
#
# LAST LINE -- DO NOT REMOVE

/etc/shorewall/masq

# implements NAT on vmbr1
#INTERFACE SOURCE ADDRESS PROTO PORT(S) IPSEC MARK
eth0 192.168.0.0/24

/etc/shorewall/proxyarp

# vmbr0 apperars on the Internet
#ADDRESS INTERFACE EXTERNAL HAVEROUTE PERSISTENT
$ADDITIONAL_SERVER_IP vmbr0 eth0

/etc/shorewall/params

ADDITIONAL_SERVER_IP=12.34.56.56

Now we can start to build guests on the Proxmox web interface. The guest will be behind a firewall, but can have access to the Internet, if they have an network interface on vmbr1, and can be accessed from the Internet, if the have an other interface on vmbr0.