Home stretch on new network - if_bridge looking better
dillon at apollo.backplane.com
Thu Feb 24 23:37:40 PST 2011
:On 02/24/11 11:50, Matthew Dillon wrote:
:So - reading over this - is it correct that the setup is roughly like:
:- assign a local interface (lan0) to a network
:- add this network to the bridge
:- create openvpn 'bridged' mode tunnels
:- add these to the bridge
In the case of my current setup, lan0, uverse0, comcast0, and
aerio0 are all physical ethernet ports. lan0 is the LAN, and
the other three connect to the three different WAN services I
Only lan0 and the tunnels (tap0, tap1, tap2) are associated with
The other physical ethernet ports (uverse0, comcast0, and aerio0)
each have a different IP and a different default route and I use
IPFW to associate packets sourced from the IP to the default route
for each port. Currently uverse0 and comcast0 are both dynamic
while aerio0 is a static IP (the old DragonFly net /26).
The OpenVPN tunnels are built using these IPs and back the tap devices.
The tap devices are then associated with the bridge and the main LAN.
The tap devices themselves, and the bridge, have *NO* IPs associated
with them. All the local IP spaces are on lan0, including some local
NATted spaces (10.x.x.x). The bridge code and the ARP code deal with
the inconsistencies and provide a consistent ARP for the bridge members.
Also, not shown here, is that I have a massive set of PF rules and ALTQs
on each of the TAP interfaces (tap0, tap1, and tap2). In particular I'm
running the ALTQs on the TAP devices with fair-share scheduling and
tuned to the bandwidth of each WAN so ping times will be low no matter
what topology the bridge is using. (Of course I can't do fair-share
scheduling on the WAN ports, uverse0, comcast0, and aerio0, because
the only thing running over them is the OpenVPN UDP packets and it
can't dig into them to see what they represent).
:so the L2 bridge / STP will 'map' according to the state of
:the ethernet bridging, which in turn relates to the openvpn tunnel
Exactly. The if_bridge module does its own 'pinging' using STP
config packets so it can detect when a link goes down. OpenVPN
itself also has a ping/restart feature. I use both. OpenVPNs internal
keepalive auto-restarts openvpn on failure, and the if_bridge's
pinging is used to detect actual good flow across the link and controls
:Without diverging any security sensitive whatnot,
:Is the VPN tunnel created to the ISP or to say, the colo space?
:(I'd assume the latter)
Yes, a colo space that the DragonFly project controls, provided by
Peter Avalos. OpenVPN itself is running encrypted UDP packets.
Very easy to set up. The colo has around 10 MBytes/sec of bandwidth
which is plenty for our project.
:Have been working on my own openvpn (routing mode) fun to a pair
:of VPS's as well over the last few days so this is of interest :D
:also - I note in the "bridge2.txt" file you 'cd /usr/pkg/etc/openvpn'
:before running - is this so openvpn can find the config files?
Yes, that's actually a bit broken. I've since changed it to put a
'cd' directive in the config file itself and then just run openvpn
with the full path to the config file. Openvpn has problems restarting
itself if you don't do this (it winds up getting confused and not being
able to find the key files if it restarts).
:if so - to note, you can add a 'cd /path/to/configdir' within the
Yah, found that :-)
:also - assuming you have statics on both end of the tunnels -
:why did you choose openvpn ethernet bridging over say IP layer + ipsec?
:(or even openvpn 'routing' mode) with something like OSPF or similar
:and - do you have hw crypto cards on either endpoint?
I originally attempted to route a subnet but the problem is we have
a full class C at the colo, but DragonFly isn't really designed to
operate with two different subnets where one subnet overlaps the other.
Ethernet switching turned out to be the better solution. The colocated
box itself is ON the class C, it doesn't have a separate IP outside
the class C space. So there was no easy way to swing a routed network.
I wouldn't even consider something as complex as OSPF for a simple
setup like this, even with a routed solution.
:(my soekris 486 gets a little bogged down by the crypto, which is why I ask)
:ok enough questions ;)
:its definitely fun trying to convert consumer internet into a 'real
:(from a gigabit LAN piggybacked on a sometimes 56k wifi link)
OpenVPN has options to run in the clear after authentication is complete,
I think, but I highly recommend using the crypto TLS support. The
instructions on setting up all the files are pretty clear (you can find
them on OpenVPN's web site).
The problem with running in the clear even after using crypto
authentication is that you really have to use UDP piggybacking if you
want the VPN to be reliable. Using TCP for the VPN is a huge mistake.
With UDP it is very easy for outside entities to inject data or otherwise
mess up your VPN if it isn't 100% encrypted. Encryption also prevents
outside entities from doing deep-packet inspection of your VPN, and
believe me they probably would if it weren't encrypted (though perhaps
the deep packet inspection engines don't yet understand OpenVPN's
CPU-wise the encryption does take a bit more horsepower, a 486 equivalent
won't be so happy, but any desktop built in the last 10 years ought to
have plenty of horsepower for a few megabytes a second worth of link
I've also found that VPNing to a colo space is far more reliable then
trying to run through a vendor-supplied static IP block. The U-Verse
static IP block service is a piece of crap. It's unusable and it can't
handle a lot of concurrent connections due to the stateful firewall
it imposes even on the static IP block. I don't know much about
ComCast's static IP service but I suspect similar issues, particularly
since ComCast is trying to work around net neutrality. I just don't
want them to be able to classify my traffic. The VPN is the best
solution for that.
<dillon at backplane.com>
More information about the Users