FRESH

Hacker News

Home

Ask HN: Vxlan over WireGuard or WireGuard over Vxlan?

45 points by mlhpdx

by volkadav

0 subcomment

https://man.openbsd.org/vxlan.4#SECURITY seems unambiguous that it's intended for use in trusted environments (and all else being equal, I'd expect the openbsd man page authors to have reasonable opinions about network security), so it sounds like vxlan over ipsec/wg is probably the better route?

by notepad0x90

1 subcomments

For site-so-site ovelay networks, use wireguard, vxlan should be inside of it, if at all. Your "network" is connected by wireguard, and it contains details like vxlan. Even within your network, when crossing security boundaries across untrusted channels, you can use wireguard.
Others mentioned tailscale, it's cool and all but you don't always need it.
As far as security, that's not even the consideration I had in mind, sure wireguard is secure, but that's not why you should have vxlan inside it, you should do so because that's the purpose of wireguard, to connect networks securely across security/trust boundaries. it doesn't even matter if the other protocol is also wireguard, or ssh or whatever, if it is an option, wireguard is always the outermost protocol, if not then ipsec, openvpn,softether,etc..whatever is your choice of secure overlay network protocol gets to be the tunnel protocol.

by q3k

3 subcomments

Drop the VXLAN. There's almost never a good reason to stretch L2 over a WAN. Just route stuff across.

by denkmoon

3 subcomments

Not super related to the OP but since we're discussing network topologies; I've recently had an insane idea that nfs security sucks, nfs traversing firewalls sucks, kerberos really sucks, and that just wrapping it all in a wireguard pipe is way better.
How deranged would it be to have every nfs client establish a wireguard tunnel and only have nfs traffic go through the tunnel?

by iscoelho

2 subcomments

VXLAN over WireGuard is acceptable if you require a shared L2 boundary.
IPSec over VXLAN is what I recommend if you are doing 10G or above. There is a much higher performance ceiling than WireGuard with IPSec via hardware firewalls. WireGuard is comparatively quite slow performance-wise. Noting Tailscale, since it has been mentioned, has comparatively extremely slow performance.
edit: I'm noticing that a lot of the other replies in this thread are not from network engineers. Among network engineers WireGuard is not very popular due to performance & absence of vendor support. Among software engineers, it is very popular due to ease of use.

by jrm4

2 subcomments

Whenever I see threads like this, I think its related but I'll be honest, my networking understanding might be limited.
I use Tinc as a daily driver (for personal things) and have yet to come up with a new equivalent, given that I probably should. Does Vxlan help here?

0 subcomment

by solaris2007

0 subcomment

If a situation where production vxlan is going over Wireguard arises, then someone in leadership failed to plan and the underlying Wireguard tunnel is coping with that failure. No doubt, OP already knows this and all too well.
The problem is no doubt a people problem. I have learned to overcome these people problems by adhering to specific kinds of communication patterns (familiar to Staff Engineers and SVPs).
There is no reason that Wireguard over vxlan over Wireguard can't work, even with another layer (TLS) on top of Wireguard. Nonetheless it is very suboptimal and proprietary implementations of vxlan tend to behave poorly in unexpected conditions.
We should remember that vxlan is next-get vlan.
The type of Wireguard traffic encapsulated within the vxlan that comes to mind first is Kubernetes intra/inter-cluster pod-to-pod traffic. But this Wireguard traffic could be between two legacy style VMs.
If I were the operator told "you need to securely tunnel this vxlan traffic between two sites" I would reach for IPsec instead of Wireguard in an attempt to not lower the MTU of encapsulated packets too much. Wireguard is a layer 4 (udp) protocol intended to encapsulate layer 3 (ipv6 and legacy ip) packets.
If I were the owner of the application I would bake mutual TLS authentication on QUIC with "encrypted hello" (both elliptic and PQ redundant) into the application. The applications would be implemented in Rust, or if not practicable to implement the application in Rust I would write into the Helm chart a sidecar that does such a mutual TLS auth part (in Rust of course).
I would also aggressively "ping" in some manner through the innermost encapsulation layer. If I had tenancy on a classic VM doing Wireguard over the vxlan I would have "ping -i 2 $remote_inside_tunnel_ipv6" running indefinitely.

by kjuulh

0 subcomment

I use vxlan on top of wireguard in my hobby set up. Probably wouldn't recommend it for an actual production use-case. But that is more or less because of how my homelab is setup (Hetzner -> Home about 20ms latency roundtrip).
I considered dropping my root wireguard and setting up just vxlan and flannel, but as I need NAT hole punching I kind of need the wireguard root so that is why i ended up with it.
Going Wireguard inside the vxlan (flannel) in my case, would likely be overkill, unless I wanted my traffic between nodes between regions to be separated from other peers on the network, not sure where that would be useful. It is an easy way of blocking out a peer however, but that could just as well be solved on the "root" wireguard node.
There might be some MTU things that would be messed up going nested wireguard networks.

by tucnak

1 subcomments

For traversing public networks, simply consider BGP over Wireguard. VXLAN is not worth it.

by justsomehnguy

0 subcomment

WG is L3 transport
VXLAN is L2-like tranport over L3
You can have EoIP over WG with any VLANs you like.
You can have a VXLAN over plain IP, over EoIP, over WG, over IPSec. Only WG and IPSec (with not NULL sec) do providecany semblance ofvencryption in transit
And mandatory X\Y problem.

by inetknght

0 subcomment

> Let’s agree going recursive (WireGuard inside VXLAN inside WireGuard) is a bad idea.
But it's not necessarily a bad idea. It depends on the circumstances, even when traversing a public network.

by stevefan1999

4 subcomments

Is there a WireGuard equivalent that does L2 instead of L3? Need this for a virtual mesh network for homelabbing. I have this exact setup, running VXLAN or GENEVE over WireGuard tunnel using KubeSpan from Talos Linux but I simply think having L2 access would make load balancer much easier

by uberduper

1 subcomments

What are your discovery mechanisms? I don't know what exists for automatic peer management with wg. If you're doing bgp evpn for vxlan endpoint discovery then I'd think WG over vxlan would be the easier to manage option.

by wmf

0 subcomment

What problem is being solved here?

by mbreese

0 subcomment

What are you trying to do? Why are you trying to link networks across the public internet?

by pjd7

1 subcomments

Tell us why you think so at least.

by DiabloD3

3 subcomments

I mean, ultimately, thats how Google routes internally.
IPSec-equivalent, VXLAN-equivalent, IPSec-equivalent.
Prevents any compromised layer from knowing too much about the traffic.

by sciencesama

1 subcomments

vxlan inception is fun ! said no one ever !

by H8crilA

1 subcomments

Not sure I understand, but why not Tailscale?