Multi-site networking

A site is not merely an IP range. It has people, services, a physical Internet connection, and a failure mode. Give every site a non-overlapping IPv4 subnet, one or more IPv6 /64s, a gateway, a DNS policy, and a list of networks it may reach. This turns a future fault into a route lookup instead of an archaeology project.

Branch office                         Head office
Users:      192.168.20.0/24          Users:      192.168.10.0/24
Servers:    192.168.21.0/24          Servers:    192.168.11.0/24
IPv6:       2a02:8012:bc57:20::/64   IPv6:       2a02:8012:bc57:10::/64
                 │                              │
                 └──── routed WireGuard VPN ────┘

Only: branch users → head-office application API
Only: head-office management → branch server management
Never: guest Wi-Fi → either private site

A routed VPN joins chosen IP prefixes. Hosts keep their own broadcasts, DHCP traffic, and local failure domains. That is almost always the right default for home-to-office, branch-to-head-office, or office-to-datacentre connectivity. Layer-2 bridging across a VPN is specialised: it carries broadcasts and neighbour traffic across sites and has very different failure behaviour. Do not use it merely because two networks need to communicate.

1. 1. Bring up the encrypted interface

   Confirm the peer handshake and tunnel addresses. This proves the endpoints can exchange encrypted packets, not that either LAN is routed.

2. 2. Install routes on both sides

   Branch needs routes to head-office prefixes through the tunnel; head office needs the reciprocal routes. Every downstream router also needs to know where to return replies.

3. 3. Enable forwarding only on gateway hosts

   A workstation is not a router. The selected Saphira edge forwards between its LAN and VPN interfaces, then nftables decides the allowed directions.

4. 4. Test one allowed flow and one denied flow

   Use an address and port from each policy. A successful ping does not prove the application policy is correct.

# On the branch gateway: can its kernel select the VPN path?
ip route get 192.168.10.25
ip -6 route get 2a02:8012:bc57:10::25

# On the head-office gateway: check the return path.
ip route get 192.168.20.25
ip -6 route get 2a02:8012:bc57:20::25

# Observe the expected tunnel and LAN interfaces during a test.
tcpdump -ni wg0 host 192.168.10.25
tcpdump -ni servergw0 host 192.168.20.25

MailDragon and webDragon can remain publicly reachable through a public edge while SSH, databases, backups, monitoring, internal DNS, and administration travel only over a management VLAN or VPN. HAProxy publishes selected public listeners. A VPN gives private networks a path. They solve different problems and a serious deployment commonly uses both.

Internet
   │
   ▼
Public HAProxy / mail edge
   ├── TCP 25, 465, 587, 993 → MailDragon service network
   └── TCP 80, 443 → webDragon service network

WireGuard VPN
   └── SSH, backups, monitoring, internal DNS, databases

The VPN is not a reason to expose SSH on the public edge.

If both sites use 192.168.1.0/24, a gateway cannot know which 192.168.1.50 you mean. Renumbering is normally the clean solution. Translation and proxy workarounds exist, but they make logs, DNS, and future expansion harder. IPv6 is easier to plan: allocate a distinct /64 per LAN from your delegated or assigned prefix and record it before connecting the site.

VPN encapsulation reduces usable packet size. A broken MTU/MSS path often looks like a partial application failure: a web page begins loading then stalls, SSH connects but freezes on larger output, or small pings work while file transfers fail. Use tracepath and captures before randomly lowering every interface MTU.

tracepath 192.168.10.25
tracepath6 2a02:8012:bc57:10::25
ping -M do -s 1300 192.168.10.25
ip link show wg0

# Compare packet movement on tunnel and LAN while reproducing the failure.
tcpdump -ni wg0
tcpdump -ni eth0

• Check physical/provider reachability and the public endpoint address before changing VPN keys or routes.
• Check the tunnel handshake, then the tunnel interface addresses, then route selection, then the return route at the far gateway.
• Check forwarding and nftables counters on the two gateway hosts; a tunnel can be healthy while forwarding is denied.
• Check DNS only after a direct address test works. DNS is a name-to-address layer, not proof of connectivity.
• Keep a documented console, provider rescue system, or separate management path for each edge. A multi-site deployment that requires the broken VPN to repair the VPN is not recoverable.

• Configure a routed site-to-site WireGuard design
• Use the boundary-by-boundary troubleshooting method