RTFM · Load balancing
LVS and ldirectord: the Saphira IPv6-first director
Linux Virtual Server forwards packets inside the kernel at wire speed; ldirectord is the small daemon that tells it which servers are healthy. Saphira patches ldirectord into a first-class IPv6 and IPv4 director and pairs it with the external healthcheck library.
What LVS actually is
Linux Virtual Server (LVS) is not a process; it is the IPVS machinery built into the Linux kernel. You describe virtual services (an address, a port or a firewall mark, a scheduling algorithm) and the real servers behind them, and the kernel then forwards matching packets without any user-space program ever touching the stream. A proxy copies every byte twice; IPVS forwards once. That is why a modest box can balance connections that would embarrass a user-space proxy.
The part you configure is ipvsadm; the tool that prints and edits the kernel's forwarding table. The part that keeps the table honest is ldirectord: it runs health checks against each real server and adds or removes them from IPVS as they come and go. Director and doctor in one package.
| Method | How a reply gets home | Choose when |
|---|---|---|
| NAT (masq) | Reply comes back through the director, which rewrites it | Backends are plain RFC1918 hosts with the director as gateway; simplest |
| Direct Routing (gate) | Backend replies directly to the client using the VIP on a loopback | Backends share the director's LAN: line rate, the Saphira default |
| Tunnelling (tun) | Backend receives the packet inside IPIP and replies directly | Backends are on other networks/routed sites |
Firewall marks: one service, many ports
Some services cannot be split by port. FTP uses two connections; DICOM may use several; a site with HTTP and HTTPS wants both paths treated as one pool. LVS solves this with firewall marks: nftables tags the packets of a service with a mark, and IPVS balances on the mark instead of a port. ldirectord speaks marks natively; its virtual= line can name a mark instead of an address.
# nftables: everything for the DICOM service carries mark 21
table inet lvs-marks {
chain prerouting {
type filter hook prerouting priority mangle; policy accept;
meta l4proto tcp tcp dport { 104, 11112, 2762 } meta mark set 21
meta l4proto tcp tcp dport { 104, 11112, 2762 } ct mark set meta mark
}
}ldirectord never needs to resolve a mark to an address: the mark-to-VIP mapping is your nftables configuration, and the healthcheck scripts treat the mark simply as the service name ($1). Large marks (2,000,000+) and the IPv6 readback marker are validated against Saphira's ipvsadm 1.31.
What Saphira patches into ldirectord
Upstream ldirectord ships inside ClusterLabs resource-agents. Saphira carries resource-agents 4.18.0 with three patches that turn it into a first-class IPv6 + IPv4 director on a pure-musl system:
| Patch | Problem | What Saphira changes |
|---|---|---|
| 0001: Socket6 optional | Saphira's Perl has no Socket6, so upstream cannot even compile; and Socket6 returns error strings where core Socket dies | Uses the Perl core Socket getaddrinfo/getnameinfo, eval-wraps every call site, and passes address-family hints per backend |
| 0002: graceful degradation | Missing optional Perl modules (LWP, Net::LDAP, Net::DNS, DBI, Authen::Radius…) made ldirectord die and report healthy servers as DOWN | Every non-core check module degrades: HTTP negotiate falls back to core HTTP::Tiny, others fall back to a connect check, email alerts are skipped; never a false DOWN |
| 0003: IPv6 parsing hardening | Bare or unbracketed IPv6 addresses were silently mangled into a truncated address plus a bogus port; first-start could dereference an undefined ipvsadm readback | Bracket-aware host:port splitting in the config parser and name resolution; the readback of a not-yet-created virtual service is treated as 'no reals' instead of a crash |
The firewall-mark and IPv6 behaviour; -f mark [-6] ordering, large marks, the ipvsadm readback marker mapping to the fwm6 key, bracketed IPv6 real servers; is validated with exact config-side key parity. Bracketed IPv6 reals in ldirectord.cf parse exactly as written.
Write IPv6 services and reals with brackets, exactly as in the config snippets below. An unbracketed bare IPv6 address is left intact by the parser so that resolution fails loudly and clearly; never silently wrong.
A working configuration
# /etc/ha.d/ldirectord.cf
checktimeout = 3
checkinterval = 5
autoreload = no
quiescent = yes
# An HTTP pool, IPv4 VIP, LVS-NAT
virtual = 203.0.113.10:80
protocol = tcp
scheduler = wlc
real = 192.168.20.11:80 gate 1
real = 192.168.20.12:80 gate 1
checktype = external
checkcommand = "/var/lib/lb/saphira/lb.saphira.http"
# The same pool reached over IPv6
virtual = [2001:db8:10::10]:80
protocol = tcp
scheduler = wlc
real = [2001:db8:20::11]:80 gate 1
real = [2001:db8:20::12]:80 gate 1
checktype = external
checkcommand = "/var/lib/lb/saphira/lb.saphira.http"
# A firewall-mark service: DICOM on marks (see the nftables example above)
virtual = 21
protocol = fwm
scheduler = rr
real = 192.168.20.21:0 gate 1
real = 192.168.20.22:0 gate 1
checktype = external
checkcommand = "/var/lib/lb/saphira/lb.saphira.dicom.echo"quiescent=yes means a failed real server keeps its table entry with weight zero: existing connections finish, new ones avoid it. Set it to no to remove the server from the table outright. schedulers worth knowing: rr (round-robin), wlc (weighted least connections, the sensible default), sh (source hash, a client sticks to one server).
UDP and One-Packet Scheduling
LVS supports UDP as a first-class virtual-service protocol: declare the virtual service with protocol = udp, hang real servers off it exactly as for TCP, and let a UDP-aware check decide health. The dns, radius and sip check pages all show this shape.
# ldirectord.cf - a UDP virtual service (DNS resolver pool)
virtual = 203.0.113.10:53
protocol = udp
scheduler = wlc
real = 192.168.20.53:53 gate 1
real = 192.168.20.54:53 gate 1
checktype = external
checkcommand = "/var/lib/lb/saphira/lb.saphira.dns"A normal UDP virtual service behaves like a connection with a short fuse: the first datagram from a client creates an IPVS state entry, later datagrams from the same client follow one real server until the entry expires, and the entry then disappears. For most UDP services (DNS, NTP, RADIUS, syslog) that is exactly right: it is cheap, and it keeps a client's exchanges on one server for the life of a conversation.
Advanced, rarely needed. For unusual stateless UDP workloads, IPVS also provides One-Packet Scheduling (OPS), where each datagram is scheduled independently instead of following the normal UDP connection entry. Most installations will not need OPS, though it is available when the traffic model calls for it: the -o flag on a UDP virtual service, or on a firewall-mark service carrying only UDP, turns every packet into an independent scheduling decision.
# Create a UDP virtual service with One-Packet Scheduling:
ipvsadm -A -u 192.0.2.10:123 -s rr -o
# ...or switch an existing UDP service into OPS:
ipvsadm -E -u 192.0.2.10:123 -o
# Firewall-mark services carrying only UDP take the same flag:
ipvsadm -A -f 21 -s rr -oOPS is an ipvsadm-layer attribute: ldirectord's configuration has no flag for it, so a service that needs OPS gets its virtual entry managed at the ipvsadm layer (by hand or from a startup script), with the same checks driving real-server health as for any other service.
OPS removes the connection entry, so per-datagram health matters more, not less: pair an OPS service with the UDP family of checks (udp, sendexpect-udp, dns, radius, sip) so a dead real server stops receiving datagrams within one check interval.
Prove it works: UDP is really being scheduled
ipvsadm -Ln lists the UDP virtual service with its reals; datagrams reach a backend (tcpdump on the real server shows them arriving); and stopping the backend's service removes it from the pool within one check interval.
Operate it
# What is the kernel forwarding right now?
ipvsadm -Ln
ipvsadm -Ln --stats
# IPv6 services
ipvsadm -Ln -6
# The service manager (OpenRC on Saphira)
rc-service ldirectord status
rc-service ldirectord restart
# Watch a check run in the foreground while tuning
/var/lib/lb/saphira/lb.saphira.tcp 21 104 192.168.20.21 104; echo $?Prove it works: The director is really directing
ipvsadm -Ln lists every virtual service with its reals; stopping one backend's service makes ldirectord drop its weight to zero within checkinterval; traffic flows to the survivors; and when the backend returns, the weight is restored automatically.
Connect it to the check library
The checkcommand lines above are the whole integration: ldirectord runs the named script with the five standard arguments and reads the exit code. Every check page under the healthchecks section shows its ldirectord and HAProxy wiring.
Did we miss something?
If this page left something unanswered, found an error, or there is another subject you would like documented, tell us. Saphira’s documentation grows from real problems people need to solve.
Send feedback or request a new section →
Prefer not to do it yourself?
Everything needed to do the work yourself is documented here and remains free; we charge for human time, not for withholding knowledge. Sometimes the missing resource is simply time. The same people who build Saphira can provide paid professional help with implementation, migration, troubleshooting and administration.