Saphira Linux hits 713 Gbit/s aggregate TCP throughput

These are measurements taken from the Saphira system used to build and test the platform, not a synthetic marketing chart. A parallel TCP run reached 713 Gbit/s over ten seconds, transferring 830 GB. That is 0.713 Tbit/s — not 0.75 Tbit/s — so 713 Gbit/s is the headline we publish.

We said the exact number because it is the true number. We do not do marketing bullshit: if we cannot prove a figure with its command, route and terminal output, we do not tell it. It's so simple it hurts.

It's so fast the wires may get warm. Or they may not — claiming that they do would be marketing. What the evidence does show is that this has turned the heat up on an otherwise untuned network stack.

The useful finding is the shape of the scaling curve. Three streams reached 404 Gbit/s; four reached 524 Gbit/s; five reached 573 Gbit/s; and the observed eight-stream peak reached 713 Gbit/s. A second eight-stream run still delivered 691 Gbit/s with zero retransmissions. Higher concurrency did not improve the completed-run total: the nineteen-stream run recorded 312 Gbit/s with 33 retransmissions, suggesting CPU scheduling, queue contention and processing overhead had become the limiting work.

The following runs use Egg's routable address, 2a02:8012:bc57:c0ff::babe, rather than loopback. They are therefore reported separately from the local TCP results above.

From Minimer, the path crosses its gateway and gateway.akadata.ltd before entering Egg. The stated topology has two 1 Gbit/s sections before the 2.5 Gbit/s segment and KVM port. Both the single-stream and four-stream results settle at about 950 Mbit/s: exactly what the narrowest 1 Gbit/s path can deliver. Four streams divide the bandwidth almost perfectly at about 238 Mbit/s each; they fill the link, rather than bypass it. A later Minimer run repeats the result at 957 Mbit/s with one stream and 949 Mbit/s with two: the same three-hop path, the same 1 Gbit/s ceiling, and a second data point.

Route provides the next step up: its traceroute reaches Egg through gateway.akadata.ltd in two hops, over the stated 2.5 Gbit/s route into Homer, KVM and VirtIO. One TCP stream delivers 2.47 Gbit/s for ten seconds, transferring 2.87 GB with no retransmissions. Four streams also total 2.47 Gbit/s and transfer 2.88 GB, again with zero retransmissions. This is a real network-path result that reaches the expected 2.5 Gbit/s ceiling rather than a localhost or host-only measurement.

Spaghetti, a KVM guest at www.az2.co.uk, provides a separate three-hop route to Egg: VirtIO → Route → 2.5 Gbit/s → Homer → KVM. Its single-stream run transfers 2.86 GB at 2.46 Gbit/s; four streams transfer 2.87 GB at 2.47 Gbit/s. Both client results record zero retransmissions. This is a second routed, virtual-machine-to-virtual-machine validation of the 2.5 Gbit/s path.

The separate one-hop peer capture from c0ff::1reaches 226 Gbit/s with four streams, and the supplied eight-stream network summary records 177 Gbit/s. Those totals cannot have crossed a physical 2.5 Gbit/s port, so this page does not describe them as a 2.5 Gbit/s-wire result. The server output establishes the endpoints and totals; its virtual topology will be published when that route is captured.

These results use the existing system configuration. No benchmark-specific sysctl tuning was applied. Internal network segments use an MTU of 9000, and Open vSwitch is used extensively on Minimer, Homer and the current Route host to join guests, gateways and service networks.

The Route host currently runs Alpine Linux. It is intended to be replaced with Saphira-S after the pppd and rp-pppoe stacks have been proven. The planned external path isrp-pppoe → Route (10 GbE) → Route (2.5 GbE) → Homer (2.5 GbE) → KVM.

Homer's external connection establishes that the available Internet service is not the 1 Gbit/s ceiling seen on Minimer. A multi-connection speed test measured 2,216.19 Mbit/s download, 2,232.67 Mbit/s upload and 4 ms ping; download latency was 74 ms and upload latency 7 ms.

Egg is a 24-vCPU Intel Core i9-13900K virtual machine running under libvirt/KVM. Its VirtIO network device is configured with 24 queues. iperf3 was run as a TCP client and server on Egg for ten seconds per completed run.

The captured commands target ::1. This is therefore an IPv6 loopback benchmark: it exercises the Linux TCP/IP stack and host CPU, but does not traverse Egg's VirtIO NIC, Open vSwitch or a physical link. The queue count is useful system context, not the path measured by these results.

DNS may be active elsewhere in the test environment, but the recorded iperf3 commands use a literal IPv6 address. Name resolution is therefore not part of this measured TCP flow, and these results are not presented as a DNS benchmark.

No 800G network card or cable carried the loopback traffic. The guest-to-guest and externally routed cases above are separately labelled with their client address, route and available VirtIO/Open vSwitch context.

The eight-stream client command is reconstructed from the server transcript, which records eight accepted connections. The supplied client capture starts after that command, but records its final receiver total below. We keep that distinction visible rather than invent a complete transcript.

The zero-retransmission three-, four- and five-stream runs show that the aggregate result did not depend on a single stream. Full client and server transcripts are kept with the project as tests/iperf3.client.txt and tests/iperf3.server.txt.

Raw TCP is only the first layer. Saphira networking includes Linux, VirtIO, Open vSwitch, nftables, IPVS/LVS and the userspace needed to build real infrastructure. The next results will move up that stack: guest network topology and queue-count comparisons, Open vSwitch flow distribution, LVS/IPVS connection scaling, HAProxy HTTP/TLS workloads, IPv4-versus-IPv6 paths, and storage-backed service traffic.

The intention is to measure the whole path, not merely the easiest number to publish. Every benchmark here will retain its role, CPU allocation, queue count, protocol, command, duration and terminal output. Extraordinary numbers are easy to type. We prefer to show the console.

That is not a slogan. It is the rule: no proof, no claim. The truth can occasionally hurt, but it is more useful than marketing bullshit.

713 Gbit/s was not the target. It was simply what happened when Saphira had work to do. The interesting benchmarks are still to come.