Every commit on the hot path is gated against three counting tests. Not a marketing claim — a build invariant.
0
allocations
Zero malloc/free per packet, per request, per tick. Pre-allocated pools, RAII slot leases, stack-resident state.
CI gate · zero_alloc_proof — counting global allocator
0
locks
No mutex, rwlock, or spinlock on the hot path. Single-threaded shards, atomics for cross-thread coordination, lock-free SPSC rings.
CI gate · loom-verified atomics + 3-state futex protocol
0
copies
The TX path writes encrypted QUIC packets directly into kernel-bound buffers. No staging, no to_vec, no memcpy. Only DMA touches the bytes after that.
CI gate · perf gate "memcpy/pkt TX = 0"
Zero memcpy — with receipts.Turn on linux-af-xdp or land on kernel ≥ 6.18 for io_uring ZCRX — both shipped features — and the transport-path memcpys are gone. Only the two hardware DMAs remain, because that's how Ethernet moves bytes. Broadcast stays at 0 memcpy when producers write through SendBuffer; response stays at 0 memcpy with ZeroResponse native builders. End-to-end floor under AF_XDP: 2 DMAs, 0 application memcpy. Default io_uring without ZC modes is the portable fallback — 2 kernel memcpys above the DMAs, clean, labeled, predictable.
Architecture
One shard, one CPU one destiny
Each shard owns its sockets, io_uring ring, payload pool, and connection table. Nothing is shared on the hot path. Tokio still drives application code via a deliberate async bridge.
Backends are per-shard on the same Io instance — run io_uring everywhere, turn on linux-af-xdp on the shards that carry the hottest UDP traffic, mix freely. The protocol handlers above see the same RecvPacket / TxSink.
RoutingSO_REUSEPORT + eBPFor DCID hash dispatcher
DCID14 b · server_id + shard16 384 cluster slots, partitioned
CPU pinningpthread_setaffinity_npcpuset for tokio + alerts
NUMAFirst-touch on shard threadlocal-node pages
Inter-threadrtrb SPSC ringsfixed at boot, no MPSC contention
You configure the pools. Any count, any slot size, as many tiers as you want. RAII checkout, lifetime-scoped read guards, atomic refcounts. Allocations happen at boot — never in poll().
Pool A · example
small slots · tuned to your packet size
0 in flighte.g. 2 KB × N for QUIC datagrams
Pool B · example
large slots · optional, for jumbo / bodies
0 in flighte.g. 256 KB for H3 response bodies
Single pool, two pools, or any tiered layout. Config { pool_slot_count, pool_slot_size, ... } on Io::new; multiple pools via IoBuilder::pool(slot_size, count). The example above is one reasonable shape.
drop & recyclerefcount → 0, slot pushed back onto free stack
0 alloc · 1 atomic CAS
Stack-first
ArrayString, ArrayVec, SmallVec. If the size is bounded, it lives on the stack. Heap is a deliberate decision.
HugePages
MAP_HUGETLB on Linux. VM_FLAGS_SUPERPAGE_2MB on Intel macOS only — Apple Silicon's 16 KB native page already cuts TLB pressure. TLB misses eliminated on pools > 2 MB.
memfd_secret
TLS keys live in pages no other process can map, no swap, zeroized on drop via Zeroizing<T>.
NUMA-local
Heavy allocations happen on the shard thread after CPU pin. Linux first-touch places pages on the local NUMA node.
Syscalls
Five became one one becomes none
Default path costs five user/kernel transitions per request. io_uring collapses them into one. AF_XDP goes further: shared-memory rings, zero syscalls per packet in steady state.
Traditional · epoll
5
epoll_wait()
recvmsg()
process
sendmsg()
epoll_ctl()
Each syscall: TLB flush, user/kernel switch, msghdr copy.
io_uring · linked SQEs
1
io_uring_enter()
RECVMSG_MULTI + pbuf ring
SENDMSG_ZC linked
FUTEX_WAIT coalesced
SQPOLL · kernel-driven
With SQPOLL: zero io_uring_enter per tick — a kernel thread polls the SQ for you.
Kernel ≥ 6.9 lets the io_uring driver poll the NIC directly. Eliminates softirq latency under contended load.
Registered buffers
Pre-pin pool pages once. Skip get_user_pages on every recv: ~200 ns saved per packet.
UDP_GSO + GRO
Segment a 64 KB superpacket into 1500-byte frames in the NIC. One sendmsg, N wire packets.
TCP Fast Open
Piggyback request data on the SYN. First HTTP byte arrives in 1 RTT instead of 2.
splice / IORING_OP_SPLICE
File → socket without ever touching userspace. The page cache moves directly to the NIC.
kTLS
TLS encrypt offloaded to the kernel. Plaintext comes from the page cache, ciphertext goes straight to the NIC.
Message flow
The bytes never move only indices do
A packet arrives, lives in one pool slot. Sync and async handlers both read from that same slot. The response is written into another slot. What crosses threads is a 16-byte lease or a 16-byte Arc pointer — not the data.
Read it twice.Two pool slots appear in this diagram. Both are pre-allocated, registered with the kernel for zero-copy DMA, and written exactly once each. Between them live the handlers — sync reads the RX slot directly, async crosses to a worker via an Arc pointer over the same slot. At no point does a memcpy happen in the application path. The two DMAs at top and bottom are hardware transfers, not copies.
Wakeup
Three ways to wait pick your latency / CPU trade
Between two messages, the shard has to wait. The Wakeup trait is a ZST at runtime — the strategy is monomorphized into the loop, no virtual dispatch. Mix per shard: futex on the API tier, spin on the order-book.
FutexWakeup
Futex
The default.IORING_OP_FUTEX_WAIT on Linux ≥ 6.7. Three-state coalesced protocol. 0 file descriptors per shard, 1 syscall per wakeup cycle, kernel handles fairness. ~700 ns wakeup latency.
Best for: API tier · general workloads · per-CPU shard with mixed load
XdpPollMode::Auto
Adaptive
Best of both. AF_XDP's 4-tier ladder — Hot · Warm · Cool · Idle — promotes to busy-poll during traffic bursts, falls back to interrupt-driven sleep when idle. Tier eval every 100 ms, hysteresis to prevent flapping.
Best for: bursty UDP feeds · market data · DNS resolvers · NTP fleets
SpinWakeup
Spin
Cores you own. Pure std::hint::spin_loop(). 100 % CPU, no syscall, no kernel involvement. Sub-100 ns wakeup. Pairs naturally with AF_XDP busy-poll for sub-microsecond end-to-end latency.
Best for: HFT order books · market makers · ultra-low-latency RPC tier
Why 3, not 1.A REST tier handling 10 K conn/s wants Futex — sleep cheaply between bursts, share the CPU. A market-data multicast feeder wants Adaptive — busy-poll during the open, sleep after-hours. An order-router wants Spin — never sleep, never miss a quote. zero-io lets you choose per shard, the trait is monomorphized so the cold paths cost zero in the hot one.
Safety
The compiler is the contract use-after-invalidate cannot compile
Zero-copy in C and C++ requires runtime discipline, README warnings, and code reviews. zero-io encodes the invariant in the type system: Event<'poll> ties every borrowed &[u8] to the &mut Io from next_event(). The next io.poll() call mutably borrows Io — and the borrow checker rejects it as long as one byte of slot data is still in scope.
cargo buildcompile error
letSome(event) = io.next_event() else { return };
let data: &[u8] = match event {
Event::StreamFrame { data, .. } => data,
_ => return,
};
io.poll(Duration::from_millis(10))?;
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^// error[E0502]: cannot borrow `io` as mutable// because it is also borrowed as immutable// first borrow occurred here, used by `data`println!("{}", data[0]); // dead code — compiler stopped you
The lifetime IS the contract
No runtime check, no allocator inspection, no test that hopes to catch it. The annotation 'poll on the Event tells rustc exactly when the bytes die — and rustc enforces it before your code reaches a CPU.
Cross-thread? Use OwnedSlot
Need the bytes after poll or on another thread? io.detach_event_data() hands you an OwnedSlot — Send + Sync, Arc-counted over the same slot. The compiler still tracks it; the slot is dropped only when the last reference goes away.
What this doesn't need
No reference counting on the hot path. No bounds-check after a memcpy. No "did the kernel still own this buffer?" question. No // SAFETY: comment in handler code. The unsafe primitives live in PoolFreeStack, audited once, behind a typestate façade.
Slot<S> typestate
Pool slots transition Empty → Reserved → Committed → Released through generic state types. Slot<Reserved> has no .read() method ; Slot<Released> has no .commit(). Use-after-release, double-commit, write-without-acquire — all impossible to type-check. Wrong code refuses to compile.
Why this changes the game
Every other zero-copy network library — DPDK, userspace TCP stacks, custom C kernels — relies on documentation and reviews to keep callers honest. Rust's borrow checker turns "don't read this after the next poll" from a comment into a compile error. The result: zero-copy without the footgun tax.
Performance
What it costs what it doesn't
Architectural targets, not field-measured numbers. Production benchmarks land with the 1.0 release.
30–55M pps
Throughput
Per-shard, AF_XDP ZC_MODE. io_uring path: 5–15.
15–25ns/pkt
Latency
RX-to-handler hot path, AF_XDP. io_uring path: 50–100.
* AF_XDP ZC_MODE — driver dependent; fallback XDP_COPY_MODE matches io_uring. See Backends for supported NICs.
HTTP allocations · per request, warm pool
run_syncmatch-arm router
0B
run_tower + matchitgeneric Tower layers
64B
zero_io_axumdrop-in axum + HeaderMap pool *
200B
hyper + axumstock Tokio stack
~3,500B · 5–7 allocs
reqwest GETpopular client
~6,000B · 10–15 allocs
* header-map-pool feature flag · reclaims axum's HeaderMap after the response chain so a per-shard pool can re-issue it. Steady-state floor ~200 B per request; first request still pays the initial allocation.
Backends
One API the right backend for your box
From an embedded gateway in a forklift to a CDN edge node to an HFT order router — same code, different config. Memory tuned to the box (64 KB pool slots × 16 frames on a Pi, 64 MB UMEM × 16 K frames on a 100 G NIC), wakeup tuned to the workload (futex on the API tier, spin on the order book), backend tuned to the kernel (io_uring everywhere, AF_XDP where the driver supports zero-copy).
Identical API surface. EVFILT_USER fflags for targeted wakeups, sendmsg_x/recvmsg_x batch syscalls, hugepages on x86.
Windows ≥ 10 1809
RIO + IOCP
tier 1
Registered buffers give zero-copy TX on par with SENDMSG_ZC. Dedicated completion queues per listener.
Linux I/O strategies · pick per shard
Strategy
Protocols
TLS
Why pick it
io_uring(default)
UDP, TCP, all protocols via the kernel stack
kTLS · including NIC HW offload (mlx5 ConnectX-6+, etc.)
Portable, CI-gated, runs every protocol. 5–15 M pps, 50–100 ns/pkt. The right default unless you have a specific reason.
AF_XDPlinux-af-xdp
UDP native (kernel-bypass via UMEM). TCP via FreeBSD userspace stack on opt-in ports — the BSD TCP state machine ported to run on top of AF_XDP frames, replacing the kernel TCP path for those listen ports only *
software only · rustls on CPU (AES-NI accelerated). No NIC HW TLS offload — kTLS is a kernel feature and AF_XDP bypasses the kernel TCP stack by design.
UDP-heavy hot paths (market data, DNS at scale, NTP fleets) and HFT-tier TCP on specific ports. 30–55 M pps, 15–25 ns/pkt.
Driver-support footnote (canonical) — AF_XDP XDP_ZC_MODE requires zero-copy support in the NIC driver: mlx5 (Mellanox/NVIDIA ConnectX-4 Lx and later), i40e / ice (Intel X710/E810), ena (AWS Nitro), virtio-net (recent kernels). Fallback XDP_COPY_MODE works on every driver but adds one kernel memcpy (matches io_uring's cost). The FreeBSD userspace TCP path is opt-in per listen port via [userspace_tcp] enabled_ports = […]; other ports keep kernel TCP through XDP_PASS. TSO / GRO / LRO / kTLS are unavailable in the AF_XDP path by design — the cost of bypassing the kernel TCP stack.
Tier note — the per-OS table above is about API parity; the strategies table above is about Linux I/O backend maturity. Linux ≥ 6.7 ships at OS-tier 1 (full API), but its AF_XDP strategy is opt-in and CI-gated only when the feature is on. macOS and Windows are tier 1 for the OS surface (kqueue / RIO have parity); Linux's I/O strategies have their own tiering.
Deployment
Embedded or daemon your call, same API
Two ways to ship zero-io. Pick once at compile time. The protocol handlers, the Io surface, the event loop — identical. The only thing that changes is who owns the privileged file descriptors.
Embedded · default
Linked in
One process. zero-io is a library inside your binary. No IPC, no daemon, no extra moving parts. Simplest possible deployment.
Restart drops in-flight connections — fine for stateless tiers, dev environments, embedded targets, or anything where a cold cycle is acceptable.
cargo add zero-io
Daemon · production tier
Separated
Two processes. A privileged daemon owns the dangerous FDs (BPF, raw sockets, UMEM, TLS keys) ; your unprivileged app talks to it through a sealed memfd.
Hot-reload the app binary without dropping a packet — the daemon keeps everything alive across the execve. Privilege isolation, zero-downtime upgrades, multi-tenant safety.
cargo add zero-io --features daemon-client
How daemon mode works
The daemon holds CAP_NET_ADMIN, CAP_BPF, CAP_SYS_RESOURCE. The app runs as a regular user, zero caps. They share state through a sealed memfd mapping — UDS carries control only.
Privilege isolation
Daemon holds the dangerous bits. App runs setuid(nobody) + seccomp + landlock. A handler bug never escalates to BPF / raw socket reach. Compatible with Kubernetes securityContext.
Shared memory · perf-optimal
Connection state, UMEM, pool slots all live in a sealed memfd. App and daemon both mmap MAP_SHARED — same physical pages, no syscall on the data path. UDS only carries setup commands.
Binary hot-reload · zero packet drop
Ship a new binary, signal the supervisor, app execve's the replacement. Daemon-owned BPF / sockets / UMEM persist across the boundary. The new binary calls recover_from_exec() and rebinds — in-flight TCP and QUIC connections continue without notice.
Secrets that can't leak
TLS keys live in memfd_secret pages — no other process can map them, no swap, no /proc/<pid>/mem exposure. Wrapped in Zeroizing<T> so they're scrubbed on drop. Even a kernel exploit on the app side never reaches them.
Cert hot-reload, no restart
Rotate TLS certificates without dropping a single connection. CertReloadHandle::reload_from_pem(...) swaps via arc-swap — old connections finish on the old cert, new connections pick up the new one. Sub-microsecond cutover.
Audit trail · structured
Every privileged action — BPF load, cert reload, ops command, signal handler — emits a structured event on target = "audit::*". Actor uid/pid, before / after state, monotonic timestamp. Compatible with NIST SP 800-53 AU-2/3, OWASP ASVS §1.4.
Ops surface, locked down
Force actions (drain-tx, conn-kill, cert-reload, BPF reload) speak through a UDS /run/...ops.sock at mode 0600. Mandatory HMAC, two-phase commit for destructives, per-class rate limit, profile allowlist (dev / staging / prod), sealed-token for prod-only operations.
Multi-tenant safe
Per-shard connection tables, per-shard pools, no cross-tenant pointers. Tenant identity bound at handshake, enforced through the audit chain. Acceptable for cooperative tenants today; adversarial multi-tenancy = run multiple daemons.
Protocols
Every protocol one library
Pluggable ProtocolHandler trait. Each protocol is a feature gate. Pay only for what you use.
QUICtier 1
HTTP/3tier 1
WebTransporttier 1
TCPtier 1
UDStier 1
WebSockettier 1
HTTP/1.1 + 2tier 1
RESTtier 1
gRPCtier 1
MQTT 3.1.1 / 5tier 2
Redis · RESP2/3tier 2
FIX 4.4 · texttier 2
SBE · CME MDP3tier 2
SMTP · MIMEtier 2
FTP · FTPStier 2
DNS · DoT/DoHtier 2
mDNS · RFC 6762tier 2
NTP · SNTPtier 2
SOCKS5tier 2
Async bridge
Tokio when you want it not when you don't
Four runtime modes. Each picks a different point on the latency / ergonomics curve. The hot path stays sync; the application stays async.
Mode
Allocs
Body stream
Use case
run_sync
0 B
no
Inline handler, sync. Zero allocations. CPU-bound RPC, parsing, transform.
run_per_core
0 / 64 B
yes
Tokio runtime per shard. .await inline; 64 B for streaming bodies via FuturesUnordered.
* with AF_XDP ZC or io_uring ZCRX (kernel ≥ 6.18). Default io_uring path uses borrowed-send + zero-alloc QPACK vendor patches.
Code
Beautiful by design server · client
Same Io, same event loop. _listen for servers, _connect for clients. Zero allocations on the hot path either way.
Server
examples/server/quic_echo.rsRust
use zero_io::{Io, Config, Event};
fnmain() -> std::io::Result<()> {
letmut io = Io::new(Config::default())?;
io.quic_listen("0.0.0.0:4433".parse()?, &CERT_PEM, &KEY_PEM)?;
loop {
io.poll(Duration::from_millis(10))?;
while letSome(event) = io.next_event() {
match event {
Event::StreamFrame { conn, stream, data, .. } => {
// `data` is &[u8] borrowed from a pool slot// invalidated on the next poll() — process now or detach
io.stream_write(conn, stream, data)?;
}
_ => {}
}
}
}
}
examples/server/axum_drop_in.rsRust
// Before: tokio + hyperlet listener = tokio::net::TcpListener::bind("0.0.0.0:8080").await?;
axum::serve(listener, app).await?;
// After: zero-io. Two lines changed. Same axum router.letmut io = Io::new(Config::default())?;
io.http_listen("0.0.0.0:8080".parse()?)?;
zero_io_axum::serve(io, app)?.wait()?;
examples/server/tower_perf.rsRust
// 64 B / req. Generic Tower + zero-io native middleware. 0 alloc per layer.use tower::ServiceBuilder;
use zero_io_async::{ZeroRuntime, ZeroCorsLayer, ZeroTraceLayer, ZeroCompressionLayer};
let svc = ServiceBuilder::new()
.layer(ZeroTraceLayer::new())
.layer(ZeroCorsLayer::permissive())
.layer(ZeroCompressionLayer::zstd())
.layer(tower::timeout::TimeoutLayer::new(Duration::from_secs(10)))
.service(my_handler);
ZeroRuntime::new(io).run_tower(svc)?.wait()?;
Client
examples/client/quic_connect.rsRust
use zero_io::{Io, Config, Event, StreamId};
fnmain() -> std::io::Result<()> {
letmut io = Io::new(Config::default())?;
let conn = io.quic_connect("203.0.113.1:4433".parse()?)?;
loop {
io.poll(Duration::from_millis(10))?;
while letSome(event) = io.next_event() {
match event {
Event::Connected { .. } => {
// 1-RTT ready · session ticket cached for next 0-RTT
io.stream_write(conn, StreamId(0), b"GET /quote HTTP/3\n")?;
}
Event::StreamFrame { data, .. } => {
// `data` borrows the same pool slot the NIC DMA'd into
process(data);
}
_ => {}
}
}
}
}
examples/client/http_get.rsRust
// 0 allocs/req on warm pool · LIFO connection reuse · TFO + Happy Eyeballs +// TLS 0-RTT + Alt-Svc h3 auto-upgrade — all transparent.use zero_io::http_client::{HttpClient, HttpClientConfig};
// Async wrapper: drop-in replacement for reqwest at ~10× the throughput.#[tokio::main]async fnmain() -> std::io::Result<()> {
let client = HttpClient::new(HttpClientConfig::default())?;
let resp = client
.get("https://api.example.com/users/42")
.header("authorization", "Bearer …")
.send().await?;
match resp.status() {
200 => println!("{}", resp.text()?),
s => eprintln!("http {}", s),
}
Ok(())
}
examples/client/redis_pipeline.rsRust
// 100 SET + 100 GET in one round-trip via writev. RESP3 inline.use zero_io::redis::{RedisClient, RedisConfig};
letmut io = Io::new(Config::default())?;
let client = RedisClient::connect(&mut io, RedisConfig::localhost())?;
letmut pipe = client.pipeline();
for i in0..100 {
pipe.set(&format!("k:{}", i), format!("v:{}", i).as_bytes());
}
let results = pipe.exec(&mut io)?; // 1 round-trip, not 100
Backpressure
It bends before it breaks cascade, observed
One cascade state machine, four states. As pool utilization climbs, each step costs a little more — until the last one closes idle connections to recover. Every transition is observable, every drop is counted.
Seven pools feed the cascade
FreeStackpool slotsmain LIFO free list
FillRingRX descriptorskernel reads, driver fills
RxRingRX completionspackets ready to process
TxRingTX descriptorsoutbound queue
CompletionRingTX donebuffers reclaim here
ScratchPoolstagingshort-lived intermediates
PerConnper-connectionstreams, h3, WT state
Budget partition.60 % RX · 10 % TX critical · 25 % TX bulk · 5 % scratch. The 10 % critical-TX reserve is the reason handshake / ACK / FIN keep flowing even when the rest is being shed.
Healthy · < 60 %.All seven pools serving every caller. Fast path is direct: no observer events fire, no counters tick, zero overhead on the hot loop.
Warning · 60–80 %.Observer hook fires CascadeEvent::Warning, cascade_warning_total increments. No drops yet — head room shrinks but every request is still served. Hysteresis: must fall back to ~55 % to re-enter Healthy.
Critical · 80–95 %.send_buffer() returns Err(Pressure) for bulk TX — handlers see backpressure on the next poll and shed work themselves. The 10 % reserved partition keeps handshake / ACK / FIN flowing. New inbound connections refused (CONNECTION_REFUSED on QUIC, RST on TCP). CascadeEvent::Critical fires with the pool snapshot.
Drain · 95 %+.Idle connections close — anything with no recent traffic and an empty TX queue gets a graceful close (QUIC NO_ERROR, TCP FIN). Active connections with in-flight bytes are preserved. Pool slots return within microseconds; cascade typically falls back to Critical or below before the next poll completes.
Read the API
Every public type. Every protocol method. Every config knob.
OwnedSlot is a payload pulled out of the per-poll lifetime and made Send + Sync. Internally an Arc over a pool slot — refcounted, recycled on drop.
SendBuffer is a writable pool slot for zero-copy TX. Acquire via io.send_buffer(n), write into as_mut_slice(), hand to send_datagram_buffer / stream_write_buffer.
IoCluster · multi-shard
Production entry point for > 1 shard. Owns N reuseport sockets (or one shared socket + DCID dispatcher) and exposes the same listen / connect surface, fanned out across shards.
14 bits encode (server_id, shard_id) jointly in QUIC SCIDs — 16 384 cluster slots partitioned across the two fields. SERVER_ID_MASK = 0x3FFF, shard count is a power of 2 within each server.
ShardIo
Per-shard handle; identical surface to Io.
Pick one.Io for tests, single-core deploys, tools. IoCluster for production servers. Don't roll your own N Io instances — you'll miss the routing.
TCP RX is push-only. Data arrives via Event::StreamFrame { kind: MessageKind::Binary }. There is no tcp_stream_read; calling stream_read on a TCP ConnId returns IoError::StreamNotPullable.
~96–112 B owned struct. Send + Sync. Path / headers / body offsets stored as a 12-byte table inside the pool slot. Zero-copy accessors return &str slices into the slot.
fn path(&self) -> &str
fn header(&self, name: &str) -> Option<&str>
fn method(&self) -> HttpMethod
fn body(&self) -> &[u8] · inline body
fn body_stream(&mut self) -> Option<BodyStream> · streaming uploads, 8-slot SPSC ring per request
ZeroResponse mirrors this on the response side. Builders: ZeroResponse::ok().json(&value), ZeroResponse::not_found(), etc.
Two-line migration from axum::serve. Cost: ~200 B steady-state with the default header-map-pool feature (which reclaims axum's HeaderMap per request); 640 B without the pool. The HeaderMap itself is structural — axum's signature requires it.
Why zero-io is what it is. Mental model first, recipes when the protocols ship. For type signatures and method tables, see the API reference.
Why zero-io
zero-io is a Rust I/O library you reach for when you want predictable
network behavior — whether on a 100 GbE server, a Raspberry Pi, or a
NanoPi running headless in a closet. It exists because, on Linux 6.7+,
you can have zero allocations, zero locks, and zero copies on the hot
path — and most existing async stacks throw at least two of those away.
It's not a niche HPC / HFT library. It's a general-purpose I/O surface
that happens to scale up to those extremes when needed, AND scales down
to constrained boards where every CPU cycle and memory page matters.
Designed for
The library was built first for systems where the network was the
bottleneck, but the same shape happens to fit a lot of others :
General-purpose networked apps — anything you'd reach for tokio
for. The trade is a poll() loop instead of async fn, in exchange
for predictable latency and lower memory.
Embedded Linux boards : Raspberry Pi 4/5, NanoPi (R76S, R6S),
Orange Pi, OrangePi Zero, RockPro64 — single-core ARM SBCs benefit
enormously from "1 shard = 1 core, no allocations". The 200-300 MB
RAM headroom matters when you only have 1-4 GiB.
L4 / L7 proxies pushing 10 Gbps+ on commodity NICs (and far below
too — a 1 Gbps reverse proxy for your home server is the same code
shape).
HTTP/1.1 / HTTP/2 / HTTP/3 servers for any size of workload —
static file serving, API backends, full-page rendering.
Anything that today reaches for epoll + raw recvmsg/sendmsg
because the async runtime was in the way.
Where it shines on small hardware
Embedded use cases get an outsized win :
No allocator pressure : a Pi Zero with 512 MB RAM running a
long-lived service can't afford glibc allocator churn at high
request rates. Pre-allocated pools = constant memory profile, no
fragmentation drift.
No GC, no JIT, no warmup : zero-io binaries start in ~ms, hit
steady state immediately. Important on device reboots.
Single-thread shard fits the hardware : a 4-core SBC runs
IoCluster::new(4) without thread oversubscription tax. Or
Io::new() for single-core boards.
Cross-compile target : aarch64-unknown-linux-gnu, armv7-unknown- linux-gnueabihf, etc. The library has zero arch-specific assumptions
beyond the kernel floor.
A full HTTP+TLS+WebSocket fanout server fits in ~3 MB stripped binary
on aarch64. The runtime memory floor is ~5 MB at idle for default
config (resizable down via Config::small() if you're really
squeezed).
What it's still not for
Browsers / wasm32 in the page — the poll() shape doesn't map
to the browser's event loop. We do have a wasm32-unknown-unknown
target for WebTransport clients via web_sys, but that's a
remote-IPC client, not a server.
One-shot CLI scripts — if your program runs for 200 ms total,
the boot cost (creating the io_uring, registering buffers) isn't
amortized. Use reqwest blocking and move on.
"I want a microservice framework with built-in service mesh /
config / DI" — different category. zero-io is a transport
library ; you bring or build the higher layers.
How it differs
The trade is explicit. You hold a poll() loop. Events borrow from
the current cycle and die at the next poll(). State is single-threaded
per shard. You opt into multi-shard with IoCluster. Cross-thread is a
deliberate detach_event_data() -> OwnedSlot, not free. You get the
three zeros in return.
If you want async ergonomics for handler bodies, zero-io-async brings
back async fn shapes on top — without losing the underlying
predictability.
zero-io is not an async runtime. There is no executor, no Future, no
.await. There is a poll() loop. You drive it ; it never drives you.
loop {
io.poll(timeout)?; // one tick : drain CQEs, run handlers, flush TX
while let Some(ev) = io.next_event() {
match ev { /* … */ } // events are valid until the next poll()
}
}
Three rules govern everything that follows.
1. One shard, one thread
Io is !Send. The shard owns its sockets, its io_uring, its pool, its
connection table. It runs on a single OS thread, period. If you want more cores,
spawn IoCluster::new(N) — N independent shards, no shared mutable state, the
kernel routes packets to them via SO_REUSEPORT or BPF.
Why : a single thread eliminates locks on the hot path. Every concurrency win
that a multi-threaded design would buy is paid back in cache misses, false
sharing, and atomic fences. Fan out at the kernel layer instead.
2. Events borrow from the current poll()
next_event() returns Event<'poll>. The 'poll lifetime is tied to
&mut Io, which means the event's data is a slice into a pool slot owned by
the shard. The borrow is released when you call poll() again. The compiler
will refuse to let you store an event past that boundary.
If you need the data longer :
Same thread, simplest : data.to_vec() — one alloc, one memcpy, problem
solved.
Cross thread or longer-lived, zero-copy :
io.detach_event_data() returns an OwnedSlot, which is Send + Sync, holds
a reference-counted handle into the pool, and survives across poll() cycles.
Drop it to release the slot.
3. Buffers come from pools, not the heap
Vec::new() is fine in cold paths — config parsing, logging, storage. On
the I/O hot path, every buffer comes from a pre-allocated pool. You call
io.send_buffer(n) to check out a slot, you write into it, the kernel
reads from it, the slot returns home on completion. No allocator
involvement.
let mut buf = io.send_buffer(payload.len())?; // O(1) checkout, no alloc
buf.write(payload); // copy_from_slice into the slot
io.udp_send(endpoint, peer, buf)?; // kernel takes the slot
The pool is sized once at boot from Config and never grown. The third
zero — zero-alloc on the hot path — is a direct consequence : if you
don't reach for the heap, the heap can't slow you down.
If you internalize these three rules — one shard one thread, events
borrow from this poll(), buffers come from pools — the rest of the API
tells itself.
See also : the poll cycle goes phase-by-phase
through what poll() actually does. Event lifetimes
covers the borrow-checker patterns and the to_vec() vs OwnedSlot
decision tree. Pool system explains where slots come
from and what states they go through.
5-minute tour
The shortest useful program. UDP echo server, single shard, default config.
This is the canonical "is the API as small as it claims" test.
use std::time::Duration;
use zero_io::{Io, Config, Event};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
io.udp_bind("0.0.0.0:8080".parse().unwrap())?;
loop {
io.poll(Duration::from_millis(100))?;
while let Some(ev) = io.next_event() {
if let Event::UdpRecv { endpoint, from, data } = ev {
let mut buf = io.send_buffer(data.len())?; // checkout pool slot
buf.write(data); // 0 copy : write into slot
io.udp_send(endpoint, from, buf)?; // ship it
}
}
}
}
What's happening, beat by beat :
Io::new(Config::default()) allocates one io_uring, one payload pool,
one connection table. All sized from Config ; defaults are sane for ~1k
connections.
udp_bind(addr) opens a socket, registers the fd with the ring, arms the
first recvmsg. Returns an EndpointId you'd keep around if you bind
several.
poll(100 ms) submits any pending SQEs, blocks up to 100 ms in
io_uring_enter, drains the completion queue. On return, next_event()
yields whatever the kernel delivered.
Event::UdpRecv { data, .. } is a slice into a pool slot. Valid until
the next poll(). We don't need it longer here, so we just read it inline.
send_buffer(n) checks out a writable slot of at least n bytes from
the same pool. Zero alloc. buf.write(data) is a copy_from_slice — that
one copy, kernel→pool→kernel, is the only memcpy on the data path.
udp_send(...) queues an SQE pointing at the slot. The slot is held by
the kernel until the send completes ; the next poll() will see the
completion CQE and free it.
That's the whole I/O surface for UDP. QUIC, TCP, HTTP, WebSocket follow the
same shape : *_listen / *_connect, Event::* variants, *_send /
stream_write / respond for output.
For working code in every protocol, see the recipes : start with
UDP echo (this same shape, three views) and
TCP echo ; the rest of the catalog covers QUIC, HTTP,
WebSocket, gRPC, MQTT, and more.
See also : 4-layer architecture for what
sits behind the udp_bind / udp_send calls ; the poll cycle
for what poll(100ms) actually did over those 100 ms.
4-layer architecture
Four layers, top to bottom. You usually only see Layer 1. You can drop to
Layer 3 when you need a custom protocol handler. Layer 4 is the OS-specific
I/O backend ; you don't touch it.
Layer 1 — Public API
What user code sees. Io, Event<'poll>, SendBuffer, Config ; the
constructors and verbs : udp_bind, quic_listen, quic_connect,
tcp_listen, tcp_connect, wt_connect, etc. The surface is small on
purpose — most protocols have a *_listen and a *_connect, the rest of the
work happens on Event variants. See the API reference for the full
type signatures.
Layer 2 — Protocol handlers
UdpHandler, QuicHandler, TcpHandler, HttpHandler, WsHandler,
WtHandler. Each implements the ProtocolHandler trait. They translate raw
kernel events into Event variants and produce TX writes. The QUIC handler
embeds quiche (vendored, with our zero-alloc dgram patches and zero-alloc
HPACK encode patches — see vendor patches for detail). The HTTP
handler embeds httparse for HTTP/1.1 and a vendored h2 for HTTP/2.
You write a Layer 2 handler if you have a wire protocol we don't ship and you
want to live inside the same shard, sharing the pool and the event loop. The
API reference has the trait signature.
Layer 3 — Shard engine
ShardResources — the bag of "everything one shard owns". run_tick() runs
one cycle. TxSink, StreamSink — backpressure-aware output. TimerWheel —
hashed wheel, O(1) insert / fire. DirtyQueue — handlers register dirty work
to be processed at the end of the tick. IoEvent — the queue from which
next_event() drains.
You drop to Layer 3 if you want fine-grained control : custom poll order,
hand-rolled tick scheduler, or you're embedding zero-io as a component inside a
larger system that drives the loop itself. ShardResources is pub ; it's not
the recommended path but it's not hidden.
Layer 4 — I/O backends
Three backends, one trait (PacketBackend / ShardBacking). Tier 1 = our
production target (full feature parity, perf-tested) ; Tier 2 = supported
with reduced capability ceiling ; Tier 3 = best-effort port. Full
breakdown in Tier 1 / 2 / 3.
Platform
Backend
Status
Linux 6.7+
UringCore + UringUdp + UringTcp
Tier 1
Linux 5.5+ optional
XdpCore + XdpUdp (AF_XDP)
Tier 1 opt-in (174b)
macOS 14+
KqueueCore + KqueueUdp + KqueueTcp
Tier 2
Windows 10 1809+
RioCore + RioUdp + RioTcp (RIO+IOCP)
Tier 3
The trait is the seam. Layer 3 calls backend.submit(sqe) / backend.drain()
without knowing what's underneath. Layer 4 differences (registered buffers,
SQPOLL, busy-poll, zero-copy TX) bubble up via capability flags read at boot.
A backend is not a fallback for another. They are different implementations
of the same contract. The compile_error! guard ensures you don't try to
build, say, AF_XDP on macOS — there's no platform-shim, just a clean refusal.
Why four layers, not three or five
Three would conflate "the API surface" with "the engine internals" — anyone
extending the library would have to reach into private fields. Five would split
the engine into "scheduler" + "I/O dispatcher" + "buffer manager", which sounds
clean on paper but doubles the trait surface for nothing — those concerns are
already isolated inside Layer 3 by struct boundaries (TimerWheel, TxSink,
etc.) without needing a public seam.
The point of the layering : if you stay at Layer 1, you write 30 lines of code
and never see the rest. If you need to push lower, the seams are exactly where
you'd expect them.
See also : pool system lives between Layer 3 and
Layer 4 ; the poll cycle is Layer 3's run_tick()
unrolled.
The poll cycle
A poll() is one tick. It does six phases in a fixed order. Knowing the order
matters because it determines when handler state machines see new data, when
TX is flushed, and when timers fire.
Phases, in order
1. Release guards
Slots that were Borrowed for Event<'poll> in the previous tick are dropped.
This is what enforces "events die at the next poll" — by the time you re-enter
next_event(), the previous events' slots are already returned to the pool.
2. Submit + wait
io_uring_submit_with_args(timeout). Submits any SQEs we queued (writes from
the previous tick that haven't drained yet) and blocks up to timeout waiting
for completions. With SQPOLL enabled, submission is asynchronous (kernel
thread polls our SQ ring), and we only wait for completions.
Timeout strategies :
Timeout
Use case
Duration::ZERO
Busy-spin tight loop ; no kernel sleep ; max throughput, max CPU
Duration::from_millis(1)
Latency-sensitive, want sub-ms reaction to new packets
Duration::from_millis(100)
Default, batches well, low CPU when idle
Duration::MAX
Sleep until I/O ; lowest CPU when idle ; relies on event-driven wakeup
The wakeup story (process futex / __ulock / Named Events) is what lets
Duration::MAX actually wake on cross-process IPC. See the wakeup section
on the home page for the visual.
3. Drain CQE
Walk the completion queue. For each CQE :
RX completion (recvmsg returned) : look up the slot by buf_id, route
to the protocol handler that owns the endpoint, queue an IoEvent.
TX completion (sendmsg returned) : transition the slot
InFlight → Completed, drop our reference, slot returns to free.
Timer completion : TimerWheel tracks them out-of-band, this CQE is
acknowledgment only.
Routing is constant-time : the SQE/CQE user_data u64 is encoded as
(tag: u8, op: u8, payload: u64) (plan 174 §174.5). The tag
discriminates the protocol handler family (UDP, QUIC, TCP, …),
dispatched via the AnyHandler enum. The payload carries the slot or
connection or stream id needed to fetch the right context. No hash, no
map lookup on the hot path.
4. Process dirty
Handlers register themselves dirty when they have state machine work to do
(QUIC ACK to send, HTTP/2 SETTINGS to ACK, timer fired, flow window updated).
The dirty queue is processed once per tick. Each dirty handler advances its
state machine, possibly producing more IoEvents and more TX writes.
5. Flush TX
TxSink is a per-shard backlog of pending writes. Each handler appends to it ;
flush_tx() drains it into io_uring SQEs, batching where possible (using
io_uring's linked SQEs for send → recv chains, or IORING_OP_SENDMSG_ZC
for zero-copy when supported).
If the kernel's submission queue is full, partial flush ; the remainder waits
for the next tick. This is one of the natural backpressure points — pool
slots stay Committed until they get an SQE slot.
6. Fire timers
TimerWheel.tick(now_monotonic). All timers due at this tick fire ; their
callbacks queue more dirty handlers or IoEvents. The wheel is hashed (O(1)
insert and tick) ; default 64 buckets at 1 ms granularity, configurable.
What you can't do : nested poll()
Calling poll() from inside an event handler is a programming error. The
borrow checker prevents it (each handler is called with &mut self of the
shard, which would be aliased), but if you reach across owned types and try to
construct a second &mut Io, you'll panic. Don't.
Latency budget
Design targets per-phase, not measured numbers — projections from internal
micro-benchmarks on a modern x86 box, idle workload. Actual figures
publish with the benchmark CI gate.
Phase
Target
1. release_guards
< ~100 ns
2. submit + enter
depends on timeout (≥ wait time)
3. drain_cqe
~30 ns / completion (target)
4. process_dirty
depends on handlers
5. flush_tx
~50 ns / submission (target)
6. fire_timers
< ~100 ns / fired timer (target)
End-to-end latency (NIC RX → user code → NIC TX) on Linux 6.7+ with SQPOLL
enabled targets low single-digit microseconds in lab conditions. We'll
publish actual numbers when the benchmark CI gate lands.
Event<'poll> is the most "Rust" thing in the API. It's also the thing
beginners trip over. This page is the explicit decision tree.
The rule, restated
fn next_event(&mut self) -> Option<Event<'_>>;
'_ is &mut Io's lifetime. The borrow checker sees : "Event holds a
reference to Io. As long as that Event exists, Io is borrowed mutably."
Calling io.poll(...) again requires &mut Io, which is already borrowed,
which is a compile error if any Event from the previous tick is still alive.
In plain words : events die at the next poll(). Always, for everything
they carry.
Pattern : process inline (default, recommended)
io.poll(timeout)?;
while let Some(ev) = io.next_event() {
match ev {
Event::UdpRecv { data, from, .. } => handle_udp(data, from),
Event::StreamFrame { data, conn, .. } => handle_stream(data, conn),
_ => {}
}
} // ev dropped here, borrow released
io.poll(timeout)?; // works, no borrow conflict
This is the shape 80% of code takes. Inside the while let, you have full
read access to data: &[u8]. You parse, you deserialize, you act, you move on.
No copy.
Anti-pattern : store events for later
let mut events = Vec::new();
while let Some(ev) = io.next_event() {
events.push(ev); // ❌ won't compile
}
io.poll(timeout)?; // would invalidate events
for ev in events { /* ... */ }
Doesn't compile. The Vec<Event<'_>> would need 'static to outlive the next
poll(), and Event<'poll> is by definition not 'static.
The compiler error looks like :
error[E0502]: cannot borrow `io` as mutable because it is also borrowed as immutable
--> src/main.rs:N:5
|
M | events.push(ev);
| ----------- immutable borrow occurs here
N | io.poll(timeout)?;
| ^^ mutable borrow occurs here
If you got this error, you wanted one of the patterns below.
Pattern : detach for cross-thread
io.poll(timeout)?;
while let Some(ev) = io.next_event() {
if matches!(ev, Event::StreamFrame { .. }) {
let owned = io.detach_event_data(); // OwnedSlot, Send + Sync
worker_tx.send(owned); // ship to async worker
}
}
detach_event_data() returns Option<OwnedSlot>. It bumps the slot's
refcount, decouples it from the poll lifetime, and gives you a handle that is
Send + Sync and survives across poll() cycles. When all clones are
dropped, the slot returns to the pool. Zero-copy, refcount-bumped.
This is the path for : tokio bridge, cross-shard relay, async DB lookups,
anything where the work outlasts the tick.
Pattern : same-thread, simpler, one copy
let snapshot: Vec<u8>;
while let Some(ev) = io.next_event() {
if let Event::StreamFrame { data, .. } = ev {
snapshot = data.to_vec(); // 1 alloc + 1 memcpy
break;
}
}
io.poll(timeout)?; // fine, snapshot is owned
process(&snapshot);
data.to_vec() is one Vec allocation and one memcpy of the bytes. If you
need the data once, on the same thread, and you're not in a hot path, this is
the simplest answer. Don't reach for OwnedSlot if to_vec() does the job.
Pattern : send Event to another thread
let ev = io.next_event().unwrap();
worker.send(ev); // ❌ won't compile : Event<'poll> isn't Send
Event<'poll> is intentionally !Send. The slot reference is single-threaded
by design. The fix : detach_event_data() first, which gives you OwnedSlot
(Send + Sync), then send that.
Decision tree
Need data after poll() returns ?
└── No → use Event<'poll> inline. Done. (0 copy)
└── Yes → Same thread ?
└── Yes → small data, hot path → io.detach_event_data() → OwnedSlot. (0 copy)
│ small data, cold path → data.to_vec(). (1 copy, simpler)
└── No → io.detach_event_data() → OwnedSlot. (0 copy, Send + Sync)
The compile-fail test suite (plan 173.4b) covers the four anti-patterns and
asserts the rustc error stays helpful. If the type system ever lets one of
these slip, that's the regression.
See also : the poll cycle — the 'poll
lifetime corresponds to phases 1-3 of one tick, and the release of a
borrow is what phase 1 of the next tick frees ; why typestate
for why these constraints live in the type system rather than as runtime
assertions ; pool system for what OwnedSlot is a
handle to.
Pool system
The pool is the load-bearing wall. It's how zero-alloc and zero-copy actually
hold up. Spend five minutes here.
Why a pool
Every packet you send or receive needs a buffer. The naive answer is Vec::new
— allocate per packet, free when done. That works fine until you're doing
1M packets/sec, at which point the allocator becomes the bottleneck (cache
contention, fragmentation, LIFO pressure on tcmalloc/jemalloc).
The pool answer : pre-allocate a fixed array of slots at boot, hand them out
on demand, return them when free. O(1) checkout, O(1) release, no allocator
involvement. Zero alloc on the hot path is a direct consequence of the pool's
existence.
What's in a slot
A slot is a fixed-size aligned buffer (default 1500 bytes for QUIC, configurable
via Config.pool_slot_size). The pool stores them in a contiguous arena. Each
slot has metadata (current state, generation counter, refcount when shared) in
a parallel array — kept separate so the buffers themselves stay tightly packed
for DMA.
Slot typestate (6 states)
A Slot<'p, 'b, S> is parameterised by its state S. The compiler enforces
state transitions ; you cannot read from a slot you haven't filled, or release
one that's still in flight.
What each state means :
State
Meaning
Who has it
Reserved
Just allocated from the free stack, not yet written
TX user with SendBuffer
Committed
Filled, ready to be submitted to io_uring
between buf.write() and udp_send()
InFlight
SQE submitted, kernel owns it
until CQE arrives
Completed
CQE came back, ready to be released
brief, before drop
Borrowed
RX path : kernel filled it, user reads via Event<'poll>
until next poll()
SharedRead
Same buffer fan-out to N consumers (GRO)
until refcount hits 0
The state is encoded in the type. Slot<_,_,Reserved> doesn't have read().
Slot<_,_,Completed> doesn't have submit(). The compiler refuses misuse —
no runtime check, no panic, no UB.
share_into<N> for GRO fanout
The kernel can deliver one big UDP "super-packet" containing N segments via
GRO (Generic Receive Offload). Instead of N memcpys to N consumers, you call
slot.share_into::<N>() once and get N Slot<_,_,SharedRead> siblings — same
buffer, refcounted N times, each consumer drops its handle independently. Last
drop releases the slot.
This is single-producer-multiple-consumer fan-out at zero alloc — the cost
is N atomic refcount bumps (sub-ns each), no buffer copies. Pub/sub at the
shard level uses it.
Brand-generic pool — what 'b is for
The lifetime parameter 'b on Slot<'p, 'b, S> is a brand. Two pools
with different brands cannot mix slots — the compiler refuses. Brands are
created via the HRTB closure pattern pool.with_brand(|brand_pool| { … }) :
pool.with_brand(|p| {
let s = p.checkout()?; // s : Slot<'p, 'b, Reserved>, brand 'b sealed
// ... use s exclusively inside this closure ...
});
Why : with multiple pools (small slots vs large slots, or per-shard pools in
multi-shard), it would be a logic bug to release a small slot back to the
large pool's free stack. The brand makes that bug uncompilable.
Doubling for safety, since some slots are RX-borrowed by handlers between
poll() boundaries and the kernel must always have spare slots to fill.
For a server pushing 200 kpps with a 100 µs tick budget : 200_000 × 100 / 1_000_000 × 2 = 40 slots, plus accept some headroom — Config::default()
gives 1024 to start with, which is fine for most things below 10 Mpps.
If you exhaust the pool, you get IoError::PoolExhausted and an
Event::PoolPressure to signal the cascade. See
backpressure cascade for the recovery
mechanics.
See also : zero-copy for the RX / TX paths through
the slot, why typestate for why the 6 states live
in the type system, event lifetimes for what a
slot looks like from the user side.
Zero-copy
This page is the detailed accounting. If you only remember the diagram from
The Three Zeros, this expands the where-and-why.
RX path
NIC kernel CQE pool slot Event
| | | | |
|---DMA write-->| | | |
| |--with buf_id->| | |
| | |---lookup----->| | pool slot now [Borrowed]
| | | |----&[u8]----->| user reads (0 copy)
| | | | |
| | | | | next poll() releases the slot
End-to-end : NIC → DMA → pool slot → &[u8]. Zero userspace memcpy.
The DMA is hardware ; we don't count it.
Steps :
pool.register_buffers(&io_uring) at boot. We hand the ring an array of
our pool slots ; the kernel keeps pointers to them.
recvmsg SQE with BUF_RING flag. The kernel waits for a packet, picks
any free slot, writes the packet into it.
CQE has buf_id — the index of the slot that got filled. We look it up
in our pool, the slot transitions Reserved → Borrowed.
Event<'poll>::UdpRecv { data, .. } carries &[u8] pointing into the
slot.
User code reads data directly. There is no userspace memcpy. The slot is
released back to free state on next poll() when the borrow expires.
TX path
handler SendBuffer pool slot kernel NIC
| | | | |
|-send_buffer-->| | | |
| |--checkout---->| | | pool slot becomes [Reserved]
| |--write(data)->| | | pool slot becomes [Committed] (THE one userspace memcpy)
|--udp_send---->| | | |
| |---sendmsg---->|---SQE entry-->| | pool slot becomes [InFlight]
| | | |---DMA read--->|
| | | | |
| | |<--CQE---------| | pool slot becomes [Completed]
| | | [free] | |
End-to-end : user → pool slot (1 memcpy : copy_from_slice) → kernel
DMA → NIC. One userspace memcpy on TX. If the source bytes are
already in a pool slot (e.g., echo from RX), use body_buffer /
SendBuffer::as_mut_slice() to skip even that copy.
Steps :
io.send_buffer(min_size) checks out a slot from the free stack. The
slot is now Reserved, the user owns a SendBuffer (typed wrapper).
buf.write(&[u8]) is one copy_from_slice from your source bytes into
the slot. This is the one TX memcpy. If your source is already in a pool
slot (e.g., echo from RX), there's still this copy ; if it's in a Vec you
were building, same.
io.udp_send(endpoint, peer, buf) queues a sendmsg SQE pointing at
the slot. The slot transitions Committed → InFlight. The kernel reads
from it (DMA to NIC) ; we don't touch it until completion.
CQE arrives on the next poll(). Slot transitions InFlight → Completed → free. Available for the next checkout.
Where the copies actually are
Stage
Memcpy ?
Userspace cost (1500 B target)
NIC → kernel buffer
No (DMA)
0 ns
Kernel buffer → pool slot
No (BUF_RING is direct DMA into our slot)
0 ns
Pool slot → user code
No (&[u8] slice, no copy)
0 ns
User write → pool slot (TX)
Yes (copy_from_slice)
tens of ns (bench-gated)
Pool slot → kernel
No (DMA, kernel reads our slot directly)
0 ns
Total userspace memcpy per round-trip : 1, on the TX side, and only because
your source bytes have to make it into a slot somehow. If your handler can
write directly into the SendBuffer (e.g., serializing a struct), even that
disappears — SendBuffer exposes as_mut_slice().
What Event<'poll> actually borrows
Event<'poll> { data: &'poll [u8], … }. The 'poll lifetime is &mut Io. Any
struct that holds Event<'poll> borrows Io mutably until it's dropped.
Practically : you can't store events in a Vec and drain later — that would
require 'static, which the lifetime denies. You either consume them inside
the while let Some(ev) = io.next_event() loop, or you call
io.detach_event_data() to promote the data to OwnedSlot (zero-copy
detachment, refcount bump on the slot).
OwnedSlot is Send + Sync. It's the piece you ship to a tokio task or
another shard via a channel. When it's dropped (or all clones are), the slot
returns to the pool. No additional copy.
"Zero copy" vs reality
We don't claim zero copies including DMA — that would be a hardware lie. We
claim zero memcpys in userspace on the data path, plus the one TX-side write
that's structurally required (your bytes have to enter the kernel-shared
buffer somehow). The CI gate (zero_alloc_proof) measures both : zero allocs
and zero __memcpy calls on a 10k-message echo workload.
If you're seeing different numbers in your own profile, profile in release
mode, link with -C target-cpu=native, and check that you're not doing a
Vec<u8>::clone() somewhere in your handler. The library doesn't memcpy ;
your code might.
See also : pool system for the slot mechanics
behind these paths ; event lifetimes for the
Event<'poll> borrow rules and the OwnedSlot detach pattern ;
The Three Zeros for the synthesis with the CI
gate accounting.
The Three Zeros
Each previous chapter showed a piece of the mechanism. This chapter is the
synthesis — what "zero alloc, zero lock, zero copy on the hot path"
precisely means, what's counted in the claim, what isn't, and how CI keeps
us honest.
Zero alloc
The hot path is everything that runs inside a poll() cycle once steady
state is reached : RX from the kernel, route to a handler, run the handler's
state machine, queue TX, fire timers, return from poll(). That entire
cycle does not touch the global allocator.
How it's enforced :
All buffers are pre-allocated at boot — the payload pool, the
connection table, the timer wheel, the dirty queue, the io_uring SQ/CQ.
Sized once from Config, never grown. The mechanism is the
pool system.
No Vec::push / String::push_str / Box::new in the path. Stack
buffers (ArrayVec, ArrayString) for short transient state ; pool
slots for everything that crosses an async boundary.
The zero_alloc_proof test wraps a counting allocator around a
representative workload (1k QUIC datagrams round-trip) and asserts the
count delta is exactly zero. CI gates it on every PR.
What's not covered : Io::new(), udp_bind(), quic_listen() — boot
allocation is fine, you only do it once. Connection accept is also fine —
it allocates a connection slot from the pre-sized table, which is O(1)
and does not touch the global allocator at steady state.
Zero lock
Single-threaded shards. No Mutex, no RwLock, no parking_lot. Not a
single atomic fence on the hot path inside one shard. The shard owns its
sockets, its io_uring, its pool, its connection table — the
mental model rule "one shard, one thread" is what
buys this.
Cross-shard relays (used by DcidDispatch for multi-shard QUIC) use
single-producer single-consumer ring buffers with Acquire / Release
ordering — wait-free, no contention, predictable latency.
The one place atomics show up on the data path : IoHandle, the
cross-thread send-side handle used by tokio bridges and external
producers. IoHandle::send enqueues a command on a lock-free MPSC ring,
which uses CAS on the producer index to reserve a slot. The shard reads
the consumer side without any atomic. Single-digit nanoseconds in the
contention-free case ; counted in the cost, not in the
hot-path zero-lock claim, because contention only matters when multiple
producers fight over the same shard.
Zero copy
Two paths, one userspace memcpy in the entire round-trip.
RX : NIC → DMA → pool slot → &[u8] to your handler. Zero memcpys
from kernel to user. The slot transitions Reserved → Borrowed ;
Event<'poll>::UdpRecv { data, .. } carries a slice into it.
TX : your bytes → pool slot (one copy_from_slice) → kernel →
DMA → NIC. The single TX-side copy is structurally required — bytes
need to enter the shared buffer somehow. If your handler writes the
bytes itself (serialising a struct into SendBuffer::as_mut_slice()),
even that disappears.
The full per-stage accounting — including BUF_RING registration,
SendBuffer mechanics, and the brand-generic pool — lives in
zero-copy. The short version : zero memcpys on RX,
one on TX, zero everywhere else.
The cost accounting, made explicit
What we count :
Allocations (counted via zero_alloc_proof)
__memcpy calls in userspace (counted by the same gate, must equal 1
per round-trip)
Atomic fences inside the shard (must equal 0)
Lock acquisitions inside the shard (must equal 0)
What we don't count, by construction :
DMA — hardware moves bytes ; not a userspace copy.
Cross-thread atomics on IoHandle ingress — counted in the cost,
excluded from the hot-path zero-lock claim.
Kernel-internal copies — outside our scope, observed only via syscall
cost.
What the three zeros buy
Predictable latency — no allocator hiccups, no lock contention, no
GC pauses. p99.9 follows p99 closely.
Linear core scaling — single-threaded shards, multi-shard at the
kernel layer means N cores = N× throughput, no Amdahl tax until you
hit NIC limits.
Honest perf budget — every nanosecond is in code you wrote.
There's no runtime hiding behind the curtain.
The cost is what you'd expect : the API is opinionated, lifetimes are
explicit, the shape isn't async/await. That's the trade.
See also : why no async for the trade explained
more fully ; why typestate for why the slot states
are encoded in the type system ; benchmarks methodology
for how the CI gate works in practice.
Single shard vs cluster
The default is Io — one shard, one thread. You upgrade to IoCluster when
one core saturates. Here's how you know, and how you switch.
When to stay single
Throughput is below 1 Mpps — single-threaded shards routinely handle
1-2 Mpps on modern x86 with io_uring. Don't multi-shard for fun.
Latency p99 matters more than aggregate throughput — cross-shard
coordination (relay rings for DcidDispatch) adds nanoseconds. A single
shard has none.
Your workload has shared state — a single shard means everything is in
one address space, no locks needed. Multi-shard pushes you toward per-shard
partitioning.
The single-shard ceiling is the bottleneck of your application code, not the
library. Profile first.
When to cluster
One core sits at 100% in poll() and the kernel says it's not waiting on
I/O. You're CPU-bound, not network-bound. Spread across cores.
You serve > 64k connections and connection state alone fills L1/L2. Per-
shard partitioning means each shard's connection table fits in cache.
You need NUMA-locality for a > 1-socket box. One shard per NUMA node,
each pinned, each with its own pool.
How IoCluster works
let cluster = IoCluster::new(ClusterConfig {
shard_count: 4, // power of 2, see cap table
routing: RoutingStrategy::ReusePortCbpf, // default
..Default::default()
})?;
cluster.quic_listen("0.0.0.0:4433".parse().unwrap(), &cert, &key)?;
let handles: Vec<_> = cluster.into_shards()
.map(|mut shard| std::thread::spawn(move || {
loop {
shard.poll(Duration::from_millis(100)).unwrap();
while let Some(ev) = shard.next_event() {
handle(ev);
}
}
})).collect();
for h in handles { h.join().unwrap(); }
IoCluster::new(config) allocates shard_count shards, each with its own
io_uring, pool, connection table. cluster.quic_listen(...) opens a socket
per shard using SO_REUSEPORT + a kernel-side BPF program that picks which
shard receives each packet (more on routing in
multi-shard routing).
cluster.into_shards() consumes the cluster and gives you an iterator of
ShardIo — same API as Io, just one per thread.
What stays the same
The poll loop body — poll(), next_event(), event handling.
The Event<'poll> rules — borrow ends at next poll()of that shard.
The pool API — each shard has its own pool, brand-isolated from the
others.
What changes
No shared mutable state — if you want shared cache or coordination
between shards, you wire it up explicitly (Arc<RwLock<_>> for cold paths,
per-shard partitioning for hot paths, MPSC channels for cross-shard work).
Connection affinity — once a connection lands on shard 3, it stays on
shard 3. Migration would require dispatcher coordination ; the default
routing strategy doesn't do it.
Application code is the same — handlers don't know they're in a cluster.
The shard's poll() loop drives them ; nothing observable from inside the
handler differs.
Past the cap, you go multi-process + L4 LB upstream (HAProxy, Envoy in TCP/UDP
mode). That's the "we're not trying to be infinite" honest answer.
See also : multi-shard routing for the
three strategies (ReusePortCbpf / DcidDispatch / XdpDcid) in detail,
multi-shard cluster recipe for a working
4-shard QUIC server with graceful drain.
Multi-shard routing
Three strategies to map incoming packets to shards. They differ in mechanism,
cap, and operational complexity. Pick by the decision tree at the bottom.
ReusePortCbpf — the default
NIC → kernel → SO_REUSEPORT group → BPF program → shard_id → shard's socket
Each shard binds the same (ip, port) with SO_REUSEPORT. We attach a BPF
program (sk_reuseport type) to the group ; for every incoming packet, the
kernel calls our program, which extracts the QUIC ServerID from the
Connection ID (or hashes the 4-tuple for non-QUIC protocols), and returns the
shard index. Each shard reads from its own kernel socket via its own io_uring.
Pros : zero userspace dispatch overhead, kernel does the routing in BPF
(20-40 ns per packet), no cross-shard coordination, scales to 16384 shards
(QUIC-LB Plaintext ServerID = 14 bits utiles).
Cons : no central rate limiting (each shard is its own admission point),
no DDoS filter (would need to live in BPF, deferred to a future step), no
load-aware Initial routing (kernel hash is uniform but might create hot spots
on tiny clusters).
This is what you want 95% of the time. Defaults exist for a reason.
DcidDispatch — central dispatcher
NIC → kernel → SO_REUSEPORT(1 fd) → 1 dispatcher thread → SPSC ring × N → N shards
│
zero-copy pool slot
handle transfer
One dispatcher thread reads all packets from a single shared socket. It walks
the QUIC DCID, looks up the shard, transfers ownership of the pool slot via
an SPSC ring to that shard. Each shard reads from its own ring, no kernel
involvement after the dispatcher.
Pros : central admission point (rate limit, DDoS filter, accept/reject
policy live here), load-aware routing (dispatcher can pick least-loaded shard
for QUIC Initials), single chokepoint for ops.
Cons : dispatcher is one CPU core, capacity ~25 Mpps before it's the
bottleneck. SPSC rings add ~50 ns per packet over ReusePortCbpf. Operational
complexity (dispatcher health, ring saturation metrics, drain protocol on
shutdown).
Use this when : you need centralized admission, or your fleet operations team
wants one place to plug rate limits / metrics / DDoS without per-shard config
management.
XdpDcid — AF_XDP kernel bypass
NIC ──XDP-attached BPF program (in driver) ──→ AF_XDP socket per shard
│
xsk_map[shard_id] = socket fd
XDP runs in the NIC driver, before the kernel network stack. Our BPF program
extracts the DCID, looks up the destination shard in xsk_map, redirects the
packet straight to that shard's AF_XDP socket. Userspace reads packets from
the AF_XDP UMEM rings.
Pros : highest pps achievable on 100GbE NICs (target range tens of
Mpps, to be measured by the bench gate), no skb allocation cost, no
kernel network stack overhead.
Cons : capped at 1024 shards (xsk_map BPF verifier limit). Kernel ≥ 5.4
required, but we cap at the same 6.7+ floor for consistency. Requires
CAP_NET_ADMIN to load the XDP program. NIC driver must support XDP native
mode (most do, but check). Bypasses kernel features (no iptables, no ss -tnp, no traditional tcpdump — needs xdpdump). Operationally heavier.
Use this when : you're at the very edge of what the kernel network stack can
do. If your top -H shows kernel softirq saturating cores doing skb allocs,
this is the answer.
Decision tree
Need DDoS / rate limit / admission policy at the cluster ingress ?
YES → DcidDispatch
NO → Are you at > 30 Mpps aggregate ?
YES → XdpDcid (AF_XDP)
NO → ReusePortCbpf (default, simplest)
QUIC Connection ID format (R3.7 minimal)
For both ReusePortCbpf and DcidDispatch, QUIC packets carry a 16-byte
Connection ID we generate on the server side :
byte 0 : reserved for a future Config Rotation field if needed (deferred
per why QUIC-LB minimal).
bytes 1-2 : ServerID big-endian, 14 bits utiles (2 bits reserved high).
This is the shard index. Cap = 2^14 = 16384 shards.
bytes 3-15 : 13 random bytes from getrandom. Per RFC 9000 these
guarantee CID uniqueness ; their randomness also doubles as anti-forgery
(an attacker has to guess 104 bits to land on an existing CID).
The format is QUIC-LB draft-20 Plaintext compliant. External LBs (F5,
HAProxy 2.8+, nginx-quic) extract ServerID directly from the wire, no shared
state with the server needed.
Connection migration
QUIC clients can migrate paths (Wi-Fi → cellular, NAT rebinding). The DCID
stays the same across migration, so our shard mapping stays the same — the
new path's first packet still routes to the same shard. No special handling.
What's deferred
Per STEP173 R3.7 : Config Rotation (hot scaling
without dropped connections), SipHash integrity tag (cryptographic anti-
forgery instead of statistical), DDoS filter BPF (per-IP rate limit at the
sk_reuseport level). All have explicit byte 0 extension hooks ; v2
backward-compatible. Implemented when real demand emerges, not before.
zero-io itself is sync. zero-io-async is the bridge for when your handler
needs to talk to async code — a tokio-based DB driver, an async fn middleware
chain, an axum route. You don't have to choose ; you compose.
The two halves
Shard side — single-threaded Io::poll() loop, owns sockets and pool,
zero-alloc / zero-lock / zero-copy. This is where I/O happens. Packets in,
packets out.
Async side — your familiar tokio runtime, multi-threaded scheduler,
async fn, futures, .await. This is where business logic lives if it's
async-shaped.
The bridge between them is two primitives :
IoHandle — Send + Sync + Clone, cheap to clone, holds a queue handle
back into the shard. Used to send TX work back from async tasks.
OwnedSlot — Send + Sync, a refcounted handle to a pool slot,
survives across poll() cycles. Used to ship RX payloads into async tasks
without copying.
Four ways to combine the shard with tokio
Pick the topology that matches your shape. None of these require a
special wrapper — you're choosing where threads live and which runtime
owns each.
One shard thread + one tokio current-thread per shard
Multi-shard cluster with per-shard async workers, no cross-shard contention
Shared thread
One shard thread + tokio runtime with the shard thread enrolled
Advanced : tokio can spawn to the shard's thread between poll() ticks
The first three are the common ones. Shared thread is for libraries
that want tokio::spawn to work inside handlers without an explicit
channel ; rarely needed.
Shape : shard + pooled tokio (most common)
let mut io = Io::new(Config::default())?;
io.tcp_listen("0.0.0.0:8080".parse().unwrap())?;
let handle = io.handle(); // IoHandle, Send + Sync
let async_rt = tokio::runtime::Runtime::new()?;
// async worker pool : reads OwnedSlot from a channel, processes, sends back response
let (tx, mut rx) = tokio::sync::mpsc::channel::<(OwnedSlot, ConnId)>(1024);
async_rt.spawn(async move {
while let Some((slot, conn)) = rx.recv().await {
let response = process_async(slot.as_slice()).await; // your async logic
let mut buf = handle.send_buffer(response.len()).await?; // back to shard pool
buf.write(&response);
handle.stream_write(conn, buf).await?; // shard handles SQE
}
Ok::<_, std::io::Error>(())
});
// shard loop : detach RX, ship to async, never block in handler
loop {
io.poll(Duration::from_millis(10))?;
while let Some(ev) = io.next_event() {
if let Event::StreamFrame { conn, .. } = ev {
if let Some(slot) = io.detach_event_data() {
if let Err(_full) = tx.try_send((slot, conn)) {
// Worker queue is full — async side can't keep up.
// Drop, NACK, or close depending on protocol policy.
metrics::async_queue_full.inc();
}
}
}
}
}
Two threads, two halves of the program, one shared pool. The shard never
blocks on async work ; the async worker never touches sockets directly.
Backpressure is explicit : when the async worker can't keep up, the
shard's try_send returns the OwnedSlot back to the caller via the
error variant. Drop it (slot returns to the pool), close the connection,
or send a NACK — the policy is yours, the runtime doesn't decide.
Where the SPSC / MPSC rings show up
IoHandle is the user-visible wrapper ; underneath, each shard owns
one MPSC command ring shared across all IoHandle clones (an
Arc<MpscProducer<NetworkCmd>> per plan 173) — N async tasks
contending via CAS-reservation, the shard alone draining at the start
of each poll(). The DcidDispatch multi-shard mode adds SPSC relay
rings (one dispatcher thread → one shard each). Both primitives are
re-exported as pub in charting-transport::base for layer-3 code ;
you usually don't reach for them directly. See cross-thread
channels for the full pattern map and
the wait-free vs lock-free distinction between SPSC and MPSC.
tower::Service integration
zero-io-async impl's tower::Service<HttpOwnedRequest> for the HTTP shard,
which means : any tower middleware ecosystem (tower-http, custom middlewares
for auth/logging/tracing/cors) plugs in directly.
use tower::{ServiceBuilder, ServiceExt};
let svc = ServiceBuilder::new()
.layer(tower_http::trace::TraceLayer::new_for_http())
.layer(my_auth_middleware())
.service(my_handler);
The cost of going through tower is on the order of ~200 ns per request
(a Box<dyn Future> poll, basically — exact figure depends on hardware,
to be measured by the bench gate). For most apps that's fine. For the
genuinely performance-critical path, write a native zero-io middleware
(no Box, no trait object) — the API reference lists the pattern.
zero-io-axum — drop-in axum compat
The zero-io-axum feature ships an axum::Server adapter that runs axum
applications on top of zero-io's HTTP shard. Compatible with the entire axum
ecosystem.
The cost : axum's HeaderMap allocates ~640 B per request for the BTreeMap
that holds parsed headers. zero-io natively avoids that with HttpOwnedRequest
(96-112 B, slice-into-pool-slot). If you're going axum, you accept the
HeaderMap tax in exchange for the ecosystem.
A migration path exists for "I want axum routing but native pool slots
internally" — the tower-http-compat feature converts the request shape at
the boundary, you write axum-style handlers but you get HttpOwnedRequest in
the inner layer. Detailed in the API reference.
See also : cross-thread channels for
the IoHandle / OwnedSlot deep dive that this page sits on top of ;
event lifetimes for how detach_event_data()
moves data across the 'poll boundary ; migration from tokio
for porting an existing tokio app.
Cross-thread channels
Your async task wants to talk to the shard. Or another shard. Or a worker
thread. How? zero-io has SPSC and MPSC ring primitives at the bottom,
but they're not what you reach for directly — you reach for the wrappers
above them. This section is the explicit map.
The primitives (base/ring.rs)
Two ring representations, both lock-free :
Primitive
Producers
Consumers
Mechanism (per base/ring.rs)
SpscRingRepr<T, N>
1
1
Lamport pattern, two AtomicU32 heads (one per side, each on its own cache line via Padded64), no CAS, wait-free
MpscRingRepr<T, N>
many
1
CAS-reservation on write_head + 2-phase publish gate. Lock-free but not wait-free (sequential publish order — preempted producer can stall the gate). Typical 10-50 ns / op without preemption.
N is a const generic, must be a power of 2. T is Copy + 'static,
typically a small struct (a slot ID, a command tag, a 16-byte token).
Bytes-payload travels via OwnedSlot, not through these rings.
Both are re-exported as pub in charting-transport::base for
layer-3 users (custom protocol handlers, custom dispatchers). You
almost never reach for them directly. Instead, you use one of the
three wrappers below.
Pattern 1 — IoHandle for I/O work
The shard exposes a Send + Sync + Clone handle :
let handle: IoHandle = io.handle(); // cheap clone, ~16 B struct
Async tasks call methods on it :
// from any tokio task
let mut buf = handle.send_buffer(payload.len()).await?;
buf.write(&payload);
handle.stream_write(conn, stream, buf).await?;
// or one-shot
handle.close_graceful(conn, Duration::from_secs(3)).await?;
Under the hood (per plan 173 / step 7.1b) : IoHandle is Clone + Send + Sync and wraps an MPSC command ring (MpscRingRepr with
CAS-based reservation, formalized in §172a.3). The async method
serializes a small command onto the ring, then bumps a
ShardWakeupHandle to wake the shard if it's sleeping in poll().
The shard drains the ring at the start of each poll() cycle and
executes commands inline.
The wakeup mechanism :
Linux 6.7+ : IORING_OP_FUTEX_WAIT — the shard sleeps inside
io_uring_enter, kernel wakes it on futex bump, no syscall on the
wakee side.
Windows 10 1809+ : Named Events via WaitOnAddress.
Cost : on the order of tokio::sync::oneshot (low hundreds of ns
for the round trip — bench-gate measures the exact figure). The
ring-overflow case returns Err(IoError::CommandRingFull) rather than
silently dropping (Amendment DX 2026-04-15) ; the caller decides
whether to retry, slow down, or shed.
Use when : your async code wants to queue I/O — send_datagram,
stream_write, close, close_graceful, plus the zero-copy _buffer
variants. Methods that return data (conn_stats, pool_stats, etc.)
are NOT on IoHandle — they require &self on Io directly, same
thread as the shard.
Pattern 2 — OwnedSlot for zero-copy data transfer
Detach RX data from the shard's Event<'poll> and ship it to async :
io.poll(timeout)?;
while let Some(ev) = io.next_event() {
if let Event::StreamFrame { conn, .. } = ev {
if let Some(slot) = io.detach_event_data() {
// OwnedSlot is Send + Sync, refcount-bumped into the pool
tx.send((conn, slot)).await?;
}
}
}
OwnedSlot carries the bytes WITHOUT copying — it holds a
refcount-bumped reference to the same pool slot the kernel filled. The
slot lives until the last OwnedSlot clone is dropped.
The cross-thread channel itself (tx/rx above) is just a
tokio::sync::mpsc (or crossbeam-channel, or whatever shape fits).
zero-io doesn't ship its own application-level channel — the
ecosystem already has good ones, and the bytes-payload concern is
handled by OwnedSlot.
Use when : RX data leaves the shard for async processing.
Pattern 3 — Application messages back to the shard
Your app has a non-I/O message ("the DB returned a row, please send
this to connection 7"). The work is :
Async task does the DB query, gets a result.
Wants the shard to send the result to a specific connection.
This decomposes entirely into Pattern 1. There is no separate "send a
custom message to the shard" surface ; the shard's input from async is
exactly the set of IoHandle methods.
// in your async task
let mut buf = handle.send_buffer(row.encoded_len()).await?;
buf.write(&row.encode());
handle.stream_write(conn, stream, buf).await?;
The MPSC enqueue + ShardWakeupHandle bump cover everything. Even a
sleeping poll(Duration::MAX) wakes immediately on the futex / ulock /
WaitOnAddress signal.
If you need shard-side logic on a custom message type that doesn't
map to IoHandle (a state-machine event that's neither I/O nor a
tokio task), write a layer-2 ProtocolHandler that the shard owns,
fed by a custom dispatcher you integrate into the shard's poll()
cycle. That's layer-3 territory and not the recommended path — most
apps never need it. The shard does not generically poll arbitrary
user-provided message queues ; the channel discipline is IoHandle
(async → shard) plus OwnedSlot over your channel of choice (shard →
async).
What zero-io intentionally does NOT ship
A typed application-level channel like tokio::sync::mpsc<T>.
The Rust ecosystem has those ; reinventing wouldn't add value. We
give you the bytes-zero-copy primitive (OwnedSlot) which is the
thing async channels can't provide.
Cross-machine messaging primitives. That's transport
(TCP/QUIC/WebSocket) — at that point you're back in the I/O surface,
not in cross-thread channels.
Summary
Goal
Tool
Async → shard, queue I/O
IoHandle methods (Pattern 1)
Shard → async, ship RX bytes zero-copy
OwnedSlot over tokio::sync::mpsc (Pattern 2)
Async ↔ async, application messages
tokio::sync::mpsc<T> (we don't reinvent)
Custom shard-side logic on app messages
layer-2 ProtocolHandler + custom dispatcher (Pattern 3, advanced — most apps don't need it)
Internal cross-shard relay (DcidDispatch)
SpscRingRepr directly — but this is library-internal
The SPSC/MPSC primitives exist and are pub. You usually don't see
them because IoHandle already wraps them ; reach for the raw rings
only when you're writing layer-3 code that lives inside the shard's
event loop.
The default backend on Linux. Every shard runs an io_uring instance ; every
SQE / CQE you read about elsewhere in the docs flows through it. AF_XDP
shards still run io_uring alongside AF_XDP — see
AF_XDP runtime. This page covers the io_uring
features the runtime uses, the kernel-version matrix, and the
configuration knobs that matter.
Why io_uring (over epoll)
epoll is reactor-shaped : you ask the kernel "tell me when this fd is
ready", the kernel says yes, then you call recvmsg / sendmsg —
N readiness syscalls plus N I/O syscalls per N operations. io_uring is
proactor-shaped : you submit the operation itself (recvmsg, sendmsg,
splice, futex_wait, …) into a shared ring, the kernel completes it,
you read the completion. Zero readiness syscalls, batched submission,
optional zero syscalls at all under SQPOLL.
Concrete numbers from the runtime's measurements (target — exact figures
publish with the bench-CI gate) :
Op count
epoll round-trip
io_uring round-trip
io_uring + SQPOLL
1 packet
~2 syscalls
1 syscall
0 syscalls
1k packets
~2k syscalls
1 syscall (batch)
0 syscalls
The runtime needs the proactor model for zero-alloc + zero-copy : the SQE
points at a pre-registered buffer, the kernel writes into it directly,
the CQE returns the buf id. With epoll, you'd have to copy from a
kernel-internal buffer to your pool slot manually.
Features the runtime uses
Every feature has a kernel-version floor and a runtime probe ; the
runtime degrades gracefully or refuses to boot depending on the
feature's criticality.
The Tier 1 floor is Linux 6.7+ because that's where IORING_OP_FUTEX_WAIT
landed. Without it, cross-thread shard wakeup falls back to eventfd
poll, which adds ~150 ns and a syscall per wakeup — defeats the
zero-syscall claim.
SQPOLL : zero-syscall mode
let cfg = Config::default()
.with_uring(|u| u.sqpoll(true).sqpoll_idle_ms(50));
SQPOLL spawns a kernel thread that polls the SQ ring. Submitting an
SQE becomes "write to userspace memory" with no io_uring_enter
syscall. The kernel thread sleeps after sqpoll_idle_ms of no work and
wakes on the first SQE after idle.
When to enable :
TX-heavy workloads at > 100 kpps. Zero submit syscalls is a
measurable win.
Multi-shard clusters with NUMA-local SQPOLL pinning.
When to leave off :
Embedded boards with tight CPU budget. The SQPOLL thread is
always-on overhead.
Single-shard low-pps services. The savings don't amortise.
Registered buffers + BUF_RING
The runtime registers the pool's buffer arena as a single contiguous
mapping at boot via IORING_REGISTER_BUFFERS. Every SQE references
slot indices into that mapping — kernel skips the page-table lookup
that an unregistered recvmsg would do.
BUF_RING extends this for RX : the runtime pre-fills a "buffer ring"
with available slot ids. A multishot IORING_OP_RECVMSG_MULTISHOT SQE
tells the kernel "drain into the buffer ring as packets arrive" ; the
kernel picks free slots itself, you get one CQE per packet with the
chosen buf_id. Zero-copy RX, no per-packet SQE.
NAPI busy-poll
let cfg = Config::default()
.with_uring(|u| u.napi_busy_poll(Duration::from_micros(50)));
IORING_REGISTER_NAPI (6.7+) tells the kernel to busy-poll the NIC's
NAPI queue inside io_uring_enter for up to N µs before sleeping. The
floor for RX latency drops from "kernel softirq schedule" (low µs) to
"NIC DMA complete" (sub-µs). Turn on for HFT-style latency budgets.
CQE_MIXED + ZCRX (kernel 6.18+)
IORING_SETUP_CQE_MIXED allows a single CQ to carry both 16-byte and
32-byte CQEs. The 32-byte form carries extra metadata for ZCRX — true
zero-copy RX where the NIC DMAs straight into your pool with header
splitting. Currently gated behind feature flag cqe-mixed.
Configuration knobs
let cfg = Config::default()
.with_uring(|u| u
.sq_size(1024)
.cq_size(4096)
.sqpoll(true)
.sqpoll_idle_ms(50)
.sqpoll_cpu(Some(3))
.iowq_cpu_mask(0xff_00)
.napi_busy_poll(Duration::from_micros(50)));
Defaults are sane for ~1k connections on modern hardware.
Capability detection at boot
$ ./your-binary
zero-io: io_uring kernel 6.13.5
zero-io: IORING_OP_FUTEX_WAIT ok
zero-io: IORING_REGISTER_BUFFERS ok
zero-io: BUF_RING ok
zero-io: IORING_OP_SEND_ZC ok
zero-io: IORING_REGISTER_NAPI ok (busy_poll = 50 us)
zero-io: IORING_REGISTER_RESIZE_RINGS ok
zero-io: IORING_SETUP_CQE_MIXED skip (6.18+ required)
zero-io: started shard 0 on core 0 (sqpoll on core 1)
The runtime probes each feature once at boot. The result is cached in
Capabilities ; the hot path queries caps.has(BUF_RING) as a bool
read, no syscall. If a required feature is missing, boot fails with
IoError::KernelTooOld { required, found }.
kTLS is a kernel feature configured via setsockopt ; io_uring
reads / writes the same socket transparently.
See also : the poll cycle for what each
io_uring phase does inside poll() ; kernel requirements
for the per-feature kernel-version table ; why Linux first
for the rationale behind the 6.7+ floor ; AF_XDP runtime
for the higher-pps path on the same shard.
AF_XDP runtime
🚧 Design preview — gated on step 174b (in active
review). The shape below reflects the current plan ; specifics may shift
with the implementation.
AF_XDP is a kernel mechanism that bypasses the network stack entirely. The
NIC writes packets directly into userspace-shared memory (UMEM rings),
filtered by an XDP-attached BPF program in the driver. No skb allocation,
no kernel network stack overhead. On Linux, it's the highest-pps path
available without bypassing the kernel altogether (which is what the
userspace TCP stack does — see userspace TCP stack).
When to use it
linux-af-xdp is opt-in via Cargo feature. Switch to it when :
Sub-microsecond latency floor matters. NAPI busy-poll on AF_XDP gets
you tighter floors than even io_uring with multishot recv.
You're CPU-bound in softirq per top -H. That's the signal.
If your bottleneck is userspace logic (parsing, business code, syscalls in
your handler), AF_XDP doesn't help — switch the protocol or batch better.
The shape of the runtime
NIC driver (XDP attached)
│
BPF: extract DCID, lookup xsk_map[shard_id]
│
┌──────────────┼──────────────┐
│ │ │
AF_XDP socket AF_XDP socket AF_XDP socket
shard 0 shard 1 shard 2 ...
│ │ │
UMEM RX ring UMEM RX ring UMEM RX ring
│ │ │
shard.poll() ←─ same Io API as io_uring
The shard's poll() reads completions from the AF_XDP UMEM rings instead of
io_uring CQEs. The rest of the API (Event<'poll>, SendBuffer, handlers)
is identical. You don't write AF_XDP-specific code in your handler.
Cap : 1024 shards
The xsk_map BPF verifier limits the map size, capping the cluster at 1024
shards in XdpDcid mode. For most workloads that's far more than enough
(1024 cores × ~50 kpps each = 50 Mpps before saturation) ; if you genuinely
need more, you cluster machines and L4-LB upstream.
Setup requirements
NIC driver must support XDP native mode (not the generic fallback,
which loses most of the perf benefit). Most modern Intel, Mellanox /
NVIDIA, Broadcom drivers do. Run ethtool -i <iface> to find the driver,
then check the kernel docs for XDP support.
Privileges : CAP_NET_ADMIN to load the XDP program. Either run with
sudo once at startup to load and pin the program, or setcap cap_net_admin+ep ./your-binary.
Kernel : 5.4+ for AF_XDP, but we cap at the same 6.7+ floor as the
rest of the library for consistency.
Hugepages (recommended) : the UMEM benefits from huge pages for the
TLB pressure ; we'll set them up automatically if vm.nr_hugepages is
configured.
What you give up
XDP runs before the kernel network stack. Tools that hook into the stack
don't see AF_XDP traffic :
iptables / nftables — no firewall rules apply. You enforce in your
BPF program if needed.
tcpdump — won't see the traffic. Use xdpdump instead.
ss -tnp — won't list AF_XDP sockets. Custom telemetry from the runtime.
/proc/net/* counters — bypass them. Your application's metrics are
the source of truth.
This is why XdpDcid is opt-in. The default ReusePortCbpf keeps you
inside the kernel stack where standard ops tooling works.
Boot logs the BPF program load, the xsk_map setup, and the per-shard UMEM
mmap. If anything fails (missing capability, driver doesn't support XDP
native, UMEM mmap fails), IoError::AfXdpSetup { reason } with an actionable
message.
Co-existence with io_uring (unified poll)
Every AF_XDP shard also has an io_uring. They run in the same
poll() loop, not in alternation. AF_XDP carries the bulk packet
data path (20-50 Mpps target) ; io_uring carries the control plane —
netlink (interface state, ARP/ND), timers, eventfds, signals, and any
non-AF_XDP file descriptors the shard needs to wait on.
How they're stitched together :
The AF_XDP socket fd is registered into io_uring via
IORING_OP_POLL_ADD. The shard's io_uring_enter returns a CQE
whenever the AF_XDP RX ring has packets to drain.
The shard's poll() cycle, on return, drains the AF_XDP UMEM RX
ring directly (not via CQE — UMEM rings are read in userspace),
routes packets to handlers, then loops back into io_uring waiting.
Netlink subscriptions live on shard 0 via
IORING_OP_RECVMSG_MULTI, parsed once, dispatched to other shards
via an MPSC queue (~20 ns hop). One netlink socket for the whole
cluster, zero extra threads.
The "io_uring coexistence via poll unifie" line in plan 174b is the
canonical statement : the shard always runs io_uring, even when its
data path is AF_XDP. They're complementary, not alternatives.
What's truly exclusive
The packet data path is exclusive per cluster :
RoutingStrategy::ReusePortCbpf or DcidDispatch : kernel network
stack via io_uring (UDP/QUIC/TCP all flow through the same path).
RoutingStrategy::XdpDcid : AF_XDP UMEM. Bypasses kernel stack
entirely for packets matching the XDP program ; non-matching packets
fall back to the kernel and io_uring picks them up if you're also
listening there.
You can run TWO IoCluster instances on the same machine — one
XdpDcid for the hot UDP/QUIC, one ReusePortCbpf for control-plane
TCP — they share the host but are otherwise independent. You can also
have AF_XDP listen on a specific NIC + queues while leaving the rest
of the box's networking on the kernel stack via io_uring.
What we don't ship is a runtime fallback where a single packet
endpoint dynamically chooses AF_XDP or io_uring per packet — that
would mean the BPF program is sometimes loaded, sometimes not,
introducing nondeterminism in the data path. Choose one per
IoCluster ; coexist them at the host level if you need both.
Status
The XDP BPF program (xdp_steer.bpf.c planned, currently unifies with
qlb_route.bpf.c in plan 173 R3.7) extracts the DCID and indexes
xsk_map[shard_id]. Userspace parity is asserted via the
bpf_xdp_routing_parity test : 10k packets injected into both XDP and
sk_reuseport flows, asserts every packet routes to the same shard via both
paths.
Plan 174b is in iterative review (R184+). When it merges, this section
upgrades from "design preview" to "current behavior".
See also : io_uring for the default Linux backend
that AF_XDP coexists with on every shard ; Userspace TCP stack
for the userspace TCP path that rides on AF_XDP ; Multi-shard routing
for the XdpDcid strategy that uses AF_XDP for packet steering ;
kernel requirements for AF_XDP and XDP feature
floors per kernel version.
Userspace TCP stack
🚧 Design preview — gated on step 174c (design only).
The shape below reflects the current plan ; specifics may shift with the
implementation.
A userspace TCP/IP stack — the FreeBSD network code ported to userspace
— running on top of AF_XDP. zero-io integrates it to give you a TCP
stack that bypasses the Linux kernel entirely. It's the last 10× of perf
you can squeeze out of a single box.
When to use it
linux-userspace-tcp is opt-in via Cargo feature, and experimental.
Switch to it when :
TCP throughput exceeds 40-50 Gbps on a 100GbE NIC. The kernel TCP
stack starts being the bottleneck at those rates ; userspace TCP can push
past it.
Connection establishment rate is the bottleneck (e.g., HTTP server
saturating with accept(2) syscalls). The userspace accept is a ring
read, no syscall.
You're already running an AF_XDP cluster and want TCP semantics
alongside the QUIC traffic, on the same NIC, without re-entering the
kernel.
If your TCP load fits comfortably in kernel stack (anything below 10 Gbps
on modern hardware), use linux-af-xdp for UDP/QUIC and the standard
io_uring TCP backend for TCP. Don't pay the userspace-TCP tax.
Each shard owns one userspace TCP instance with its own TCP control blocks,
its own retransmission queue, its own listening sockets. No locking between
shards. The shard's poll() drives the userspace timer wheel and TCP state
machines, then drains incoming connections / data into the familiar
Event::* variants.
What stays the same
The public API. tcp_listen, tcp_connect, Event::Connected,
Event::StreamFrame, stream_write, stream_shutdown — identical to the
io_uring TCP backend. Application code switches via Cargo feature ; no
code changes.
What changes
Connection table is per-shard — same as XdpDcid. A connection landed
on shard 3 stays on shard 3 ; no migration without explicit dispatcher
coordination (we don't ship that).
TCP options behavior — the userspace stack inherits FreeBSD's TCP
defaults, which differ subtly from Linux's (TIME_WAIT handling, ACK
delay, congestion control variants). We ship a small adapter that maps
Config knobs to the userspace stack's equivalents, but parity is not
byte-for-byte.
No iptables / tc / kernel firewalls — same as AF_XDP. You're
outside the kernel network stack. Enforce in BPF or in userspace.
Memory — the userspace stack pre-allocates buffer pools (mbuf-style,
inherited from FreeBSD's design). Default budget is ~512 MiB per shard.
Configurable via the TOML.
Boot time — userspace-TCP init is heavier than io_uring. Expect a
few hundred ms added to startup depending on UMEM size (exact figure
measured during impl).
Setup requirements
NIC driver : same as AF_XDP (XDP native mode).
Privileges : CAP_NET_ADMIN, CAP_NET_RAW, CAP_SYS_ADMIN for
hugepages.
Hugepages : the userspace stack benefits significantly from 2 MiB or
1 GiB pages. Allocate via /proc/sys/vm/nr_hugepages before starting.
Linux 5.4+ for AF_XDP ; we still cap at 6.7+ for the rest.
Cap : 1024 shards (same as XdpDcid)
xsk_map BPF verifier limit applies. The userspace stack adds no further
cap.
Userspace and kernel TCP coexist on the same shard
Opt-in is per port, not per shard. A single shard runs both stacks at
once :
Every other TCP port → XDP_PASS → kernel → io_uring TCP (the default
backend).
The XDP BPF program inspects the destination port on RX and steers to the
matching XSK socket, or hands the packet to the kernel via XDP_PASS. So one
shard can serve an HFT FIX engine on port 9080 through the userspace stack
while serving HTTPS on port 443 through kernel TCP — same NIC queue, same
poll loop, same tcp_listen call. No second cluster, no per-shard
backend choice.
What this doesn't allow : a single port served by both stacks at once.
If the userspace stack claims port 9080, kernel sockets bound to 9080
receive nothing — the XDP program steers the traffic to AF_XDP before
the kernel ever sees it. A port goes to one stack or the other.
What we don't support
Live migration — shutting down a shard means dropping all its
connections. There's no graceful migration to another shard.
close_graceful(timeout) works per-connection within a shard ; whole-
shard graceful drain requires application-level coordination.
Cross-shard TCP state sharing — each shard owns an independent
userspace-stack instance with its own connection table. Connections
don't move between shards.
tcp_listen is the same call for both kinds of port. The XDP BPF program
chooses the path based on enabled_ports. Loading from TOML uses the
[userspace_tcp] enabled_ports = [...] section ; the field above is
the programmatic equivalent.
Why "experimental"
The FreeBSD network code is mature (decades of production use), but the
integration with zero-io's pool / handler / typestate model is new.
Until we've burned-in : tests stress-test the integration, real-world
workloads validate the perf claims, and we've shipped to a production
customer, the feature flag stays experimental and the API can break
between versions.
If you're considering it for production : talk to us first. The
performance ceiling is genuinely higher ; the operational story is
heavier.
Status
Plan 174c is design-only. Implementation comes after 174b (AF_XDP
foundation) is shipped, since the userspace stack rides on top of AF_XDP
infrastructure.
See also : AF_XDP runtime for the foundation
this rides on ; io_uring for the kernel-TCP path that
serves non-opt-in TCP ports on the same shard ; TLS hot-reload
for cert rotation on userspace-stack-fronted services ;
why QUIC-LB minimal for the cluster-routing
model the userspace stack inherits.
kqueue (macOS Tier 2)
The macOS backend. macOS doesn't have io_uring ; the closest equivalent
is kqueue + kevent, the BSD-family event notification mechanism. The
runtime ships a kqueue backend at Tier 2 — usable for development,
testing, and small production workloads, with an explicit perf ceiling
below the io_uring path on Linux.
When you'll use it
macOS development — you're on a Mac, you want the runtime to work.
This is the path. Same Io API, same Event shape, same handler
code as on Linux.
macOS deployment — small services on macOS hardware (build
agents, CI runners, on-Mac internal tooling).
For production workloads above ~100 kpps, deploy on Linux. The macOS
ceiling isn't a deficiency in the port ; it's kqueue itself — no
registered buffers, no zero-copy submit-and-forget, no SQPOLL.
What we use
Mechanism
Role
kqueue()
Create the event queue (one per shard)
kevent()
Submit changes + wait for events in one syscall
EVFILT_READ / EVFILT_WRITE
Readiness notifications per fd
EVFILT_TIMER
Timer wheel ticks
EVFILT_USER
Cross-thread shard wakeup (NOTE_TRIGGER)
EVFILT_VNODE
File watching (TLS hot-reload, config)
Same six poll-cycle phases as the io_uring path, different mechanism.
Handler code doesn't see the difference.
Cross-thread shard wakeup
io_uring uses IORING_OP_FUTEX_WAIT. macOS doesn't have futex ; the
runtime uses Apple's __ulock_wait2 / __ulock_wake (private libsystem
syscalls). IoHandle enqueues a command on the MPSC ring, bumps the
ulock, the shard's kevent returns immediately via EVFILT_USERNOTE_TRIGGER paired with the ulock signal.
Latency budget : low microseconds end-to-end.
Thread priority — Apple QoS
let cfg = Config::default()
.with_kqueue(|k| k.qos_class(QosClass::UserInteractive));
macOS schedules threads via QoS classes. The shard pins itself to
USER_INTERACTIVE by default — top scheduling priority, won't be
preempted by background work.
Tier 2 ceiling — what's missing vs io_uring
Feature
io_uring (Linux)
kqueue (macOS)
Effect
Registered buffers
yes
no
Each recvmsg does the page-table lookup
BUF_RING multishot RX
yes
no
One syscall per packet
Zero-copy TX (SEND_ZC)
yes
no
One TX-side memcpy unavoidable
SQPOLL
yes
no
Submit syscalls don't disappear
NAPI busy-poll
yes (6.7+)
no
RX latency floor is kernel-schedule
IORING_OP_SPLICE
yes
partial (sendfile)
TCP proxy uses sendfile fallback
IORING_OP_FUTEX_WAIT
yes
no (use __ulock)
Functional parity
In practice the macOS path runs ~30-50% the throughput of the Linux
path on otherwise-identical hardware.
What the macOS path doesn't have at all
AF_XDP — Linux-specific kernel feature.
kTLS — Linux-specific.
Userspace TCP stack (174c) — depends on AF_XDP.
If you'd be paying these features on Linux, you need a Linux box. No
path to them on macOS short of a Linux VM.
Build + capability probe
$ cargo build --target aarch64-apple-darwin --release
$ ./your-binary
zero-io: kqueue darwin 23.5.0 (macOS 14.5)
zero-io: EVFILT_USER ok
zero-io: EVFILT_TIMER ok
zero-io: __ulock_wait2 ok (private libsystem)
zero-io: QoS USER_INTERACTIVE ok
zero-io: started shard 0
The macOS Sonoma 14+ floor is for stability of the private __ulock
ABI ; older macOS versions return ENOSYS and the runtime refuses to
boot. Tested on Sonoma (14) and Sequoia (15).
Operational notes
Profiling : Instruments (Time Profiler, System Trace) sees the
shard's syscalls and CPU time. dtrace works at the syscall::*
probe level.
See also : io_uring for what the Linux Tier 1
path provides ; Tier 1 / 2 / 3 for the broader
support-tier policy ; why Linux first for
why io_uring's feature set drives the runtime's design.
RIO + IOCP (Windows Tier 3)
The Windows backend. Windows doesn't have io_uring or kqueue ; the
closest equivalents are RIO (Registered I/O, the Windows analogue
of io_uring's registered buffers) for the data path and IOCP (I/O
Completion Ports) for the wait-for-completion mechanism. The runtime
combines both at Tier 3 — best-effort port for Windows
deployment.
Tier 3 — what that means
The runtime builds and runs on Windows.
The full Io API (UDP, TCP, QUIC, HTTP, WebSocket) is functional.
Bench-gate is informational only ; we don't enforce perf parity
with Linux.
CI runs the integration suite on Windows ; we fix correctness
bugs but don't backport perf optimisations as aggressively.
Some advanced features (AF_XDP, userspace TCP, kTLS, NAPI busy-poll,
splice) have no Windows equivalent — IoError::NotSupportedOnPlatform
with a clear message.
If your service is Windows-shaped (Windows-domain authentication,
COM/WMI integration, Windows-only platform), Tier 3 is the path. If
you're choosing Windows for performance, you're choosing the wrong OS
for this runtime.
What we use
RIO (Winsock2 Registered I/O) :
Mechanism
Role
RIORegisterBuffer
Pin pool slots into a single registered buffer
RIOCreateRequestQueue
Per-socket submission/completion queue
RIOReceive / RIOReceiveEx
Submit RX requests, kernel writes into pool slots
RIOSend / RIOSendEx
Submit TX requests, kernel reads from pool slots
RIONotify
Signal IOCP when completions arrive
RIODequeueCompletion
Drain completions in batches
IOCP (I/O Completion Ports) :
Mechanism
Role
CreateIoCompletionPort
Bind RIO completions to a port
GetQueuedCompletionStatusEx
Drain completions, block up to timeout
PostQueuedCompletionStatus
Cross-thread shard wakeup
Six poll-cycle phases, same shape as io_uring. Handlers don't see the
wire mechanism.
CPU pinning
Concept
Linux
Windows
Pin to one core
pthread_setaffinity_np(CPU_SET)
SetThreadAffinityMask
Pin to a CPU group
cpuset
SetThreadGroupAffinity
Pin to a CPU set (post 1809)
pthread_setaffinity_np
SetThreadSelectedCpuSetMasks
Realtime priority
SCHED_FIFO
SetThreadPriority(THREAD_PRIORITY_TIME_CRITICAL)
The runtime uses SetThreadSelectedCpuSetMasks (Windows 10 1809+) so
you can pin a shard to a NUMA node + core set, same as Linux's cpuset
model.
Cross-thread shard wakeup
IoHandle::send enqueues a command on the MPSC ring, then
PostQueuedCompletionStatus(iocp, ...) posts a synthetic completion
that the shard's GetQueuedCompletionStatusEx returns immediately.
Functional parity with Linux futex / macOS ulock — different primitive,
same outcome.
Tier 3 ceiling — what's missing vs io_uring
Feature
io_uring (Linux)
RIO + IOCP (Windows)
Effect
Registered buffers
yes
yes (RIO)
Parity
Multishot RX
yes (BUF_RING)
no
One submission per RX
Zero-copy TX
yes (SEND_ZC)
partial
One TX-side memcpy at the kernel boundary
SQPOLL (zero-syscall submit)
yes
no
Submit syscalls remain
NAPI busy-poll
yes (6.7+)
no
RX latency floor is kernel-scheduler
Splice (kernel fd→fd)
yes
partial (TransmitFile)
TCP proxy via splice unavailable
AF_XDP
Linux-specific
no
No userspace-bypass path
kTLS
yes
no
TLS in userspace via rustls / schannel
In practice the Windows path runs ~25-40% the throughput of the Linux
path.
splice for zero-copy proxying — TransmitFile covers static
files, not arbitrary fd-to-fd transfer. The
TCP proxy recipe returns
IoError::NotSupportedOnPlatform { platform: "windows" }.
Build + capability probe
> cargo build --target x86_64-pc-windows-msvc --release
> .\your-binary.exe
zero-io: RIO + IOCP windows 10.0.22631 (Windows 11 23H2)
zero-io: RIORegisterBuffer ok
zero-io: RIOReceiveEx ok
zero-io: RIONotify (IOCP) ok
zero-io: PostQueuedCompletionStatus ok
zero-io: SetThreadSelectedCpuSetMasks ok
zero-io: started shard 0 on CPU set {0}
Operational notes
Profiling : Visual Studio Diagnostic Tools, Windows Performance
Recorder (wpr) for ETW traces, PerfView for callstacks. The
runtime emits ETW providers under zero-io.runtime.
See also : io_uring for what the Linux Tier 1
path provides ; Tier 1 / 2 / 3 for the
support-tier policy ; why Linux first for the
rationale behind the perf-tier choice.
Configuration
Most knobs are auto-tuned. When you need to override, the runtime
provides three layers — TOML on disk, env vars, programmatic Config
— with later layers winning over earlier. Every effective value is
logged at boot so you always know what's auto, what's overridden, and
what the resolved number was.
The three layers (later wins)
1. TOML file (zero_io.toml or path passed to Config::from_toml)
2. env vars (ZERO_IO_* e.g. ZERO_IO_POOL_SLOT_COUNT=8192)
3. programmatic builder (Config::default().pool_slot_count(8192))
Reading order : TOML deserializes into a Config (auto where missing),
env vars override matching fields, the programmatic builder overrides
both.
Effective<T> — the auto / override pattern
Every tunable that has an "auto" mode (typically 0 for counts, []
for lists) returns an Effective<T> at boot :
Boot validation refuses overlapping pinning sets (5 forbidden
combinations checked) and refuses settings that would saturate the
shard's address space.
--show-config exits without starting the runtime ; useful to verify
deploy-time TOML / env interpretation before turning on traffic.
See also : adaptive runtime for what the
runtime tunes dynamically vs at boot ; wait strategies
for the SQPOLL / NAPI / timeout decision ; backpressure cascade
for the per-protocol overload thresholds the config knobs control.
Wait strategies
poll(timeout) blocks the shard until either work arrives or the
timeout expires. How the shard waits — busy-spin, kernel sleep,
SQPOLL, NAPI busy-poll — is a tunable with a real latency-vs-CPU
trade-off. Pick by workload shape.
The three primitives
The runtime composes three independent wait mechanisms. They stack —
SQPOLL handles submission, NAPI handles RX latency floor, the
top-level poll(timeout) handles "give up CPU after N ms".
Duration::ZERO is the HFT pattern.
Duration::from_millis(100) is the "reasonable default".
Duration::MAX is the "wake-only" pattern — pure event-driven, fine
when traffic is bursty.
There's no auto-adapt — the runtime can't guess your latency budget.
2. SQPOLL (submit-side, on / off)
IORING_SETUP_SQPOLL (kernel 5.13+) tells the kernel to spawn a
helper thread that polls the shard's SQ ring. With it on, submitting
an SQE is "write to userspace memory" — no io_uring_enter syscall.
Config::default().with_uring(|u| u
.sqpoll(true)
.sqpoll_idle_ms(50) // helper sleeps after 50 ms of no SQEs
.sqpoll_cpu(Some(3))); // pin helper to core 3
Setting
Cost
Win
When right
sqpoll(false) (default)
per-batch syscall
0 always-on
low pps, embedded, dev
sqpoll(true) + idle 50ms
helper thread (idle: ~0 CPU after 50 ms)
submit-syscall-free under load
medium-to-high pps
sqpoll(true) + idle MAX
helper thread always running
always submit-syscall-free
sustained high pps
sqpoll(true) + sqpoll_cpu(N)
as above, pinned core N
NUMA-local submit
NUMA-aware multi-shard
3. NAPI busy-poll (RX-side floor)
IORING_REGISTER_NAPI (kernel 6.7+) tells io_uring_enter to busy-poll
the NIC driver's NAPI queue for up to N µs before sleeping. RX
latency floor drops from low µs to sub-µs.
Config::default().with_uring(|u| u
.napi_busy_poll(Duration::from_micros(50))); // off if Duration::ZERO
Setting
Cost
Win
When right
napi_busy_poll(0) (default)
0 always-on
nothing
most workloads
napi_busy_poll(50us)
~50 µs CPU per poll cycle
RX latency floor halves
latency-sensitive
napi_busy_poll(200us)
proportional CPU
sub-µs RX
HFT, low-latency trading
Requires NIC driver support (most modern Intel / Mellanox / Broadcom).
Falls back to "no busy-poll" if not available.
NAPI busy-poll withDuration::MAX is a contradiction — busy-poll
only kicks in inside io_uring_enter. Use NAPI with finite timeouts.
Cross-thread wakeup — always on
When an external thread wants to wake a sleeping shard :
Platform
Mechanism
Wake latency
Linux 6.7+
IORING_OP_FUTEX_WAIT
sub-µs
macOS 14+
__ulock_wait2 / __ulock_wake
low-µs
Windows 10 1809+
PostQueuedCompletionStatus to the IOCP
low-µs
Wired automatically per platform. You don't configure this. The
cross-thread channels page has the
deep dive.
Decision matrix by workload
Workload
poll(timeout)
SQPOLL
NAPI busy-poll
Comment
HFT order entry, sub-µs floor
ZERO
on, idle MAX
on, 100-200 µs
Full core, no sleeps
Latency-sensitive trading data
1 ms
on, idle 50 ms
on, 50 µs
Low-µs floor, sub-ms top
API server, 50k req/s
10 ms
on, idle 50 ms
off
Batch-friendly
Generic web server, 5k req/s
100 ms (default)
off
off
Simplest, lowest idle CPU
Embedded SBC, low-pps
100 ms
off
off
Minimal CPU floor
Idle-mostly cron-like service
MAX
off
off
Wake only on traffic
Anti-patterns
Duration::ZERO + low pps — full core to wait for one packet
per second. Use a finite timeout or wake-on-event.
NAPI busy-poll on a Hyper-V VM — virtual NICs typically don't
expose NAPI. Falls back silently.
SQPOLL with sqpoll_idle_ms = 0 — equivalent to "never sleep" ;
always-on full-core CPU budget for the helper thread.
poll(MAX) without IoHandle consumers — fine until you add an
async task that can't reach the shard.
Cost/benefit summary (target figures)
Numbers below are design targets ; the bench-CI gate publishes
measured values.
Choice
Idle CPU per shard
RX wakeup latency
Submit cost / SQE
Default (poll 100ms, no SQPOLL, no NAPI)
< 1%
~1-5 µs
1 syscall / batch
SQPOLL on, idle 50ms
helper-thread idle
unchanged
~0 syscalls (warm)
NAPI 50µs
proportional to call freq
< 1 µs
unchanged
poll ZERO + SQPOLL idle MAX + NAPI 100µs
100% (HFT mode)
sub-µs
~0 syscalls
See also : io_uring for the SQPOLL / NAPI feature
floors and config knobs in detail ; the poll cycle
for what each phase does inside poll() ; cross-thread channels
for the wakeup primitive that lets Duration::MAX actually wake.
Adaptive runtime
What the runtime adjusts dynamically, what stays static at boot, and
why the line is drawn where it is. The short version : capability
detection and topology probing happen at boot ; sizing of memory
arenas is also locked at boot ; the only true runtime adaptation is
io_uring ring resize on kernel 6.13+. Everything else is one-shot,
deterministic, and logged.
Adaptive at boot — what the runtime probes
Before the first poll() returns, the runtime detects :
What
How
Why static after
Kernel version + io_uring features
uname + per-feature SQE probes
Features either exist or don't ; mid-life kernel upgrade requires restart
Each line is one decision. The boot log is the audit trail.
Adaptive at runtime — what changes after boot
1. SQ/CQ ring resize via IORING_REGISTER_RESIZE_RINGS (kernel
6.13+) — the only ring you might need to grow. The runtime monitors
the high-water mark of pending SQEs vs the ring depth ; if the ring
is regularly full, it resizes upward (doubling, capped at
max_ring_depth). Resize is a single syscall, microseconds, doesn't
drop in-flight SQEs.
zero-io: shard 0 — SQ ring high-water 1024/1024 for 30 s, resizing to 2048
zero-io: shard 0 — SQ ring resized 1024 → 2048 (RESIZE_RINGS)
If the kernel is < 6.13, the runtime doesn't shrink ring depth
silently — boot picks a depth based on observed peak in the first few
seconds, then sticks.
2. SQPOLL idle timeout (passive) — the kernel's SQPOLL helper
thread itself decides when to sleep based on sqpoll_idle_ms (set
once at boot). It wakes immediately on a fresh SQE submission. No
runtime adjustment.
3. NAPI busy-poll budget (passive) — same. The kernel decides
when to stop busy-polling based on the configured budget.
4. Connection table compaction (rare, opt-in) — if you set
shard.compact_on_idle(true), the shard runs a brief compaction pass
when traffic falls below a threshold. Reorders hot connections to the
start of the table for cache locality.
That's the entire list. There is no auto-grow on the pool. There is
no auto-spawn of new shards. There is no dynamic CPU repinning.
What's static by design — and why
Pool slot count. Set at boot, never grown. Reason : the slot
arena is one contiguous registered buffer ; growing it would require
re-registering with io_uring (drops in-flight SQEs) and re-mapping
the BUF_RING.
Connection table size. Set at boot, never grown. Reason : each
connection slot is a fixed-size struct in a contiguous array.
Shard count. Set at boot. Reason : adding shards mid-flight
requires re-routing established connections (DCID-based stickiness),
which we don't ship.
SQPOLL on/off. Set at boot. Reason : turning SQPOLL on/off
requires re-initialising io_uring with different setup flags.
NIC backend choice. Set at boot. Reason : io_uring vs AF_XDP
is a different kernel mechanism ; switching mid-flight equivalent in
cost to a restart.
Why no auto-grow on the pool
This is the most-asked "why not". Three reasons :
Predictability beats elasticity. A service that doubles its
memory at peak load is the worst pattern for capacity planners.
The cascade IS the design. Pool exhaustion is a signal that
propagates back through the protocol layer. See
backpressure cascade.
Auto-growth is a bug source. Pool pointers cross threads, get
encoded in CQEs, registered with io_uring. Growing concurrently
with active SQEs is a recipe for use-after-free.
Adaptive load-shedding
The runtime ships passive shedding hooks per protocol :
Protocol
Pressure signal
Default policy
UDP
Event::PoolPressure if pool > 90%
Drop new datagrams ; emit metric
QUIC
Same + flow-control window pull
Reject new streams ; emit metric
TCP
Same + zero recv window
Don't accept new connections (kernel queue fills)
HTTP
Same + per-conn budget
Return 503 on new requests
Each can be overridden via the protocol's listen config. Default policy
is "drop, observe, emit metric" — you decide whether to escalate.
Cluster-wide adaptations
DcidDispatch (the central-dispatcher routing strategy) optionally
does load-aware Initial routing : on a fresh QUIC handshake, the
dispatcher picks the least-loaded shard rather than hashing.
Bind sockets across shards via SO_REUSEPORT or the BPF program.
Run a short health probe to confirm each shard's alive.
Only then accept external traffic.
Boot time scales with shard count : ~10 ms / shard on Linux Tier 1.
See also : configuration for the static
knobs the boot probes feed ; backpressure cascade
for the overload model that justifies "no auto-grow" ; io_uring
for the per-feature kernel-version matrix the runtime probes against.
Backpressure cascade
Overload is inevitable on any production service. The runtime gives
you four pressure points, each with a default behavior and a hook to
override. The cascade flows pool → connection → listener → cluster ;
each level can absorb pressure or pass it upstream.
The cascade overview
Cluster ingress (kernel ringbuf, BPF rate-limit)
▲
│ pass up : flood detected
┌───────────┴───────────┐
│ │
Listener-level admission (refuse new accepts)
▲
│ pass up : conn budget hit
┌───────────┴───────────┐
│ │
Connection-level shedding (close slow conns)
▲
│ pass up : conn unhealthy
┌───────────┴───────────┐
│ │
Pool-level pressure (Event::PoolPressure)
▲
│ first signal
│
Pool occupancy > 90%
Each level can absorb pressure (apply policy locally), or pass it
upstream. Default policy at every level is "drop with telemetry" —
your code escalates explicitly.
Level 1 — Pool pressure
The first signal. Triggered when payload-pool occupancy crosses a
threshold (default 90% of pool_slot_count).
loop {
io.poll(timeout)?;
while let Some(ev) = io.next_event() {
match ev {
Event::PoolPressure { occupancy, threshold, slot_kind } => {
metrics::pool_pressure.inc();
if occupancy > 0.95 {
close_idle_connections(&mut io, 100);
}
}
_ => {}
}
}
}
Event::PoolPressure fires once per poll() cycle as long as the
threshold is crossed. Carries occupancy ratio, threshold, and which
slot kind.
What you do : shed traffic upstream, close idle connections, refuse
new requests. The runtime won't grow the pool ; the signal is your
chance to apply policy.
Level 2 — Connection-level shedding
When pool pressure persists or per-connection bandwidth misbehaves,
close offending connections.
The runtime tracks per-connection metrics :
let stats = io.conn_stats(conn);
// stats.bytes_in_unack — TCP / QUIC stream backlog
// stats.last_recv_at — silent connection
// stats.cwnd, stats.rtt — TCP / QUIC congestion state
// stats.pool_slots_held — payloads outstanding for this conn
Default shedding criteria :
Symptom
Default action
pool_slots_held > 64 (handler isn't draining)
close_graceful(conn, 1s)
bytes_in_unack > 10 MiB (peer not reading TX)
close_graceful(conn, 1s)
last_recv_at > 30s ago and listener at 80% capacity
The filter builds a small per-IP token bucket in BPF, drops packets
over budget at XDP_DROP. For AF_XDP clusters, the same filter lives
in the XDP program attached to the NIC driver — drops happen before
the kernel network stack ever sees the packet.
What it can't do : application-layer rate limits (per-API-key,
per-user). Those are application-layer ; you wire a tower middleware
or a manual check inside your handler.
Per-protocol cascade summary
Protocol
Level 1 (pool)
Level 2 (conn)
Level 3 (listen)
Level 4 (cluster)
UDP
drop new datagrams
n/a (connectionless)
drop on rate-cap
XDP_DROP per-IP
QUIC
reject new streams
close conn (CONNECTION_CLOSE)
refuse new conns
XDP_DROP per-IP
TCP
(kernel slows window)
close (FIN or RST)
refuse accept
XDP_DROP / SYN cookies
HTTP
(uses TCP cascade)
(uses TCP cascade)
503 reply
XDP_DROP per-IP
WebSocket
(uses TCP cascade)
(uses TCP cascade)
refuse Upgrade
XDP_DROP per-IP
Telemetry — what to watch
Counter
What it tells you
zero_io_pool_pressure_total
Level 1 fires
zero_io_conn_shed_total{reason}
Level 2 fires
zero_io_listen_reject_total{listener}
Level 3 fires
zero_io_ingress_drop_total{reason}
Level 4 fires
zero_io_pool_occupancy_ratio
leading indicator (gauge)
zero_io_conn_pool_slots_held{conn_id}
per-conn pressure (high cardinality, opt-in)
Cascade-firing should be rare in steady state. Spikes correlate with
real overload — alert on pool_pressure_total > 0 for 1 minute.
Anti-patterns
Catching Event::PoolPressure and ignoring it. The signal is
meant to drive policy.
Setting max_connections too high. Past a threshold, more
connections is more memory, more table lookups, more cascade
pressure.
Disabling the ingress filter "to be safe". Without level 4,
abusive peers can saturate kernel softirq before any userspace
check fires.
Per-IP rate limits at level 4 with NAT/CGNAT clients. A whole
ISP shows up as one IP. Use level 3 (per-API-key) for those.
Custom levels
The cascade is not extensible at runtime. Application-layer pressure
goes inside your handler, not into the runtime.
See also : pool system for what fills up at
level 1 ; adaptive runtime for why the
runtime doesn't auto-grow to relieve pressure ; observability
for the metrics that fire ahead of cascade events ;
multi-shard routing for level 4 routing
strategies.
Observability
Three surfaces : per-shard counters, per-cycle structured tracing
spans, and platform-native exporters (Prometheus, ETW, dtrace, BPF).
Pick by ops shape — if your existing infra is Prometheus, the runtime
exposes a Prometheus endpoint ; if you're on Windows, ETW is wired ;
on macOS, dtrace and Instruments work natively. None of these are
mandatory ; the runtime runs fine without anything observed.
Layer 1 — Counters
Every shard exposes structured stats reads. They're cheap (atomic
loads) and safe to call from any thread :
Per-connection details (io.conn_stats(conn_id)) carry the same shape
plus connection-specific fields (RTT, cwnd, bytes-in-unack, peer
address) — opt-in because per-conn metrics have unbounded
cardinality.
Layer 2 — Tracing spans
Structured spans via the tracing crate. Off by default ; opt-in via
the tracing-uring Cargo feature :
[dependencies]
zero-io = { version = "...", features = ["tracing-uring"] }
Span-level overhead is non-trivial (low-µs per span). The runtime
gates it behind a feature flag so the default zero-alloc / zero-lock
hot path stays clean.
Layer 3 — Prometheus
The zero-io-prometheus crate ships a /metrics endpoint shape
that's already wired to Layer 1 counters :
use zero_io_prometheus::PrometheusExporter;
let exporter = PrometheusExporter::new(&io)?;
io.http_listen_with(
HttpListenConfig::new("127.0.0.1:9090".parse().unwrap())
.handler(exporter.handler())
)?;
Metric names follow Prometheus conventions — zero_io_<group>_<name>,
labels for low-cardinality dimensions only (shard id, listener,
protocol). High-cardinality labels (conn id, user id, request id)
are deliberately not exposed at this layer ; the
scripts/check-metrics-cardinality.sh audit (CLAUDE.md §1.5) blocks
PRs that introduce them — unbounded labels would explode TSDB
storage. Per-conn observability goes through tracing spans.
For multi-shard clusters, the exporter aggregates across shards by
default ; pass PrometheusExporter::per_shard(true) to expose each
shard separately (label shard_id).
Layer 3a — ETW (Windows)
The Windows backend emits ETW (Event Tracing for Windows) providers
under the GUID zero-io.runtime. Capture with :
USDT probes are a few-ns cost when nobody's tracing, free when
inactive.
Layer 4 — OpenTelemetry
The zero-io-otel crate adapts Layer 2 spans to OpenTelemetry's
trace API :
use opentelemetry::global;
use zero_io_otel::OtelLayer;
let tracer = init_otel_pipeline();
tracing_subscriber::registry()
.with(OtelLayer::new(tracer))
.with(EnvFilter::from_default_env())
.init();
Same span schema as Layer 2, exported via OTLP.
Cardinality budget — the unwritten rule
Don't add labels with unbounded cardinality (per-user, per-IP, per-
request). Production TSDB storage cost grows quadratically with
cardinality, and unbounded labels are how on-call gets paged at 3 AM
about disk-full alerts.
Per-conn / per-request observability lives in tracing (sampled), not
in metrics (aggregated).
Suggested alerting
Three-alert minimum :
Alert
Threshold
Why
pool_pressure_total > 0 for 1m
First level of cascade firing
Real overload, not transient
listen_reject_total > 0 for 30s
Third level firing
Service genuinely refusing work
shard_poll_p99 > 100ms
Shard-level stall
Handler doing work it shouldn't
Plus the standard golden-signals (request rate, error rate, p99
latency) per protocol — those come from Layer 1 counters via
Prometheus.
See also : profiling for the diagnostic side
of the same observability surface ; backpressure cascade
for the events these metrics correlate with ; adaptive runtime
for the boot-time decisions logged once.
Profiling
When the service is slower than expected, profiling triages where the
time is going. The runtime is built so the answer is usually
findable in three steps : CPU → syscalls → allocations, in that
order.
The three-step triage
Service slower than SLO
│
┌───────────┴───────────┐
│ 1. Is CPU saturated ? │
└───────────┬───────────┘
│
YES → flamegraph │ NO → step 2
(perf record) │
│
┌────────────────────────┴───────────────────────┐
│ 2. Are we waiting on syscalls or on the kernel ?│
└────────────────────────┬───────────────────────┘
│
YES → strace / bpftrace │ NO → step 3
io_uring tracepoints
│
┌────────────────────────┴───────────────────────┐
│ 3. Is allocation in the hot path ? │
└────────────────────────┬───────────────────────┘
│
YES → heaptrack / dhat │ NO → it's in your handler logic
`zero_alloc_proof` │ — protocol-specific tracing
bench gate │
90% of "service feels slow" issues fall in step 1 or step 3.
Step 1 — CPU bound
top / top -H — first look. If one core is at 100% in
zero-io-shard-N, you're CPU bound on that shard. If kernel softirq
is high (%si column), io_uring is busy — try GSO/GRO, or migrate
to AF_XDP.
perf top -p $(pidof your-binary) — live sampling. Shows top
functions by CPU time. The runtime emits frame pointers (-C force-frame-pointers=yes) so stacks resolve cleanly.
Look for : (a) tall stacks in your handler — that's the hot path ;
(b) wide bases in __memcpy — somebody's allocating ; (c) wide bases
in _raw_spin_lock — kernel-side contention.
Step 2 — Syscall bound
If CPU isn't saturated but throughput is bounded, the shard might be
spending too much time in syscalls.
strace -c -p $(pidof your-binary) — counts syscalls. With
io_uring + SQPOLL, you'd expect io_uring_enter to be entirely
absent ; lots of recvmsg / sendmsg directly means the runtime
isn't in io_uring mode.
perf trace -p $(pidof your-binary) — same, with timing per
syscall.
bpftrace for io_uring tracepoints — the most precise tool :
If a specific op has a long-tail latency, the kernel is the
bottleneck — likely NIC queue depth, IRQ affinity, or softirq
starvation.
USDT probes — runtime-emitted, no setup needed beyond bpftrace.
See observability — Layer 3c for the probe
list.
Step 3 — Allocation bound
If CPU is fine and syscalls are fine but throughput is bounded, an
allocation in the hot path is the most common culprit.
zero_alloc_proof (CI gate, your handler) — adapt for your
handler :
#[test]
fn my_handler_zero_alloc() {
let mut io = Io::new(Config::default()).unwrap();
// … warm up …
let baseline = COUNTER.alloc_count.load(Ordering::Relaxed);
run_test_workload(&mut io, 1000);
let after = COUNTER.alloc_count.load(Ordering::Relaxed);
assert_eq!(after - baseline, 0, "handler allocated on hot path");
}
Any non-zero delta is a regression.
heaptrack ./your-binary — records every allocation, lets you
explore by call stack. Slow (10x+) but exhaustive.
dhat (Rust crate) — lower overhead than heaptrack for
production-shape profiling.
Allocator-aware tracing — wire a counting allocator just for the
hot path :
let baseline = ALLOC_COUNT.load(Ordering::Relaxed);
// … one poll cycle …
let delta = ALLOC_COUNT.load(Ordering::Relaxed) - baseline;
if delta > 0 {
eprintln!("allocs in poll cycle: {delta}");
}
Per-backend profiling specifics
io_uring (Linux Tier 1)
/proc/$pid/fdinfo/<n> for the io_uring ring fds — per-ring
stats : SQ size, CQ size, head/tail, op counts.
bpftool prog show to list attached BPF programs (sk_reuseport,
xdp, ingress filter).
bpftrace tracepoints under io_uring:*.
AF_XDP (Linux Tier 1 opt-in)
XDP runs before the kernel network stack. Standard tools don't see
it :
xdpdump -i eth0 instead of tcpdump.
bpftool map dump for xsk_map state, dropped counters,
redirect counters.
ethtool -S eth0 | grep xdp for NIC-driver XDP counters.
kqueue (macOS Tier 2)
Instruments — Time Profiler / System Trace — Apple's profiling
GUI.
dtrace -n 'syscall:::entry / pid == $target /'.
Activity Monitor — quick first look.
RIO + IOCP (Windows Tier 3)
Windows Performance Recorder (WPR) + Windows Performance
Analyzer (WPA) — record an .etl trace, open in WPA. Runtime
emits ETW under zero-io.runtime.
PerfView for callstack flamegraphs from ETW.
Process Explorer + Process Monitor for syscall-level traces.
Latency histograms
Average latency lies. Always profile p99, p99.9, p99.99 — the tail is
where SLOs live and die. The runtime emits per-handler histograms via
the histogram-ddsketch crate when tracing-uring is on :
mpsc::send / IoHandle wide → too many cross-thread sends,
batch in your async producer.
tracing_subscriber::* wide → tracing layer overhead, drop
tracing-uring for production.
If the wide function is in a kernel-side stack (softirq, ksoftirqd,
io_wq), it's a kernel-tuning issue — IRQ affinity, NIC offload
settings, or the wrong backend choice.
Quick recipes (copy-paste)
# Top CPU functions, live (5s sample)
sudo perf top -p $(pgrep your-binary)
# Flamegraph from a 30s trace
sudo perf record -F 99 -g -p $(pgrep your-binary) -- sleep 30 \
&& perf script | stackcollapse-perf.pl | flamegraph.pl > flame.svg
# Syscall summary
sudo strace -c -p $(pgrep your-binary)
# io_uring submit-to-complete latency, per-op histogram
sudo bpftrace ./scripts/uring-latency.bt
# AF_XDP packet capture
sudo xdpdump -i eth0 -w trace.pcap
See also : observability for the per-shard
counter / tracing surface that profiling builds on top of ;
wait strategies for the SQPOLL / NAPI knobs
profiling helps you tune ; benchmarks methodology
for how the project's own perf gate works.
Tier 1 / 2 / 3
zero-io runs on three platforms, with explicit and different commitments.
This section is the source of truth for what each tier means and what to
expect.
Tier matrix
Tier
Platform
Backend
Status
CI
1
Linux 6.7+
io_uring (172) + AF_XDP optional (174b)
Full feature set, zero-alloc gated
Required, gating
2
macOS 14+
kqueue + sendmsg / recvmsg
Parity API, perf parity goal
Required, non-gating until promotion
3
Windows 10 1809+
RIO + IOCP
Parity API, perf parity goal
Required, non-gating
What "Tier 1" actually means
Linux 6.7+ is where the three zeros are measured and gated. The
zero_alloc_proof test runs against a representative workload (UDP, QUIC,
TCP, HTTP) and asserts the counting allocator delta is exactly zero. The
memcpy_proof test asserts no userspace memcpy on the data path. Both block
merges if they regress.
Provided buffer ring (BUF_RING) for true zero-copy RX
SQPOLL for kernel-side submission polling (no syscall on TX hot path)
Registered file descriptors and registered buffers
NAPI busy-poll for sub-µs latency floors
AF_XDP for ultra-high pps (gated to Tier 1 because of the kernel BPF
dependency, even though XDP itself works on macOS BSD-derived kernels in
theory)
kTLS offload (kernel-side TLS encryption ; offloads AES-GCM to the kernel,
enables splice for zero-copy HTTPS file serving)
Linux-only API : multicast_join SSM, cert_hot_reload, splice,
sendfile. These return IoError::NotSupportedOnPlatform { platform } on
Tier 2/3 — they don't panic, they don't silently degrade.
Tier 2 — macOS
macOS gets the same public API. udp_bind, quic_listen, tcp_listen,
Event::*, IoCluster — all work. The backend (KqueueCore + KqueueUdp +
KqueueTcp) uses kqueue for readiness, plain sendmsg / recvmsg for I/O,
QoS classes (USER_INTERACTIVE) for thread priority. __ulock for cross-
process futex.
What's different :
No zero-copy TX — sendmsg always copies. We aim to keep the userspace
side zero-copy ; the kernel just adds one memcpy on TX we can't avoid.
No registered buffers — we still pool, but the kernel has its own
buffers, so each recvmsg is an extra copy from kernel to our pool. (We
keep the userspace &[u8] slice into our pool, so Event<'poll> semantics
hold.)
No multishot accept — accept is a syscall per connection.
Performance — expect a notable gap to Linux throughput for protocols
that lean on these features (HTTP file serving, high-pps UDP) ; concrete
ratio is bench-gate territory. For protocols that are CPU-bound in
userspace (QUIC handshake, TLS), parity is achievable.
Promotion path : when benchmarks land within ±30% of Tier 1 across the
protocol matrix and CI stays green for 30 consecutive days, macOS becomes
Tier 1.
Tier 3 — Windows
Windows gets the same public API. The backend (RioCore + RioUdp +
RioTcp) uses Registered I/O for the data path (Microsoft's equivalent of
io_uring's registered buffers) and IOCP for completion notification. Named
Events for cross-process wakeup.
RIO supports zero-copy in some shapes (registered buffer pool), so the gap to
Tier 1 is smaller than macOS in raw throughput terms. The gap is mostly
operational : Windows server tooling (netsh, ETW) is more invasive than
Linux's, and our test surface there is smaller for now.
Linux-only features (compile_error!-gated)
linux-af-xdp — AF_XDP backend (Tier 1 opt-in)
linux-userspace-tcp — userspace TCP stack on AF_XDP (FreeBSD's
network code ported to userspace ; Tier 1 opt-in, experimental)
linux-ktls — kernel TLS offload
linux-splice — splice(2) for zero-copy file → socket
Each is a Cargo feature ; compile_error! if you try to build on macOS or
Windows. No silent stub, no fake fallback. The platform mismatch is loud.
Cross-platform parity, written down
API : same public types, same method signatures, same error variants.
Behavior : same observable semantics for the cross-platform subset (UDP,
QUIC, TCP, HTTP, WebSocket, WebTransport, TLS).
Performance : Tier 1 is the reference. Tier 2/3 don't promise parity, they
promise correctness and best-effort perf.
If you're shipping a Tier 1 product, target Linux. If you're shipping a
desktop client that talks to a server, all three tiers cover you. If you're
shipping a Windows-first desktop product with deep RIO needs, file an issue
— we're committed but small numbers move the needle.
Because io_uring is the only mainstream kernel I/O interface where you can
have zero alloc, zero copy, and zero context switch on the hot path,
simultaneously. On every other platform you give one of those up.
What io_uring uniquely provides
Submission queue (SQ) and completion queue (CQ) shared with the kernel
via mmap. No syscall to read or write entries — userspace produces SQEs
by writing to the ring, kernel produces CQEs by writing to the same shared
memory. Coordination via single MOV instructions to memory-mapped
cache-coherent slots.
SQPOLL : a kernel thread polls the SQ ring. You write SQEs and
literally make zero syscalls on the TX hot path. Throughput-critical paths
pay no enter cost.
Provided buffer ring (BUF_RING) : you pre-register your pool slots
with the ring ; the kernel writes incoming packet data directly into your
slots and tells you which one in the CQE. True zero-copy RX.
Registered file descriptors : accept_multi returns connection FDs
that are pre-registered (no dup2, no fcntl(F_SETFL), no kernel-side fd
table churn). Subsequent read / write on these FDs is faster.
SENDMSG_ZC (zero-copy send) : kernel transmits directly from your
buffer, signals completion when the NIC has DMA'd. One memcpy budget on
TX collapses to zero (just user→pool slot, then DMA).
NAPI busy-poll : pin a kernel-side polling thread to spin instead of
sleep waiting for NIC DMA. Sub-microsecond RX latency floors when you can
afford the CPU.
IORING_OP_FUTEX_WAIT : kernel waits on a userspace futex from the
ring. Cross-process wakeup with no futex() syscall on the wakee side.
What kqueue / RIO don't have
kqueue is event readiness only. Once you know a socket is readable, you
still call recvmsg, which is a syscall, which copies kernel buffer →
user buffer. There's no BUF_RING equivalent.
RIO has registered buffers (closer parity than kqueue), but completion
notification still goes through IOCP, which is one or more syscalls
per batch. The combination doesn't reach io_uring's shared-ring throughput.
You can absolutely build something fast on those platforms. We do — Tier 2
and Tier 3 backends. But "fast" there means competitive with a well-tuned
epoll / recvmsg loop, not "the same shape as Tier 1."
Why kernel ≥ 6.7
IORING_OP_FUTEX_WAIT landed in 6.7. It's load-bearing for cross-process
wakeup without paying a futex() syscall on the waker. The provided buffer
ring (BUF_RING) landed in 5.19. SENDMSG_ZC in 6.0. We could have
fragmented support across kernel versions ; instead, we picked one floor and
held it.
The floor isn't ancient — kernel 6.7 shipped early 2024. Ubuntu 24.04 LTS
ships 6.8. RHEL 9 backports the relevant io_uring features. Container hosts
running > 1y old kernels are increasingly rare.
If you're stuck on 5.x, that's a real constraint — we say so honestly. The
answer isn't "we degrade gracefully," it's "use a kernel that supports
modern io_uring or use a different library." IoError::KernelTooOld at
boot, with a message that names the missing op and its required version.
Decision rationale
There are three kinds of cross-platform stories :
Lowest common denominator — restrict the API to what every platform
supports. Result : you'd never use BUF_RING, SENDMSG_ZC, SQPOLL.
You'd be roughly as fast as a generic epoll library. We refuse this.
Platform shims — write #[cfg(macos)] blocks that emulate Linux
features with VirtualLock-as-mlock, etc. Result : the alternative impl
is dead code on Linux (where the real code runs) and degraded code on
macOS (where the real backend should be kqueue, not a Linux-emulator).
We refuse this. (CLAUDE.md §0.2.1 has the project-wide rule.)
Honest tiers — same API, different backends, explicit perf
commitments per tier. Linux gets the full feature set ; macOS / Windows
get the API and best-effort perf. Linux-only features return
NotSupportedOnPlatform on other tiers, never panic, never silently
degrade. This is what we do.
The cost is honesty. The benefit is that the Linux story is uncompromised,
and the macOS / Windows story is real, not a fiction.
Kernel & version requirements
The exact floors and what each version buys.
Required minimums
Platform
Version
Hard requirement reason
Linux
6.7
IORING_OP_FUTEX_WAIT for cross-process wakeup
macOS
14 Sonoma
__ulock_wait2 semantics + kqueue modern flags
Windows
10 1809
RIO + IOCP combination + WaitOnAddress
Below these floors, Io::new() returns IoError::KernelTooOld { required, found } immediately, with an actionable message :
io_uring IORING_OP_FUTEX_WAIT requires Linux >= 6.7, but detected Linux 5.15.0.
Upgrade your kernel.
We don't fall back to older interfaces. There is no epoll mode on Linux. If
you're stuck on a kernel below 6.7, this library is not for you ; use what
your kernel supports.
Linux feature × kernel matrix
These are the io_uring features we use and the kernel they landed in. All
present in 6.7+ ; we don't drop below.
Feature
Kernel
What it enables
IORING_OP_RECVMSG / SENDMSG
5.5
UDP I/O via the ring
IORING_REGISTER_BUFFERS
5.6
Pre-registered TX buffers
IORING_OP_PROVIDE_BUFFERS
5.7
First version of provided buffers (deprecated)
IORING_FEAT_FAST_POLL
5.7
No epoll roundtrip on hot socket
IORING_OP_LINKED_TIMEOUT
5.5
Per-op timeout linking
IORING_OP_ACCEPT
5.5
Accept via the ring
IORING_OP_ACCEPT_MULTI
5.19
Multishot accept (one SQE → many CQEs)
IORING_OP_RECV_MULTISHOT
6.0
Multishot recv
IORING_BUF_RING (BUF_RING)
5.19
True zero-copy RX, modern provided-buffers
IORING_OP_SENDMSG_ZC
6.0
Zero-copy sendmsg
IORING_SETUP_SQPOLL
5.5
Kernel-side SQ polling
IORING_SETUP_SUBMIT_ALL
5.18
Submit all queued SQEs in one enter
IORING_SETUP_COOP_TASKRUN
5.19
Lower wakeup overhead
IORING_SETUP_DEFER_TASKRUN
6.1
Even lower wakeup overhead
IORING_OP_FUTEX_WAIT / WAKE
6.7
Cross-process wakeup without syscall on wakee ← our floor
Kernel ≥ 6.7 is what we test and CI-gate on. Newer kernels enable additional
optimizations (NAPI busy-poll has been stable since 5.x, IORING_OP_FUTEX*
in 6.7), and we use them when present.
What each Linux version brings (6.7 → 6.19)
Capability probe at boot picks up whatever the running kernel supports.
The table is per-version delta — what each release adds on top of the
previous. Versions ship roughly every 9-10 weeks. Sources : linux-next
release notes + io_uring CHANGELOG ; mirrored in
charting-transport/src/io/linux_uring/capabilities.rs.
Kernel
Released
What zero-io picks up
6.7 ← floor
Jan 2024
IORING_OP_FUTEX_WAIT / _WAKE / _WAITV (cross-process wakeup without syscall on wakee), IORING_SETUP_REGISTERED_FD_ONLY, IORING_FEAT_REG_REG_RING (register registered ring)
6.8
Mar 2024
IORING_OP_INSTALL_FD (install pre-existing fd into the ring), various NAPI busy-poll fixes
6.9
May 2024
IORING_REGISTER_NAPI (per-ring NAPI ID for busy-poll), IORING_SETUP_SINGLE_ISSUER count bumped to 128 (was 32)
6.10
Jul 2024
IORING_FEAT_RECVSEND_BUNDLE (vectorized recv/send with multiple buffers in one CQE), IORING_OP_BIND / _LISTEN (TCP setup via the ring)
6.11
Sep 2024
Various perf fixes ; nothing in our hot path
6.12
Nov 2024
IORING_FEAT_MIN_TIMEOUT (don't wake on absolute timeout if no completions yet — prevents spurious wakeups), multishot timer ops
6.13
Jan 2025
IORING_REGISTER_RESIZE_RINGS (grow SQ/CQ in-place without recreating the ring — useful for live tuning), IORING_OP_FIXED_FD_INSTALL
6.14
Mar 2025
Network stack fixes ; nothing in our hot path
6.15
May 2025
IORING_FEAT_NO_IOWAIT (don't enter iowait state — frees the CPU for other tasks during quiet ticks), IORING_REGISTER_MEM_REGION (register a memory region for ZCRX), IORING_REGISTER_ZCRX_IFQ (zero-copy RX preview — kernel writes directly into a userspace-pinned region without going through socket buffers)
6.16
Jul 2025
ZCRX iteration ; multi-NIC support
6.17
Sep 2025
ZCRX maturation, broader NIC driver coverage
6.18
Nov 2025
IORING_SETUP_SQ_REWIND (rewind SQ tail on submit failure — avoids losing already-prepared SQEs), IORING_SETUP_CQE_MIXED (mixed-size CQE for ZCRX, prerequisite for production-ready zero-copy receive)
6.19
Jan 2026
(latest stable as of writing — ZCRX continues to mature ; capability probe picks up whatever the running kernel actually supports)
You don't need to track this table manually. Io::new() boot logs name
each capability detected on your machine, and Io::caps() returns a
runtime-queryable struct. The takeaway : kernels 6.13+ are noticeably
nicer (live ring resize, no-iowait, ZCRX preview), kernels 6.18+ unlock
production-grade zero-copy RX.
Detection at boot
Io::new() runs a capability probe and logs a one-line report :
Any feature that probes ✗ is auto-disabled. We won't silently use a fallback
that breaks the perf story ; we just don't enable that optimization for this
boot. The full capability struct is queryable at runtime via Io::caps() if
you need to gate higher-level logic.
Optional features and their version asks
Feature
Min kernel
Notes
AF_XDP backend (linux-af-xdp)
5.4 (we cap at 6.7+ uniformly)
Requires CAP_NET_ADMIN and a NIC driver with XDP native mode
Userspace TCP stack (linux-userspace-tcp)
5.4 (we cap at 6.7+)
Experimental, AF_XDP-based, FreeBSD-ported
kTLS offload (linux-ktls)
5.10
TLS 1.2/1.3 ; needs OpenSSL or our rustls integration
splice zero-copy (linux-splice)
always
Used for HTTPS file serving with kTLS
Multicast SSM (linux-multicast-ssm)
always
Source-Specific IGMPv3 joins
macOS feature × version matrix
Feature
macOS
Notes
__ulock_wait2 (Apple's futex)
14 Sonoma
For cross-process wakeup
kqueueEVFILT_READ / WRITE / TIMER
10.6 (always there)
Core readiness
EVFILT_USER
10.6
User-triggered events
SO_REUSEPORT
10.7
Per-socket multi-shard binding
SO_REUSEPORT_LB
14 Sonoma
Load-balanced reuseport variant we exploit
The 14 Sonoma floor is mostly about __ulock_wait2 and SO_REUSEPORT_LB
robustness. Older macOS will be rejected at Io::new() with the same shape
of error as Linux.
Windows feature × version matrix
Feature
Windows
Notes
RIO basic
8 / Server 2012
Available since then but feature-poor
RIO + IOCP combination
10 1809 (RS5)
Stable enough for production
WaitOnAddress (futex-equivalent)
8
We use it from 10 1809+
Modern threadpool API
10 1809
For shard thread management
The 10 1809 floor matches Microsoft's own LTSC support window for RIO. Server
2019 (1809) is supported.
Summary
Linux 6.7+ for production. macOS 14+ for development on M-series and Intel
Macs. Windows 10 1809+ if you must — we'll keep parity, you'll get correct
behavior, perf will be a few iterations behind Tier 1.
Why no async (and when you need it)
async/await is a beautiful tool for the right shape of program. It's the
wrong tool for the network hot path. This is the long version of why
zero-io is a sync poll() loop, and where zero-io-async enters the story.
What async costs you
State machine generation — every async fn compiles into a generator
that's an enum of states. Reasonable, except : on a hot path that runs
millions of times per second, the discriminant load + jump on every
resume is a measurable overhead. Some compilers vectorize it away ; many
don't.
Allocation at .await boundaries — futures that capture references
to non-'static data force boxing. Box::pin shows up in the symbol
table of any non-trivial async runtime. We don't allocate on the hot
path ; async fights us on this.
Trait objects for handlers — tower::Service is great, but it's
Box<dyn Future + Send> under the hood. Each request goes through one
vtable indirect, one heap allocation, one Box::pin. On the order of
~200 ns per request on x86 (target figure ; bench-gate validates) ;
~2 % of a 10 µs request budget ; ~20 % of a 1 µs request budget.
Backpressure via channels — async libraries propagate backpressure
through mpsc channels. Channels have their own buffers, their own
contention, their own copies. The backpressure cascade in zero-io
(Healthy → Warning → Critical → Drain) doesn't need any of that ; it's a
pool occupancy gauge, observable atomically by every handler in O(1).
Cooperative scheduling latency — async tasks yield at .await
points. Between yields, they monopolize the worker thread. A long
CPU-bound section in one task delays every other task on that worker.
In practice : a sync loop that you control runs predictably ; an async
loop runs at whatever the scheduler decides.
What async gives you
Composability — tokio::join! parallel sub-tasks, select!
cancellation, tokio::spawn task scheduling. Useful for code that's
naturally concurrent (DB pool with N drivers, gRPC with M streams).
async fn ergonomics — readable sequential logic that's actually
non-blocking. For business logic, this is the right shape.
Library ecosystem — anything in async Rust : reqwest, sqlx,
redis, tonic. Vast and mature.
These are real wins, and we don't try to take them away from you. We just
want them on the right side of the boundary.
Sync fast path inside the shard. Async slow path in tokio. Bridge via
OwnedSlot (RX) and IoHandle (TX). No allocation on the shard side ;
allocations and futures live in the tokio runtime where they're at home.
When you don't need the bridge
Pure protocol handlers — TCP echo, UDP forwarder, WebSocket fan-out, anything
where the handler is a state machine that processes input and produces
output deterministically — work entirely sync. You don't need
zero-io-async. Compile it out.
When you do need the bridge
Async DB driver behind your API (sqlx, redis, mongodb)
Async outbound HTTP / gRPC calls
Existing tokio-shaped middleware you want to keep
Tower Services, axum apps
Computation that benefits from tokio::spawn_blocking (hashing, image
processing, anything CPU-bound that shouldn't block the shard)
The bridge is in Async integration. The cost is
explicit — detach_event_data is an atomic refcount bump, channel send
adds queue ops, tokio task spawn schedules onto the executor — order-of-
magnitude tens to low hundreds of ns each on x86 (bench-gate validates
exact figures). You see it in the profile,
you choose to pay it where it makes sense.
The principle
Async is a great way to express concurrency in your code. Sync poll() is
a great way to express concurrency at the I/O layer. They aren't enemies ;
they're at different altitudes. We give you both, with a sharp boundary
between them, and your job is to put each piece of work on the side where it
belongs.
Why typestate
Slot<'p, 'b, S> carries a state parameter S. The 6 states (Reserved,
Committed, InFlight, Completed, Borrowed, SharedRead) live in the type
system, not in a runtime enum. This is unusual ; this page is why we did it.
The runtime alternative
// the version we didn't write
struct Slot {
buf: *mut u8,
state: AtomicU8, // Reserved=0, Committed=1, InFlight=2, ...
}
impl Slot {
fn write(&mut self, data: &[u8]) {
match self.state.load(Acquire) {
STATE_RESERVED => { /* ok */ }
other => panic!("write on slot in state {other}"),
}
// ...
}
fn submit(&mut self) {
match self.state.load(Acquire) {
STATE_COMMITTED => { /* ok */ }
other => panic!("submit on slot in state {other}"),
}
// ...
}
}
This is what you'd write in C, Go, or 90% of Rust libraries. It works. It's
also a runtime panic waiting to happen — the moment somebody calls submit
on a Reserved slot, the program dies, and the bug only shows up at the
exact code path that triggers it.
The fundamental problem : "this slot is in state X" is a fact at compile
time in 99% of cases (the function signature dictates which state it's in),
but the runtime check forces the compiler to assume nothing.
Slot<_, _, Reserved> doesn't have a submit method. The compiler refuses
to call it. There is no runtime check, no panic, no UB. The illegal state
transition is uncompilable.
What we get for the trouble
Zero runtime cost — no state field, no atomic load, no match. The
state lives in PhantomData<S> which is () at runtime.
Self-documenting API — the function signature tells you what state
the slot is in coming in, what state it's in going out. You don't have
to read the body or the docs.
Refactor-safe — change the state machine, the compiler walks every
call site that's now wrong. With runtime states, you have to remember to
audit every match arm.
No "what if" defensive code — handlers don't have to write
if state == Reserved { ... } else { unreachable!() }. The
unreachability is structural.
What it costs
Verbose to write — six state structs × ~10 methods on each = ~60
impl blocks. Worse, because some methods exist in multiple states
(e.g., release() works from both Completed and Borrowed), they
duplicate.
Generic explosion in error messages — when something doesn't compile,
the error mentions Slot<'static, 'b: 'p, Borrowed> and similar.
Readable once you're used to it ; intimidating on first encounter.
Library-internal complexity — the generic parameters propagate up to
any helper that takes a slot. We have type aliases (type ReservedSlot<'p, 'b> = Slot<'p, 'b, Reserved>) to keep call-site signatures readable.
When typestate doesn't pay
If the state lives at runtime anyway (e.g., in a database, in a config
file), reflecting it into types is theater. Use a runtime check.
If the call sites are < 5 and the state machine has < 3 states, the cost of
typestate exceeds the gain. Use a runtime check.
If multiple threads can change the state concurrently and the API has to
serialize them, the typestate has to encode the lock, which gets weird fast.
Use a runtime check (often with a different abstraction altogether, like an
actor model).
When it pays — the slot case
Slot is hot-path, single-threaded (within a shard), used in dozens of
call sites, with a state machine that has six well-defined states and clear
transition rules. Every illegal transition we'd otherwise have to defend
against in runtime code is now a compile error. Refactoring the state
machine (e.g., adding SharedRead for GRO fanout in §172a.5.5) was a
mechanical exercise — the compiler told us every place that needed to
change.
The trade-off is right for our case. It would be wrong for most APIs.
That's why most APIs don't do this.
The 'b parameter is a brand. Two slots from two different pools have
different 'b (introduced by an HRTB closure : with_brand(|brand_pool| {...})). Mixing slots between pools is a compile error. Same idea as
typestate (encoding a runtime invariant in the type system), different
purpose (preventing pool confusion vs. preventing state confusion).
If you've used ghost_cell or read about "brand lifetimes" in Rust : that's
the pattern. We use it because pool confusion would be a real bug class
without it (small slot pool vs. large slot pool, per-shard pools), and the
HRTB closure pattern adds zero runtime cost.
Bottom line
Typestate is not a default. It's a tool for cases where the cost of a
runtime panic is high, the call sites are many, and the state machine is
small enough to encode without exploding the type surface. Pool slots are
exactly that case.
Why QUIC-LB minimal
The QUIC Connection ID format we ship in v1 is the simplest thing that meets
the goal. It got there by going through six iterations of "what about this
threat model?" — and then deleting most of them. The story is a useful case
study in If it's complicated, it's wrong.
The goal
Multi-shard QUIC servers need a way to route packets from the same connection
to the same shard. QUIC's Connection ID (CID), chosen by the server, can
encode the shard index. As long as the routing layer (BPF in the kernel, or a
proxy upstream) can extract the shard from the CID, packets land on the right
shard regardless of NAT rebinding or path migration.
That's the whole functional requirement. Everything else is "what else could
we encode in the CID?".
The R0 design — naive
byte 0 bytes 1-15
[shard_id] [random]
Cap : 256 shards. CBPF program reads byte 0, returns byte 0 % shard_count.
Worked. But : QUIC-LB draft-20 specifies the standard "Plaintext" CID format
that external load balancers (F5, HAProxy, nginx-quic) understand. R0 was
incompatible. Anyone wanting to put zero-io behind one of those LBs would
have had to write custom config.
The R3 design — "everything"
After research into QUIC-LB and threat modeling : six features were added,
all "obviously good ideas" :
QUIC-LB Plaintext format — [CR(2b) | rsvd(2b)] [ServerID(14b)] [Counter(40b)] [SipHash(64b)]. Cap 16384 shards, draft-20 compliant.
Config Rotation (CR field) — hot-swap routing config without dropping
connections. Needed for ops scale-out without downtime.
SipHash integrity tag (last 64b) — detect forged CIDs. Without it, an
attacker could forge a CID with our ServerID and probe the server.
Key rotation + grace period — SipHash key rotates every 24h, old key
accepted during a grace period.
DDoS filter BPF — per-IP rate limit + tag verify in the kernel.
3-tuple SipHash for non-QUIC fallback hashing.
Control channels for runtime reconfig.
This was about 1600 lines of plan, ~15 dependencies, a 2-slot key store with
ABA-safe atomic CAS, multiple BPF maps, and ~9 distinct correctness bugs
found across six review passes. None of it shipped.
The R3.7 design — radical simplification
After 119+ findings across review passes 1-6, the conclusion in R3.7 was :
the root problem is accumulation of complexity. Every feature was
justifiable on its own ; the combination was unmanageable.
What stayed :
byte 0 bytes 1-2 bytes 3-15
[reserved=0] [ServerID BE 14b] [13 random bytes (2^104 forgery)]
What was deleted (and why deletion was the right call) :
Feature
Why deleted
Config Rotation
Operational nicety. Real cost of "scale via restart with GOAWAY" : seconds of in-flight connection drain. Real cost of CR : ~300 lines of code, dual-key state, multi-pass review. We pay restart-time, we save complexity.
SipHash integrity tag
Anti-forgery, but : 13 random bytes = 2^104 forgery space. An attacker has to guess 104 bits of randomness to land on an existing CID. That's already infeasible. Statistical anti-forgery without crypto. ~200 lines saved.
Key rotation + grace
Depended on SipHash. Gone with it.
DDoS filter BPF
Userspace per-IP rate limit (already in §172.11) covers the same threat model with simpler ops. BPF DDoS would need its own BPF map maintenance. ~150 lines saved.
3-tuple SipHash
Kernel built-in 4-tuple hash on bpf_sk_select_reuseport is good enough for non-QUIC fallback (TCP, raw UDP). ~100 lines saved.
Control channels
"Runtime reconfig" needs Config Rotation underneath ; without CR, no need.
Net : ~900 lines deleted, ~9 structural bugs eliminated by construction, BPF
verifier risk near zero, the design fits on one page.
The criterion
CLAUDE.md §0.1 : "If it's complicated, it's wrong." Each removed feature was
defensible in isolation. The aggregate was indefensible. Going simple wasn't
giving up — it was recognizing that the problem we wanted to solve in v1
didn't justify the apparatus we'd built around it.
What we kept future-compatible
byte 0 is reserved (set to 0). It can carry a Config Rotation tag (3 bits)
reserved (5 bits) when needed. v1 CIDs have CR=0 implicit, so they remain
valid in v2. No migration window required. The simple v1 design is forward-
compatible by deliberate design — we removed features, not their hooks.
When the deferred features come back
Each is gated on a real signal :
Config Rotation : when an operator needs hot-reconfig in production
(multi-tenant, SaaS where downtime is contractually expensive). Not
before.
SipHash integrity : when a customer reports active CID-forgery
attacks. 2^104 statistical resistance covers everyone we currently know
of.
DDoS filter BPF : when userspace rate limit is genuinely the
bottleneck. We have telemetry to detect that.
A future step (number TBD) will add them, in order of demand. v1 ships
without them.
The lesson we keep referring back to
Complex designs don't fail because they're wrong. They fail because the
sum of their parts is unmaintainable. The fastest way to ship correctness
is to ship less. Add features when you can prove you need them, not when
you imagine you might.
If it's complicated, it's wrong
CLAUDE.md §0.1, the project's first principle. It governs every design call,
every PR review, every refactor. This page is what it actually means in
practice.
Not "simple is good"
The principle is not "prefer simple solutions" — that's a tautology. The
principle is : when a design accumulates complexity and you can't articulate
why each piece is load-bearing, the design is wrong, not just verbose. You
revert and restart.
The trigger isn't "I don't like reading this." The trigger is :
I added feature X to handle threat Y, but Y is hypothetical.
This module has 6 traits and only one impl of each.
The state machine has 11 states and 3 of them are reached only by tests.
I can't explain to a teammate in 5 minutes why the type signature is
what it is.
The PR title is "small refactor for clarity" and the diff is 800 lines.
When you see one of those, the answer isn't "add comments" or "add tests."
The answer is "revert, take the simple version, ship it, see if the missing
part actually hurts."
Examples from this codebase
Slot typestate — kept, justified
The 6-state typestate is complex by any reasonable measure. Why we kept it :
the runtime alternative panics at the wrong time, the compile-time version
catches every misuse at the call site, refactoring is mechanical. The
complexity is paid for by an outsized correctness gain. See why
typestate.
QUIC-LB R3 → R3.7 — simplified, justified
Six features removed (Config Rotation, SipHash tag, key rotation, DDoS
filter, 3-tuple hash, control channels). All were defensible individually ;
the aggregate was unmaintainable. ~900 lines deleted. 1-2 review passes to
stabilize R3.7 instead of 6+ for R3. See why QUIC-LB minimal.
mode: Embedded | Daemon — never added
Early design proposed an IoCluster::mode field to differentiate "embedded
in another process" from "standalone daemon." Both modes worked the same
way operationally ; the difference was cosmetic. The field would have
proliferated through every layer (config TOML, CLI, ops API). Never added.
The two modes are actually one thing : you compile the binary, you run it.
Vendor patches vs forks
quiche, slint, io-uring are patched in vendor/. We could have
forked them — full source control, freedom to redesign. We didn't. Each
patch is < 500 lines, scriptable to reapply on upstream bumps, and the
patch boundaries are obvious in git diff. A fork would have been "easier
to work in" and impossible to keep current.
BPF program — single-file in R3.7
R3 had a multi-file BPF module (base/bpf/{quic_shard,udp_shard,tcp_shard, common}.bpf.c). Plausible for a complex routing matrix. R3.7 collapsed
to a single qlb_route.bpf.c because the actual logic that ships in v1
fits in 50 lines of BPF. Multi-file would have been right for the design
that didn't ship.
The 97% confidence rule
Adjacent principle, also from CLAUDE.md §0.1 : "97% confidence before
changes — ask follow-up questions otherwise."
It means : if you're not sure why the existing code is the way it is, you
don't get to refactor it. You ask. The codebase has hundreds of small
"why is this here" choices, and most of them have load-bearing reasons.
Removing them because they look extraneous is how regressions ship.
The rule cuts both ways : if you're not sure your new design is the right
one, you don't get to write 800 lines of it. You write the smallest
plausible version, ship it, see if it holds.
The cost
This principle costs us features. Real ones, that real users have asked for.
The trade is : we ship less, faster, with fewer regressions, and we keep the
code small enough that one person can read the whole I/O hot path in an
afternoon.
If you read the API reference and think "this is a lot," consider
that the alternative — a feature-complete-from-day-one async runtime with
every protocol, every mode, every optional thing turned on — is the version
that exists in 50 other libraries already. We're trying to be the one where
the design is auditable.
When to apply it to your own code
You're using zero-io to build something. The same principle applies one
level up :
One protocol per concern — don't fan out into "supports HTTP/1.1,
HTTP/2, HTTP/3, gRPC, WebSocket, SSE, plus a custom framing" for a
service that only ever speaks HTTP/1.1.
One config, not three — if your TOML has both enable_x and
disable_x_legacy_compat, pick one.
Per-shard partitioning before per-shard caches — if you can route
state to the shard that owns it, you don't need a per-shard cache with
invalidation. Caches add complexity orders of magnitude faster than
partitioning does.
Delete after you ship — every time you finish a feature, look for
three things you wrote on the way that are no longer needed. Delete
them. The compiler will tell you what was actually load-bearing.
The discipline is annoying at first. It's the difference between a codebase
that gets faster to work in over time and one that gets slower.
Recipe : UDP echo
🚧 Design preview — gated on step 174. The async
wrapper is gated on step 181 (zero-io-async).
Bind to a port, echo every datagram back to its sender. Three surfaces of
the same program, picked by what you're used to.
Native zero-io — the smallest poll-loop UDP server, ~12 lines of
substance. Zero alloc on the hot path, zero copy from NIC to handler.
use std::time::Duration;
use zero_io::{Io, Config, Event};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
io.udp_bind("0.0.0.0:8080".parse().unwrap())?;
println!("udp echo on 0.0.0.0:8080");
loop {
io.poll(Duration::from_millis(100))?;
while let Some(ev) = io.next_event() {
if let Event::UdpRecv { endpoint, from, data } = ev {
let mut buf = io.send_buffer(data.len())?;
buf.write(data);
io.udp_send(endpoint, from, buf)?;
}
}
}
}
What's happening :
Config::default() allocates a 1024-slot pool of 1500 B each.
udp_bind opens the socket and arms the first recvmsg SQE.
poll(100ms) blocks up to 100 ms in io_uring_enter, drains
completions on return.
Event::UdpRecv { data, .. } is a slice into the pool slot the
kernel filled (zero copy from NIC).
send_buffer(n) checks out a fresh slot from the same pool ;
buf.write copies bytes into it (one memcpy on TX, structurally
required).
udp_send queues the sendmsg SQE. Slot returns to the pool when
the CQE arrives on the next poll().
Drop-in shape compatible with tokio::net::UdpSocket. One task drives
the echo ; UDP is connectionless so no per-peer fan-out needed.
use zero_io_async::net::UdpSocket;
#[tokio::main]
async fn main() -> std::io::Result<()> {
let sock = UdpSocket::bind("0.0.0.0:8080").await?;
println!("udp echo on 0.0.0.0:8080");
let mut buf = vec![0u8; 1500];
loop {
let (n, peer) = sock.recv_from(&mut buf).await?;
sock.send_to(&buf[..n], peer).await?;
}
}
zero_io_async::net::UdpSocket is API-compatible with
tokio::net::UdpSocket. Underneath, the shard runs the same poll-loop +
io_uring path the native view uses ; the async wrapper bridges via
IoHandle.
Drop-in shape compatible with std::net::UdpSocket. Single thread,
blocking recv_from / send_to. Right path for small fleets, admin
endpoints, or when std-shape wins ergonomically.
use zero_io::sync::UdpSocket;
fn main() -> std::io::Result<()> {
let sock = UdpSocket::bind("0.0.0.0:8080")?;
println!("udp echo on 0.0.0.0:8080");
let mut buf = [0u8; 1500];
loop {
let (n, peer) = sock.recv_from(&mut buf)?;
sock.send_to(&buf[..n], peer)?;
}
}
zero_io::sync::UdpSocket mirrors std::net::UdpSocket — same method
names, same blocking semantics. The implementation is a thin sync
wrapper around the zero-io shard ; recv_from blocks until the
underlying poll loop delivers a datagram.
Performance — comparing the three surfaces
Surface
Allocs / pkt hot path
Latency overhead vs native
Concurrency
Native
0 on steady state
reference
single-threaded shard
Async
a few (tokio future state machine)
+200-400 ns / I/O op
single tokio task, multiplexed
Sync
a few (per blocking call)
+500-1000 ns / I/O op
one OS thread
For 1 M+ pps you target the native view ; for a few thousand pps
either wrapper is fine.
Tuning for high pps (any surface)
The native view exposes the most knobs directly :
let cfg = Config::default()
.pool_slot_count(8192) // headroom for in-flight
.with_uring(|u| u.sqpoll(true)) // no syscall on TX hot path
.udp_endpoint(|e| e.gso(true).gro(true)); // GSO/GRO segmentation offload
let mut io = Io::new(cfg)?;
io.udp_bind("0.0.0.0:8080".parse().unwrap())?;
GSO / GRO bundles segments at the NIC layer ; one syscall covers many
packets. SQPOLL eliminates the enter syscall on the TX hot path.
With both, TX is literally userspace writes to a shared ring.
The async / sync wrappers expose the same knobs via bind_with(cfg)
variants.
On Tier 2 / Tier 3 returns IoError::NotSupportedOnPlatform { platform }.
See the multicast SSM recipe for the full SSM /
ASM / IP_PKTINFO surface.
What doesn't work in the native view (and why)
let mut events = Vec::new();
while let Some(ev) = io.next_event() {
events.push(ev); // ❌ won't compile
}
io.poll(Duration::from_millis(100))?; // would invalidate stored events
Events borrow &mut Io ; storing them across poll() is a compile
error. See Event lifetimes for the detach
patterns.
When to pick which
Async : tokio app, drop-in import, ~10 lines change.
Sync : admin-port DNS resolver, IoT broadcast listener, dev
tool, learning Rust networking. std-shape exactly.
Native : > 1 M pps, sub-µs jitter, mixing UDP with QUIC / TCP /
HTTP in one poll loop, embedded boards where every alloc costs.
Recipe : TCP echo
🚧 Design preview — gated on step 176 (TCP handler).
The async wrapper is gated on step 181 (zero-io-async).
The bread-and-butter network recipe. Three surfaces of the same TCP
echo server, picked by what you're used to.
Native zero-io — full poll loop, single shard, no tokio. This is the
shape that gives you the three zeros guarantees.
use std::time::Duration;
use zero_io::{Io, Config, Event, MessageKind};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
io.tcp_listen("0.0.0.0:7000".parse().unwrap())?;
println!("tcp echo on 0.0.0.0:7000");
loop {
io.poll(Duration::from_millis(100))?;
while let Some(ev) = io.next_event() {
match ev {
Event::Connected { conn, peer, .. } => {
println!("accept {conn:?} from {peer}");
}
Event::StreamFrame { conn, kind, data, .. } => {
// TCP push model : kind carries Data / End markers.
if matches!(kind, MessageKind::Data) {
let mut buf = io.send_buffer(data.len())?;
buf.write(data);
// TCP has one implicit stream per connection
let s = io.tcp_stream(conn);
io.stream_write(conn, s, buf)?;
}
if matches!(kind, MessageKind::End) {
let _ = io.close(conn);
}
}
Event::Disconnected { conn, .. } => {
println!("close {conn:?}");
}
_ => {}
}
}
}
}
Same I/O budget as the async / sync variants — the difference is
where the loop lives. Here it lives in your code, you control the
poll timeout, you see every event explicitly. Gain : zero alloc on
the hot path (the async / sync wrappers add per-call alloc for the
future state-machines and the thread sync primitives).
The shortest async path — drop-in shape compatible with tokio::net
ergonomics. Each accepted connection gets its own task ; the task
loops on read / write until the peer closes.
use zero_io_async::net::{TcpListener, TcpStream};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
#[tokio::main]
async fn main() -> std::io::Result<()> {
let listener = TcpListener::bind("0.0.0.0:7000").await?;
println!("tcp echo on 0.0.0.0:7000");
loop {
let (mut sock, peer) = listener.accept().await?;
tokio::spawn(async move {
let mut buf = vec![0u8; 4096];
loop {
let n = match sock.read(&mut buf).await {
Ok(0) => break, // peer closed
Ok(n) => n,
Err(_) => break,
};
if sock.write_all(&buf[..n]).await.is_err() {
break;
}
}
let _ = sock.shutdown().await;
println!("closed {peer}");
});
}
}
zero_io_async::net::TcpListener is API-compatible with
tokio::net::TcpListener. Underneath, the shard is still poll-loop +
io_uring ; the async wrapper bridges via IoHandle + OwnedSlot.
Existing tokio code that just imports the type from zero_io_async
keeps working.
The shortest sync path — drop-in shape compatible with std::net
ergonomics. One thread per connection, blocking reads/writes. Lowest
mental overhead ; not the path you pick for ≥ 10k connections, but
exactly right for a small fleet of long-lived connections (admin
endpoints, IoT devices, dev tools).
use zero_io::sync::{TcpListener, TcpStream};
use std::io::{Read, Write};
use std::thread;
fn main() -> std::io::Result<()> {
let listener = TcpListener::bind("0.0.0.0:7000")?;
println!("tcp echo on 0.0.0.0:7000");
for stream in listener.incoming() {
thread::spawn(move || {
let mut sock = stream.expect("accept");
let mut buf = [0u8; 4096];
loop {
let n = match sock.read(&mut buf) {
Ok(0) => break,
Ok(n) => n,
Err(_) => break,
};
if sock.write_all(&buf[..n]).is_err() {
break;
}
}
});
}
Ok(())
}
zero_io::sync::TcpListener mirrors std::net::TcpListener — same
method names, same blocking semantics, same incoming() iterator.
Performance — comparing the three surfaces
Surface
Allocs / req hot path
Latency overhead vs native
Concurrency model
Native
0 on steady state
reference
single-threaded shard, multiplexed
Async
a few (future state machines + waker)
+200-400 ns / I/O op
tokio task per conn, multiplexed
Sync
a few (kernel→buf copy on read, mutex per blocking call)
+500-1000 ns / I/O op
OS thread per conn
Throughput-critical paths target the native view. Most apps live with
the async or sync wrappers and never feel the overhead. Pick by
ergonomic shape, not by perf chart, until your profile says otherwise.
TLS variant (any of the three)
The zero-io TCP shard has TLS / kTLS support built in — the wrappers
expose it transparently :
// async
let listener = TcpListener::bind_tls("0.0.0.0:443", &cert, &key).await?;
// sync
let listener = TcpListener::bind_tls("0.0.0.0:443", &cert, &key)?;
// native
io.tcp_listen_with(
TcpListenConfig::with_tls("0.0.0.0:443".parse().unwrap(), &cert, &key)
.ktls(true) // kernel TLS offload, Linux only
)?;
With linux-ktls enabled, post-handshake encryption offloads to the
kernel. From userspace, you read / write cleartext ; the kernel
encrypts on the wire. Zero CPU overhead in your handler beyond the
plain TCP version.
Multi-shard scaling (any surface)
For thousands of concurrent connections, you graduate to multi-shard.
The async / sync wrappers expose IoCluster::async_listener(...) /
IoCluster::sync_listener(...) which distribute via SO_REUSEPORT
under the hood. The native view uses IoCluster::new + tcp_listen
on each shard (see multi-shard cluster).
When to pick which
Async : you already have a tokio app. Drop in zero_io_async,
measure. If the gain matters, keep it ; if not, you spent ~10 lines
of import changes.
Sync : admin endpoint, dev tool, small fleet of long-lived
connections (say < 200), or you're teaching someone Rust networking
for the first time. The shape matches std exactly.
Native : you're optimizing for tail latency, you want zero
allocations, or your event loop has more than just TCP (mix UDP +
QUIC + TCP all in one poll cycle).
QUIC server with TLS, accepting connections and echoing both datagrams and
streams. ~50 lines either way.
use std::time::Duration;
use zero_io::{Io, Config, Event, MessageKind};
fn main() -> std::io::Result<()> {
let cert = std::fs::read("cert.pem")?;
let key = std::fs::read("key.pem")?;
let mut io = Io::new(Config::default())?;
io.quic_listen("0.0.0.0:4433".parse().unwrap(), &cert, &key)?;
println!("quic on 0.0.0.0:4433");
loop {
io.poll(Duration::from_millis(10))?;
while let Some(ev) = io.next_event() {
match ev {
Event::Connected { conn, peer, .. } => {
println!("conn {conn:?} from {peer}");
}
Event::Datagram { conn, data } => {
let mut buf = io.send_buffer(data.len())?;
buf.write(data);
io.send_datagram(conn, buf)?;
}
Event::StreamFrame { conn, stream, kind, data } => {
let mut buf = io.send_buffer(data.len())?;
buf.write(data);
let s = io.open_uni_stream(conn)?;
io.stream_write(conn, s, buf)?;
if kind == MessageKind::End {
io.stream_shutdown(conn, s)?;
}
}
Event::Disconnected { conn, .. } => {
println!("conn {conn:?} closed");
}
_ => {}
}
}
}
}
use zero_io_async::quic::{Endpoint, ServerConfig};
#[tokio::main]
async fn main() -> std::io::Result<()> {
let cfg = ServerConfig::with_pem_files("cert.pem", "key.pem")?;
let endpoint = Endpoint::server(cfg, "0.0.0.0:4433".parse().unwrap()).await?;
println!("quic on 0.0.0.0:4433");
while let Some(incoming) = endpoint.accept().await {
tokio::spawn(async move {
let conn = incoming.await?;
tokio::spawn({
let c = conn.clone();
async move {
while let Some(dg) = c.recv_datagram().await? {
c.send_datagram(&dg)?;
}
Ok::<_, std::io::Error>(())
}
});
while let Some(stream) = conn.accept_uni().await? {
tokio::spawn(async move {
let bytes = stream.read_to_end(64 * 1024).await?;
let mut s = conn.open_uni().await?;
s.write_all(&bytes).await?;
s.finish().await?;
Ok::<_, std::io::Error>(())
});
}
Ok::<_, std::io::Error>(())
});
}
Ok(())
}
zero_io_async::quic is API-shaped after quinn — Endpoint,
Connecting, Connection, RecvStream / SendStream. Drop-in for
existing quinn code via import swap.
TLS setup
cert.pem and key.pem : standard PEM-encoded chain + private key. For
local dev, generate with openssl req -x509 -newkey ec:<(openssl ecparam -name prime256v1) -keyout key.pem -out cert.pem -days 365 -nodes -subj "/CN=localhost".
For production : Let's Encrypt or your CA. The cert_hot_reload API (step
182, Linux-only) reloads cert + key without dropping connections.
Multi-shard variant
let cluster = IoCluster::new(ClusterConfig {
shard_count: 4,
routing: RoutingStrategy::ReusePortCbpf, // default, 16384 cap
expected_protocols: ExpectedProtocols::default().with_quic(),
..Default::default()
})?;
cluster.quic_listen("0.0.0.0:4433".parse().unwrap(), &cert, &key)?;
let handles: Vec<_> = cluster.into_shards()
.map(|mut s| std::thread::spawn(move || run_shard(s))).collect();
for h in handles { h.join().unwrap(); }
Each shard runs the same run_shard body (the loop from the single-shard
example). Connection affinity is automatic via the QUIC-LB Plaintext
ServerID in the CID — packets from the same connection always land on the
same shard.
What's deliberately not in this recipe
0-RTT : enabled via QuicListenConfig::enable_early_data(true). The
application has to handle the replay-safety story (idempotent or
replay-protected requests only).
Connection migration : automatic, no code needed. The CID is stable
across path changes ; the shard mapping stays consistent.
Backpressure cascade : the pool can saturate ; handle
Event::PoolPressure to shed load gracefully. See
backpressure cascade.
A real handler — protocol dispatch on the first stream frame, application
framing.
Metrics (per-conn conn_stats(conn) for RTT, cwnd, bytes ; aggregate
pool_stats() for pool occupancy).
Graceful shutdown (close_graceful(conn, timeout) per connection on
signal).
Recipe : QUIC client
🚧 Design preview — gated on step 175. Async wrapper
on step 181 (zero-io-async).
Connect to a QUIC server, send a stream, read the response, exit.
use std::time::Duration;
use zero_io::{Io, Config, Event, QuicConnectConfig};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
let conn = io.quic_connect_with(
"127.0.0.1:4433".parse().unwrap(),
QuicConnectConfig::new("localhost")
.alpn(&["h3", "echo"])
.insecure(true) // dev only ; trust system roots in prod
)?;
let mut sent = false;
let mut received = Vec::new();
loop {
io.poll(Duration::from_millis(10))?;
while let Some(ev) = io.next_event() {
match ev {
Event::Connected { .. } if !sent => {
let s = io.open_bidi_stream(conn)?;
let payload = b"hello quic";
let mut buf = io.send_buffer(payload.len())?;
buf.write(payload);
io.stream_write(conn, s, buf)?;
io.stream_shutdown(conn, s)?;
sent = true;
}
Event::StreamFrame { data, .. } => {
received.extend_from_slice(data);
}
Event::Disconnected { .. } => {
println!("got: {}", String::from_utf8_lossy(&received));
return Ok(());
}
_ => {}
}
}
}
}
use zero_io_async::quic::{Endpoint, ClientConfig};
#[tokio::main]
async fn main() -> std::io::Result<()> {
let mut endpoint = Endpoint::client("0.0.0.0:0".parse().unwrap())?;
endpoint.set_default_client_config(
ClientConfig::with_native_roots()
.alpn(&["h3", "echo"])
);
let conn = endpoint.connect("127.0.0.1:4433".parse().unwrap(), "localhost")?
.await?;
let (mut send, mut recv) = conn.open_bi().await?;
send.write_all(b"hello quic").await?;
send.finish().await?;
let bytes = recv.read_to_end(64 * 1024).await?;
println!("got: {}", String::from_utf8_lossy(&bytes));
conn.close(0u32.into(), b"bye");
endpoint.wait_idle().await;
Ok(())
}
zero_io_async::quic is quinn-shaped. Endpoint::client binds a UDP
socket ; connect returns a Connecting future yielding a Connection
once the handshake completes.
TLS verification
In production, QuicConnectConfig::new("server.example.com") uses the
platform's system CA roots by default. The insecure(true) above is for
dev only — it disables certificate verification. There's no flag for "I
know what I'm doing" — you set insecure explicitly so the next reader
sees it.
The session cache is a user-provided Arc<dyn SessionCache> — TLS session
tickets persist across program runs if you back it with disk storage.
0-RTT data is sent before the TLS handshake completes ; the application
must accept replay risk for the first request.
Connection lifecycle
quic_connect returns a ConnId immediately. The connection isn't
established yet ; you'll see Event::Connected when the handshake finishes.
Sending data before Connected is allowed (it queues), but the data sits
in the handshake state machine until 1-RTT keys are derived. With 0-RTT
enabled and a valid session ticket, the data goes immediately.
Event::Disconnected { error_code, reason } covers both clean close
(error_code: 0) and protocol errors. Inspect error_code to distinguish.
What you'd add for a long-lived client
Reconnection logic with exponential backoff on Disconnected.
Application-level keepalive if your peer expects it.
Connection migration handling (Event::PathMigration) if you care
about which path is active.
Idle timeout via QuicConnectConfig::idle_timeout(Duration) so dead
peers get cleaned up.
Recipe : HTTP server
🚧 Design preview — gated on step 179. Async
wrapper on step 181 (zero-io-async).
HTTP/1.1 + HTTP/2 server. GET + POST, JSON response, file serving via
body_file (zero-copy with splice on Linux).
use std::time::Duration;
use zero_io::{Io, Config, Event};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
io.http_listen("0.0.0.0:8080".parse().unwrap())?;
println!("http on 0.0.0.0:8080");
loop {
io.poll(Duration::from_millis(10))?;
while let Some(ev) = io.next_event() {
if let Event::HttpRequest { conn, request_id, method, path, .. } = ev {
let resp = io.respond(conn, request_id);
match (method, path) {
("GET", "/health") => {
resp.status(200).body_inline(b"ok").send()?;
}
("GET", "/api/info") => {
let json = br#"{"name":"zero-io","ok":true}"#;
resp.status(200)
.header("content-type", "application/json")
.body_inline(json)
.send()?;
}
("GET", path) if path.starts_with("/static/") => {
let fs_path = format!("./public{}", &path[7..]);
resp.status(200).body_file(&fs_path)?.send()?;
}
_ => {
resp.status(404).body_inline(b"not found").send()?;
}
}
}
}
}
}
zero_io_async::http is hyper-shaped — Request / Response /
Service ergonomics, with the shard handling the wire :
use zero_io_async::http::{HttpListener, Request, Response, Body, StatusCode};
use std::convert::Infallible;
#[tokio::main]
async fn main() -> std::io::Result<()> {
let listener = HttpListener::bind("0.0.0.0:8080").await?;
println!("http on 0.0.0.0:8080");
listener.serve(|req: Request<Body>| async move {
let resp = match (req.method().as_str(), req.uri().path()) {
("GET", "/health") => Response::new(Body::from("ok")),
("GET", "/api/info") => Response::builder()
.header("content-type", "application/json")
.body(Body::from(r#"{"name":"zero-io","ok":true}"#))
.unwrap(),
("GET", p) if p.starts_with("/static/") => {
Response::new(Body::from_file(format!("./public{}", &p[7..])).await?)
}
_ => Response::builder().status(StatusCode::NOT_FOUND)
.body(Body::from("not found")).unwrap(),
};
Ok::<_, Infallible>(resp)
}).await
}
Body::from_file uses the same splice zero-copy path as the native
view's body_file underneath, so you don't lose the kernel-offload
optimisation by going async. For richer routing, layer in
zero-rest on top.
Each body_* call appends a body fragment. body_inline(&[u8]) copies (one
memcpy). body_buffer(SendBuffer) is zero-copy — you owned a pool slot,
ResponseBuilder takes ownership. body_file(path) mmaps the file (via
OpenFileCache) and uses splice(file, socket) if kTLS is enabled — zero
syscall, zero copy.
POST with body
Event::HttpRequest { conn, request_id, method: "POST", path: "/api/upload", body, .. } => {
// body is HttpRequestBody — small inline (< body_inline_threshold) is &[u8],
// large body delivers via Event::HttpBodyChunk events afterward.
match body {
HttpRequestBody::Inline(bytes) => process_inline(bytes),
HttpRequestBody::Streaming => {
// store request_id, accumulate Event::HttpBodyChunk
pending_uploads.insert(request_id, Vec::new());
}
}
}
Event::HttpBodyChunk { request_id, data, end } => {
if let Some(buf) = pending_uploads.get_mut(&request_id) {
buf.extend_from_slice(data);
if end {
let full = pending_uploads.remove(&request_id).unwrap();
process_full_body(full);
}
}
}
The threshold (body_inline_threshold, default 8 KiB) decides whether a
body is delivered inline or streamed. Streaming uses zero-copy through pool
slots ; you handle the chunks.
HTTP/2
By default, http_listen accepts both HTTP/1.1 and HTTP/2 (h2c upgrade or
direct h2 over TLS via ALPN). HTTP/2 multiplexing means one connection
carries many concurrent streams ; request_id is the stream identifier
internally, you don't normally care.
With linux-ktls feature enabled, the TLS handshake completes and then
encryption offloads to the kernel. body_file + splice becomes
literal-zero-copy (file → kernel encrypts → NIC) without ever touching
userspace memory.
The async client shares the same HttpPool infrastructure as the
native view — connection reuse, Happy Eyeballs, ALPN, h3
auto-discovery — but exposes an await-shaped surface.
POST with JSON
let body = serde_json::to_vec(&payload)?;
pool.post("https://api.example.com/upload")?
.header("content-type", "application/json")
.body(&body)
.send(&mut io)?;
body(&[u8]) copies into a pool slot. For large bodies, use the streaming
variant (body_stream(impl Read)) which feeds chunks as the connection
drains.
What HttpPool does
Connection reuse — keeps idle connections per (scheme, host, port)
in a FixedMap, LIFO checkout. Connection establishment cost (DNS, TCP,
TLS) is paid once per host.
Happy Eyeballs (RFC 8305) — issues IPv6 + IPv4 connect attempts in
parallel with a 250 ms IPv4 head start, picks whichever wins. Behavior
configurable via HttpPoolConfig.
TLS session resumption — keeps an in-memory ticket cache. On
reconnect to the same host, 0-RTT if the server supports it.
HTTP/2 multiplexing — one TCP+TLS connection carries many concurrent
HTTP requests. pool.get(...) may go on an existing h2 stream rather
than opening a new connection.
HTTP/3 (QUIC) auto-discovery — when Alt-Svc: h3=":443" arrives in
a response header, the pool opens a QUIC connection in parallel for
next-time-this-host requests.
Streaming response body
Event::HttpResponse { request_id, status, body, body_stream } => {
if let Some(stream_id) = body_stream {
// body is small inline ; the rest will arrive via HttpBodyChunk
process_inline(body);
// accumulate further chunks
} else {
process_inline(body); // entire body fit in body
}
}
Event::HttpBodyChunk { request_id, data, end } => {
process_chunk(request_id, data);
if end {
finalize(request_id);
}
}
Same body delivery model as the server side. The body_inline_threshold
config knob controls inline vs streaming.
Speculative pre-connect
pool.preconnect("https://api.example.com")?;
Spins up DNS, TCP, TLS for that host without sending a request. When the
real request comes, it's nearly instant. Use sparingly — wastes resources
if the request never happens.
Retries on connection failure, 5xx, idempotent methods only by default.
Configure for non-idempotent if you understand the replay risk.
TLS ALPN
The pool advertises ["h3", "h2", "http/1.1"] by default ; the server
picks. HttpPoolConfig::alpn(&["h2", "http/1.1"]) to restrict.
Recipe : WebSocket
🚧 Design preview — gated on step 179b. Async wrapper
on step 181 (zero-io-async).
RFC 6455 framing — Text / Binary / Ping / Pong / Close. Auto-Pong,
fragmentation reassembly, permessage-deflate (RFC 7692), TLS / kTLS,
HTTP/1.1 Upgrade or HTTP/2 (RFC 8441). Client and server, native or async.
Echo server, single shard, no tokio :
use std::time::Duration;
use zero_io::{Io, Config, Event, MessageKind};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
io.ws_listen("0.0.0.0:9001".parse().unwrap())?;
println!("ws on 0.0.0.0:9001");
loop {
io.poll(Duration::from_millis(10))?;
while let Some(ev) = io.next_event() {
match ev {
Event::Connected { conn, .. } => {
println!("ws conn {conn:?}");
}
Event::StreamFrame { conn, kind, data, .. } => {
if matches!(kind, MessageKind::Text | MessageKind::Binary) {
let mut buf = io.send_buffer(data.len())?;
buf.write(data);
io.ws_send(conn, kind, buf)?;
}
}
Event::Disconnected { conn, .. } => {
println!("ws conn {conn:?} closed");
}
_ => {}
}
}
}
}
ws_listen accepts plain TCP and runs the HTTP/1.1 Upgrade dance
itself — applications never see the handshake bytes, only Event:: Connected once the WebSocket is live. For TLS, see the TLS variant
section below.
use zero_io_async::ws::{WsListener, Message};
use futures_util::{SinkExt, StreamExt};
#[tokio::main]
async fn main() -> std::io::Result<()> {
let listener = WsListener::bind("0.0.0.0:9001").await?;
println!("ws on 0.0.0.0:9001");
while let Some((mut sock, peer)) = listener.accept().await.transpose()? {
tokio::spawn(async move {
while let Some(msg) = sock.next().await {
match msg {
Ok(Message::Text(s)) => sock.send(Message::Text(s)).await?,
Ok(Message::Binary(b)) => sock.send(Message::Binary(b)).await?,
Ok(Message::Close(_)) => break,
Ok(_) => {} // Ping/Pong auto
Err(_) => break,
}
}
println!("closed {peer}");
Ok::<_, std::io::Error>(())
});
}
Ok(())
}
WsListener::accept returns (WsStream, SocketAddr). WsStream is
Stream<Message> + Sink<Message>, so anything written for
tokio-tungstenite ports almost directly. Underneath, the shard runs
the same poll-loop the native view uses.
use std::time::Duration;
use zero_io::{Io, Config, Event, MessageKind};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
let conn = io.ws_connect("wss://echo.websocket.events/")?;
loop {
io.poll(Duration::from_millis(50))?;
while let Some(ev) = io.next_event() {
match ev {
Event::Connected { conn: c, .. } if c == conn => {
let mut buf = io.send_buffer(5)?;
buf.write(b"hello");
io.ws_send(conn, MessageKind::Text, buf)?;
}
Event::StreamFrame { conn: c, kind, data, .. } if c == conn => {
if matches!(kind, MessageKind::Text | MessageKind::Binary) {
println!("recv: {}", std::str::from_utf8(data).unwrap_or("(bin)"));
let _ = io.close(conn);
}
}
Event::Disconnected { conn: c, .. } if c == conn => return Ok(()),
_ => {}
}
}
}
}
ws_connect accepts a URL — ws:// plain or wss:// TLS, picked
from the scheme. The TLS handshake is automatic ; on wss://,
Event::Connected fires only after both the TLS handshake and the
WebSocket upgrade have completed.
use zero_io_async::ws::{connect, Message};
use futures_util::{SinkExt, StreamExt};
#[tokio::main]
async fn main() -> std::io::Result<()> {
let (mut sock, _resp) = connect("wss://echo.websocket.events/").await?;
sock.send(Message::Text("hello".into())).await?;
if let Some(Ok(msg)) = sock.next().await {
println!("recv: {msg:?}");
}
sock.close(None).await?;
Ok(())
}
Same tokio-tungstenite-shaped connect returning a (WsStream, Response) pair. _resp carries the HTTP/1.1 101 Switching Protocols
headers if you need them (subprotocol, extensions). Drop-in import
swap from tokio_tungstenite::connect_async.
TLS variant (wss://)
// native server
io.ws_listen_with(
WsListenConfig::with_tls("0.0.0.0:443".parse().unwrap(), &cert, &key)
)?;
// async server
let listener = WsListener::bind_tls("0.0.0.0:443", &cert, &key).await?;
Clients use wss:// in the URL — TLS is automatic. With linux-ktls,
post-handshake encryption offloads to the kernel : the WebSocket frame
loop runs over already-encrypted bytes without paying the CPU cost.
Fragmentation
WebSocket allows splitting a logical message across multiple frames
(continuation frames). The runtime reassembles automatically — Event:: StreamFrame and the async Stream<Message> only fire on complete
messages. To opt out and get raw frames (advanced), WsListenConfig:: raw_frames(true) / WsConnectConfig::raw_frames(true).
Compression
permessage-deflate (RFC 7692) is enabled when the websocket-deflate
Cargo feature is on. Negotiated automatically during the WebSocket
handshake ; transparent to the application.
[dependencies]
zero-io = { version = "...", features = ["websocket", "websocket-deflate"] }
Ping / Pong
Auto-Pong is on by default — incoming Ping frames are answered with Pong
without delivering an event. Disable with auto_pong(false) (in the
listen / connect config) if you want to handle Ping events yourself.
ws_ping(conn) (native) or sock.send(Message::Ping(payload)) (async)
to send a Ping ; the matching Pong arrives as MessageKind::Pong /
Message::Pong.
Backpressure
WebSocket connections live across many frames. If a peer is slow and
your TX queue grows, the native view sees Event::PoolPressure and the
async view's sock.send returns Err(SendQueueFull). Standard
cascade — shed load by closing slow connections, or buffer in your own
application queue.
Browser compatibility
Standard RFC 6455 + RFC 7692. All modern browsers work. The runtime
negotiates HTTP/1.1 Upgrade by default ; for HTTP/2 WebSockets (RFC
8441), WsListenConfig::h2_websocket(true).
Multi-shard
Same pattern as HTTP : IoCluster with N shards, cluster.ws_listen (...), SO_REUSEPORT distributes incoming TCP. WebSocket connections
are sticky to one shard for life (no migration) ; the right model for
long-lived stateful connections.
Recipe : REST API (zero-rest)
🚧 Design preview — zero-rest is a separate crate (step 180)
built on top of zero-io's HTTP shard. JSON / path params /
middleware ; keeps the zero-alloc story.
A small REST framework — Router with path patterns, type-safe
extractors for query / body / path params, JSON in/out, optional
caching middleware. Ergonomically axum-shaped, structurally
zero-alloc-friendly.
If you need 0 alloc on a specific endpoint — match path strings inline
in the HTTP handler.
use std::time::Duration;
use zero_io::{Io, Config, Event};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
io.http_listen("0.0.0.0:8080".parse().unwrap())?;
loop {
io.poll(Duration::from_millis(10))?;
while let Some(ev) = io.next_event() {
if let Event::HttpRequest { conn, request_id, method, path, .. } = ev {
let resp = io.respond(conn, request_id);
match (method, path) {
("GET", p) if p.starts_with("/users/") => {
let id_str = &p[7..];
// parse u64 from id_str, build response inline
resp.status(200)
.header("content-type", "application/json")
.body_inline(br#"{"id":1,"name":"..."}"#)
.send()?;
}
("GET", "/users") => {
resp.status(200)
.header("content-type", "application/json")
.body_inline(b"[]")
.send()?;
}
_ => { resp.status(404).body_inline(b"not found").send()?; }
}
}
}
}
}
This is the bottom-of-the-stack form. Fine for 5-10 routes ;
unmaintainable past that. Use zero-rest for real API surfaces.
zero-rest is ~640 B alloc per request (the route table + extractor
overhead). For the genuine hot path, drop to native HTTP. For
everything else this is the recommended shape.
Middleware
zero-rest exposes a tower-compat middleware chain with native
zero-alloc adapters for the common cases :
CORS — Cors::permissive() or Cors::with_origins(&[...]).
Auth — Auth::bearer(|token| async { ... }). Header-only, no
body parsing on the auth path.
Tracing — Tracing::new() emits structured spans per request.
Compression — automatic via OutputFilter::Zstd / Gzip based
on Accept-Encoding.
Cache — CacheMiddleware::new(CacheStore::file("./cache")) for
proxy-cache use. Stale-while-revalidate, cache-lock for thundering
herd protection.
Path patterns
zero-rest uses matchit (the same router axum uses). Patterns :
/users — literal
/users/{id} — path param
/users/{id}/posts/{post_id} — multiple
/files/*path — catch-all (rest of path captured)
Path matching is O(log n) on the route count and 0 alloc per request.
When zero-rest, when raw HTTP
Use zero-rest when : you have ≥ 10 routes, you want async handlers,
you want middleware, you want familiar JSON ergonomics. 95% of REST APIs.
Use raw HTTP when : your path is genuinely the bottleneck (1M+ req/s
on one route, every nanosecond counts), you don't want any allocation
not under your direct control. Niche.
Recipe : DNS resolver
🚧 Design preview — gated on step 180b. Built into
zero-io core, no separate crate. UDP + TCP fallback for large responses,
caching, IPv4 + IPv6.
A non-blocking, cache-coherent DNS resolver. Used internally by the HTTP
client for hostname resolution (Happy Eyeballs RFC 8305) ; available
directly when you want explicit control over resolution.
Event-driven shape — for embedding the resolver inside the same shard as
your other I/O. No tokio.
use std::time::Duration;
use zero_io::{Io, Config, Event};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
let resolver = io.dns_resolver()?;
let lookup_id = resolver.lookup_a("api.example.com")?;
let srv_id = resolver.lookup_srv("_xmpp-client._tcp.example.com")?;
loop {
io.poll(Duration::from_millis(100))?;
while let Some(ev) = io.next_event() {
match ev {
Event::DnsResolved { id, result } if id == lookup_id => {
match result {
Ok(addrs) => {
for addr in addrs { println!("a: {addr}"); }
}
Err(e) => eprintln!("dns err: {e}"),
}
}
Event::DnsResolved { id, result } if id == srv_id => {
// ...
}
_ => {}
}
}
}
}
The resolver is owned by the shard ; lookup IDs correlate requests
to Event::DnsResolved results. No threads spawned, no tokio
required ; UDP queries go through the same io_uring as everything
else.
use zero_io::dns;
#[tokio::main]
async fn main() -> std::io::Result<()> {
let ips = dns::resolve("example.com").await?;
for ip in ips {
println!("{ip}");
}
let v6_only = dns::resolve_v6("example.com").await?;
let srvs = dns::resolve_srv("_xmpp-client._tcp.example.com").await?;
for srv in srvs {
println!("{}:{} priority={} weight={}",
srv.target, srv.port, srv.priority, srv.weight);
}
Ok(())
}
The async helpers wrap a per-thread resolver pool. Cache TTL respected
per-record. /etc/resolv.conf watched for changes.
The HTTP client uses dns::resolve_for_dial(hostname), which returns
both A and AAAA records and starts both connection attempts in
parallel with a 250 ms IPv4 head start. The first to complete TLS
handshake wins ; the other is canceled.
You don't normally call this directly — it's wired into
http::Client and quic_connect. But it's available for custom
TCP/UDP clients.
DoH / DoT (DNS over HTTPS / TLS)
Beyond plain UDP/TCP DNS, zero-io supports :
DoH (https://...) — over zero-io's HTTP/2 client. Use when
the network blocks port 53.
DoT (dot://...:853) — over the TCP shard with TLS.
Both surfaces are configured the same way as plain DNS, just with
different scheme :
The cache is per-shard, shared across resolutions. TTL respected per
record (capped at DnsConfig.max_ttl, default 1h). Negative caching
(NXDOMAIN) for a brief window (DnsConfig.negative_ttl, default 60s)
to avoid hammering authoritative servers on broken hostnames.
IPv6
Auto-prefer-IPv6 via the ipv6_first(true) config. Match glibc
default with false (the install-default).
zero-redis speaks the Redis protocol over zero-io's TCP shard. The
async client surface is redis-rs-like for familiarity ; the
zero-alloc story is preserved on the hot path (parsing RESP responses
into pool slots, no Vec allocation for typical responses).
Pipelining (multiple commands in one round trip) and pub/sub
subscription.
use zero_redis::{Client, Pipeline};
#[tokio::main]
async fn main() -> Result<(), zero_redis::RedisError> {
let client = Client::connect("redis://127.0.0.1:6379").await?;
// Pipeline — 3 commands, 1 round trip
let mut pipe = Pipeline::new();
pipe.cmd("MULTI");
pipe.cmd("SET").arg("a").arg("1");
pipe.cmd("SET").arg("b").arg("2");
pipe.cmd("EXEC");
let results = client.pipeline(pipe).await?;
println!("{} replies", results.len());
// Pub/sub
let mut sub = client.subscribe(&["events", "alerts"]).await?;
tokio::spawn(async move {
while let Some(msg) = sub.next().await {
println!("{} : {}", msg.channel(), msg.payload_str());
}
});
// Publish from another connection (separate client recommended for pub/sub)
let pub_client = Client::connect("redis://127.0.0.1:6379").await?;
pub_client.publish("events", "user signed in").await?;
Ok(())
}
Pub/sub holds the connection in subscribe-mode, so it's typically a
dedicated client. The crate enforces this — calling normal commands
on a subscribed client returns Err(SubscribeMode).
use zero_redis::{Client, RedisError};
#[tokio::main]
async fn main() -> Result<(), RedisError> {
let client = Client::connect("redis://127.0.0.1:6379").await?;
client.set("counter", 0).await?;
let n: i64 = client.incr("counter").await?;
let v: String = client.get("counter").await?;
client.expire("counter", 60).await?;
Ok(())
}
Client holds a single connection ; ClientPool for connection
pooling (default 8 connections per pool). Parsing of RESP3 responses
zero-allocates for primitive types.
TLS
rediss:// (note the double s) for TLS. Client::connect("rediss:// host:6380") does the TLS handshake on connect. With linux-ktls,
post-handshake encryption offloads to the kernel.
Cluster mode
For Redis Cluster :
use zero_redis::cluster::ClusterClient;
let client = ClusterClient::connect(&[
"redis://node1:6379",
"redis://node2:6379",
"redis://node3:6379",
]).await?;
Slot map fetched at connect, refreshed on MOVED / ASK responses.
Pipelining across slots handled internally (commands routed to the
right node). Pub/sub in cluster mode uses SSUBSCRIBE (sharded
pub/sub) to scale.
Comparison vs redis-rs
redis-rs
zero-redis
Runtime
tokio (async) or sync
zero-io shard via async bridge, OR sync poll
Allocs / GET hot path
several
aim for 0 (return type permitting)
Pipeline
yes
yes, same shape
Pub/sub
yes
yes
Cluster
yes
yes
Sentinel
yes
not yet (post-1.0)
A redis-rs codebase migrates to zero-redis mostly via import
changes. The command builder API is API-compatible.
In-process state : if you don't need cross-process sharing, just
use a HashMap. Network overhead is a constant cost ; sometimes the
right answer is "don't go through the network."
MQTT is the ubiquitous IoT pub/sub protocol. Lightweight binary framing
on top of TCP (or QUIC for MQTT 5 + QUIC binding). zero-mqtt ships a
client + a broker, both running on zero-io's TCP shard.
A minimal MQTT broker — accept connections, handle SUBSCRIBE / PUBLISH
/ PINGREQ, fan out to subscribers via the trie-based topic index.
The broker uses zero-io's TCP shard underneath. 100k+ concurrent
clients per shard is realistic on modern hardware. The topic trie is
single-threaded per shard ; cross-shard messages relay via the same
SPSC rings as the rest of zero-io.
Async client that subscribes + publishes. Auto-reconnect with exponential
backoff. QoS 1 acknowledgements handled internally.
The mqtt:// scheme is plain TCP ; mqtts:// is TLS ; mqtt+ws:// is
WebSocket-tunneled. All three handled by the same Client interface.
MQTT 5.0 features
zero-mqtt is MQTT 5 capable :
User properties (key/value metadata on PUBLISH/SUBSCRIBE).
Topic aliases (compress topic strings on the wire).
Reason codes (granular error reporting).
Session expiry intervals (precise control over persistent sessions).
Shared subscriptions ($share/group/topic for load-balancing among N
consumers).
Negotiate down to 3.1.1 if the peer doesn't speak 5 — controlled at
connect time.
Persistent sessions
ClientConfig::clean_start(false) + a stable client_id makes the
broker keep your subscriptions and pending QoS 1/2 messages for the
session expiry interval. On reconnect, you skip re-subscribing ; the
broker replays missed messages.
The broker stores session state in-memory by default. For
disk-backed sessions (survival across broker restart), enable
BrokerConfig::session_store(SessionStore::Disk("./mqtt-sessions")).
Redis pub/sub : if you already have Redis, simpler. No
persistent sessions across reconnect though.
NATS : higher throughput per-publisher, simpler protocol, no
persistent state by default. Different ergonomics.
WebSocket : when the client is a browser. Or use mqtt+ws://
to reach MQTT brokers from browsers via WebSocket tunneling.
Recipe : TLS cert hot-reload
🚧 Design preview — gated on step 182. Linux-only :
uses kTLS for zero-disruption rotation. macOS/Windows fall back to
graceful close + reaccept.
Rotate TLS certificates without dropping in-flight connections.
Operationally critical when your CA shortens cert lifetimes (Let's
Encrypt = 90 days, ACME-shortlived = 6 hours) or when you're rolling out
an emergency revocation.
Native control — your application explicitly triggers reload at the
right moment (e.g., from a SIGHUP handler or an ops-API call).
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::time::Duration;
use zero_io::{Io, Config};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
let endpoint = io.http_listen_with(
zero_io::HttpListenConfig::with_tls(
"0.0.0.0:443".parse().unwrap(),
&std::fs::read("cert.pem")?,
&std::fs::read("key.pem")?,
)
)?;
// Trigger reload on SIGHUP
let reload = Arc::new(AtomicBool::new(false));
let r = reload.clone();
ctrlc::set_handler(move || r.store(true, Ordering::Release))?;
loop {
io.poll(Duration::from_millis(100))?;
while let Some(_ev) = io.next_event() { /* handle */ }
if reload.swap(false, Ordering::AcqRel) {
let new_cert = std::fs::read("cert.pem")?;
let new_key = std::fs::read("key.pem")?;
io.cert_hot_reload(endpoint, &new_cert, &new_key)?;
println!("cert reloaded");
}
}
}
cert_hot_reload does the atomic swap of the listener's rustls
config. Connections that completed handshake with the old cert keep
their session ; new connections (and resumed sessions via tickets
issued post-reload) use the new cert.
Watch a directory ; when cert.pem or key.pem changes, atomically
swap the in-memory cert/key. New connections use the new cert ;
in-flight connections finish with the old one.
The watcher uses inotify on Linux (fanotify if CAP_SYS_ADMIN),
debounces multi-write atomic-rename patterns (mv -f tmp cert.pem),
verifies the new cert parses and matches the key before swapping.
On error, keeps the old cert active.
What's swapped, what's not
Swapped : leaf cert + intermediate chain + private key. Used for
every new TLS handshake (full or 0-RTT) after the swap.
NOT swapped : in-flight TLS handshakes (they complete with the cert
they started with), session tickets issued before the swap (they remain
valid for their original lifetime, which is why you set short ticket
lifetimes — typically 1h — on rotating certs).
kTLS interaction
With the linux-ktls Cargo feature, after handshake the encryption
offloads to the kernel. cert_hot_reload updates the userspace rustls
config that's used for the handshake ; the kernel's per-socket session
keys are unaffected. Long-lived connections continue with their
established session keys until they negotiate new ones (usually via
KeyUpdate frame, which kTLS handles transparently up to a count limit
configurable in TlsServerConfig).
Operational reload triggers
SIGHUP — the example above. Convention for "config reload".
Ops API — via the charting-status CLI : cert-reload --endpoint 443. UDS-based, audit-logged.
File watcher — inotify on the cert directory. Atomic-rename
detection : ignore single WRITE events, react only to MOVED_TO (the
pattern certbot / cert-manager emit).
Error handling
If the new cert fails to parse, or the key doesn't match the cert,
cert_hot_reload returns Err(IoError::Tls) and keeps the old cert
active. Log + alert ; never panic on a cert reload failure.
A 4-shard QUIC server with CPU pinning, NUMA awareness, and graceful
shutdown. The full pattern for production multi-shard.
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::Duration;
use zero_io::{IoCluster, ClusterConfig, RoutingStrategy, ExpectedProtocols, Event};
fn main() -> std::io::Result<()> {
let cert = std::fs::read("cert.pem")?;
let key = std::fs::read("key.pem")?;
let cluster = IoCluster::new(ClusterConfig {
shard_count: 4,
routing: RoutingStrategy::ReusePortCbpf,
cpu_affinity: true, // pin shards to cores 0..N
expected_protocols: ExpectedProtocols::default().with_quic(),
..Default::default()
})?;
cluster.quic_listen("0.0.0.0:4433".parse().unwrap(), &cert, &key)?;
let shutdown = Arc::new(AtomicBool::new(false));
{
let s = shutdown.clone();
ctrlc::set_handler(move || s.store(true, Ordering::Release))?;
}
let handles: Vec<_> = cluster.into_shards().enumerate()
.map(|(idx, mut shard)| {
let stop = shutdown.clone();
std::thread::Builder::new()
.name(format!("zero-io-shard-{idx}"))
.spawn(move || run_shard(idx, &mut shard, stop))
.unwrap()
}).collect();
for h in handles { h.join().unwrap()?; }
Ok(())
}
fn run_shard(idx: usize, shard: &mut zero_io::ShardIo, stop: Arc<AtomicBool>)
-> std::io::Result<()>
{
println!("shard {idx} ready");
while !stop.load(Ordering::Acquire) {
shard.poll(Duration::from_millis(50))?;
while let Some(ev) = shard.next_event() {
match ev {
Event::Datagram { conn, data } => {
let mut buf = shard.send_buffer(data.len())?;
buf.write(data);
shard.send_datagram(conn, buf)?;
}
_ => {}
}
}
}
// graceful drain
println!("shard {idx} draining");
let conns: Vec<_> = shard.connections().collect();
for conn in conns {
let _ = shard.close_graceful(conn, Duration::from_secs(3));
}
let drain_deadline = std::time::Instant::now() + Duration::from_secs(5);
while shard.conn_count() > 0 && std::time::Instant::now() < drain_deadline {
shard.poll(Duration::from_millis(10))?;
while let Some(_) = shard.next_event() {}
}
println!("shard {idx} stopped ({} conns left)", shard.conn_count());
Ok(())
}
What's load-bearing
cpu_affinity: true — pthread_setaffinity_np pins each shard to a
core. The kernel scheduler doesn't migrate threads between cores. Cache
hits stay hot.
thread::Builder::new().name(...) — named threads show up in
top -H / perf top / your APM tooling as zero-io-shard-N. Trivially
helpful in production.
The shutdown protocol — sets AtomicBool from signal handler ; each
shard checks it on its poll() loop boundary. Then close_graceful per
conn, drain poll() calls until done or deadline, exit thread. No
abrupt drops.
NUMA-aware variant
let cluster = IoCluster::new(ClusterConfig {
shard_count: 8, // 4 cores per NUMA node × 2 nodes
routing: RoutingStrategy::ReusePortCbpf,
cpu_affinity: true,
numa_aware: true, // pool memory placed near pinned core
..Default::default()
})?;
numa_aware: true enables first-touch allocation : the shard's pools are
allocated from inside the shard's thread, after pinning, so the kernel
places the pages on the correct NUMA node. Cross-node memory access (slow
on big multi-socket boxes) is avoided.
4 (or N) zero-io-shard-* threads each at ~25% CPU under load.
Userspace time dominant ; softirq time low (kernel network stack stays
out of the way thanks to io_uring).
No __schedule / context switches if SQPOLL is on.
If you see lock contention in parking_lot, tokio, or anywhere across
shards, that's a bug — the design is lock-free across shards by
construction.
Sizing
shard_count = num_cpus by default. For network-bound workloads,
this is right. For CPU-bound workloads (heavy compute in the handler),
you might leave a core for OS work and use num_cpus - 1.
Power of 2 is required by some BPF routing programs (mask-based
shard ID). Cluster will reject non-power-of-2 with IoError::Config(...).
Cross-shard channels for general data. Connections are sticky to one
shard. The shard owns its connection state. No coordination needed.
Locks anywhere in your handler. The shard is single-threaded.
Tokio runtime unless you have async work — see
Async integration for the bridge pattern.
Recipe : TCP proxy with splice
🚧 Design preview — gated on step 176. splice zero-
copy is Linux-only (Tier 1). Async wrapper on
step 181.
Forward all bytes from an incoming TCP connection to an upstream, both
directions, never copy through userspace. The kernel-side splice syscall
moves bytes between socket FDs without touching userspace memory.
use std::collections::HashMap;
use std::time::Duration;
use zero_io::{Io, Config, ConnId, Event};
fn main() -> std::io::Result<()> {
let upstream: std::net::SocketAddr = "127.0.0.1:8080".parse().unwrap();
let mut io = Io::new(Config::default())?;
io.tcp_listen("0.0.0.0:9000".parse().unwrap())?;
let mut pairs: HashMap<ConnId, ConnId> = HashMap::new(); // accept ↔ upstream
loop {
io.poll(Duration::from_millis(10))?;
while let Some(ev) = io.next_event() {
match ev {
Event::Connected { conn: client, .. } => {
let server = io.tcp_connect(upstream)?;
io.splice(client, server)?; // bidirectional zero-copy relay
pairs.insert(client, server);
pairs.insert(server, client);
}
Event::Disconnected { conn, .. } => {
if let Some(peer) = pairs.remove(&conn) {
let _ = io.close(peer);
pairs.remove(&peer);
}
}
_ => {}
}
}
}
}
use zero_io_async::net::{TcpListener, TcpStream};
#[tokio::main]
async fn main() -> std::io::Result<()> {
let upstream: std::net::SocketAddr = "127.0.0.1:8080".parse().unwrap();
let listener = TcpListener::bind("0.0.0.0:9000").await?;
loop {
let (client, _peer) = listener.accept().await?;
tokio::spawn(async move {
let server = TcpStream::connect(upstream).await?;
// splice on Linux ; userspace zero-copy fallback elsewhere.
zero_io_async::net::splice_bidi(client, server).await?;
Ok::<_, std::io::Error>(())
});
}
}
zero_io_async::net::splice_bidi resolves to the same io.splice SQE
the native view uses on Linux, and to the userspace zero-copy relay on
other tiers. The future completes when both directions close.
A splice(fd_in, NULL, fd_pipe, NULL, len, SPLICE_F_MOVE) followed by
splice(fd_pipe, NULL, fd_out, NULL, len, SPLICE_F_MOVE) moves bytes
between two sockets via a kernel pipe. The bytes never cross userspace
memory ; the CPU just twiddles pointers in the kernel page cache.
io.splice(client, server) sets up both directions in one call, with
the runtime managing the pipe buffer pool and submitting the right
io_uring SQEs for the relay. You don't see any data — there's no
StreamFrame event for spliced connections.
When to use it
Pure proxying — you don't need to inspect the data. Load balancer,
TLS terminator with kTLS, generic L4 forward.
Throughput-critical — splice is the fastest possible way to forward
TCP bytes on Linux. We're talking line-rate 100 Gbps with zero CPU on
the data path.
If you need to inspect bytes (logging, header rewriting, app-level
firewall), splice isn't the answer — read the StreamFrame events,
process them, and stream_write to the upstream connection. That's the
2-shard async-bridge pattern, not splice.
With linux-ktls, you accept TLS on the client side, terminate it
in-kernel, splice the cleartext to an upstream plaintext socket. Zero copy,
zero userspace involvement, line-rate TLS termination.
io.tcp_listen_with(
TcpListenConfig::with_tls("0.0.0.0:443".parse().unwrap(), &cert, &key)
.ktls(true)
)?;
// splice() automatically uses kTLS on the encrypted side
Backpressure
When the upstream is slow, the kernel pipe buffer fills up. splice reads
from the client into the pipe ; the pipe is full ; client TCP pushes back
on its peer (window goes to zero). Backpressure flows naturally end-to-end.
Cross-platform
splice is Linux-only. On Tier 2 / Tier 3, io.splice(...) returns
IoError::NotSupportedOnPlatform { platform }. The fallback for those
platforms : userspace relay using zero-copy pool slots — almost as fast in
practice, but technically one userspace memcpy on each direction.
The relay API exists on all tiers ; it transparently uses splice on
Linux when available, falls back otherwise.
Multi-shard variant
Each shard has its own listener and its own connections to upstream. SO_
REUSEPORT distributes incoming connections ; the per-shard upstream
connection pool keeps connection establishment cost low.
Each shard's upstream pool is independent. Connections from the same
client to your proxy land on the same shard (4-tuple hash) ; the upstream
mapping is per-shard.
Recipe : WebTransport
🚧 Design preview — gated on step 177. WebTransport
rides on QUIC + HTTP/3, so QUIC server is also
required.
WebTransport (W3C draft) lets browsers open bidirectional streams +
datagrams over a single QUIC connection. The native browser API is a clean
alternative to WebSocket for low-latency apps. From Rust you can run both
sides ; from a browser you connect via the JS WebTransport constructor.
Native poll loop — wt_listen accepts H3 CONNECT requests ;
Event::SessionReady signals a new WebTransport session.
use std::time::Duration;
use zero_io::{Io, Config, Event};
fn main() -> std::io::Result<()> {
let cert = std::fs::read("cert.pem")?;
let key = std::fs::read("key.pem")?;
let mut io = Io::new(Config::default())?;
io.wt_listen("0.0.0.0:4433".parse().unwrap(), &cert, &key)?;
println!("wt on 0.0.0.0:4433");
loop {
io.poll(Duration::from_millis(10))?;
while let Some(ev) = io.next_event() {
match ev {
Event::SessionReady { conn } => {
println!("wt session {conn:?}");
}
Event::Datagram { conn, data } => {
let mut buf = io.send_buffer(data.len())?;
buf.write(data);
io.send_datagram(conn, buf)?;
}
Event::StreamFrame { conn, stream, kind, data } => {
let mut buf = io.send_buffer(data.len())?;
buf.write(data);
let s = io.open_uni_stream(conn)?;
io.stream_write(conn, s, buf)?;
if kind == zero_io::MessageKind::End {
io.stream_shutdown(conn, s)?;
}
}
Event::Disconnected { conn, .. } => {
println!("wt closed {conn:?}");
}
_ => {}
}
}
}
}
wt_listen automatically configures ALPN h3 + the WebTransport
extension SETTINGS frames, accepts CONNECT requests on :authority, and
surfaces sessions as Event::SessionReady.
Server with the async wrapper — task per session, async stream and
datagram handling :
WtSession exposes recv_datagram / send_datagram /
accept_uni_stream / accept_bi_stream / open_uni_stream /
open_bi_stream. Task per session, all I/O is await-shaped.
Native poll-loop client — open a session, send a datagram, receive an
echo :
use std::time::Duration;
use zero_io::{Io, Config, Event};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
let conn = io.wt_connect("https://example.com:4433/echo")?;
loop {
io.poll(Duration::from_millis(50))?;
while let Some(ev) = io.next_event() {
match ev {
Event::SessionReady { conn: c } if c == conn => {
let mut buf = io.send_buffer(5)?;
buf.write(b"hello");
io.send_datagram(conn, buf)?;
}
Event::Datagram { conn: c, data } if c == conn => {
println!("recv: {}", std::str::from_utf8(data).unwrap_or("(bin)"));
let _ = io.close(conn);
}
Event::Disconnected { conn: c, .. } if c == conn => return Ok(()),
_ => {}
}
}
}
}
wt_connect(url) parses the https://host:port/path URL, drives the
QUIC handshake, sends the H3 CONNECT request with :protocol = webtransport,
and surfaces Event::SessionReady once the server has accepted. Bidi /
uni streams via open_bi_stream(conn) / open_uni_stream(conn).
Async client — same shape as the server helper, but for connect :
use zero_io_async::wt;
#[tokio::main]
async fn main() -> std::io::Result<()> {
let mut sess = wt::connect("https://example.com:4433/echo").await?;
sess.send_datagram(b"hello").await?;
if let Some(dgram) = sess.recv_datagram().await? {
println!("recv: {:?}", dgram);
}
sess.close().await?;
Ok(())
}
Mirrors the W3C JS API roughly : connect returns a WtSession, methods
to send/receive datagrams, accept/open streams. Use this when your app
is already a tokio app and the call site is already async.
The W3C WebTransport constructor speaks the same protocol the Rust
client does ; either side can be browser or native.
Browser compatibility
WebTransport ships in Chrome 97+, Edge 97+, Firefox 114+. Safari support
landed in Tech Preview but not stable as of writing. For browsers without
WebTransport, fall back to WebSocket on the same hostname/port via
Alt-Svc, or open both sockets and let the client pick.
When to prefer WT over WS
Browser-side multiplexing : N concurrent streams over one connection.
Datagram support (unreliable, ordered-not-required) — better fit for
real-time tick streams or game state than WebSocket text frames.
Lower head-of-line blocking on bad networks (QUIC streams are
independent ; WS over a single TCP connection blocks everything on
any lost segment).
WS is still better when : broad client compatibility (older browsers,
corporate proxies), or proxying through L7 LBs that don't speak HTTP/3
yet.
Connection migration
Inherits from QUIC. A WT session over a migrated QUIC connection
continues seamlessly — same conn, same shard, same session.
Recipe : gRPC (zero-grpc)
🚧 Design preview — separate crate (step 186), built on zero-io's
HTTP/2 shard. Code generated from .proto via zero-grpc-build.
gRPC = HTTP/2 + protobuf framing. zero-grpc gives you a service trait
generated from your .proto files, runs over zero-io's HTTP/2 shard,
keeps the zero-alloc story for the hot path. Server streaming, client
streaming, bidi streaming — all four of gRPC's call modes.
If you want to talk gRPC without the codegen — for instance, a generic
gRPC proxy that doesn't care about the proto — drop to native HTTP/2.
use std::time::Duration;
use zero_io::{Io, Config, Event, MessageKind};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
io.http_listen_with(
zero_io::HttpListenConfig::new("0.0.0.0:50051".parse().unwrap())
.h2_only(true)
.alpn(&["h2"])
)?;
loop {
io.poll(Duration::from_millis(10))?;
while let Some(ev) = io.next_event() {
// gRPC = POST /<service>/<method>, content-type application/grpc,
// body = (5 byte length-prefix + protobuf)+, trailers = grpc-status
if let Event::HttpRequest { conn, request_id, path, .. } = ev {
handle_grpc_call(&mut io, conn, request_id, path)?;
}
}
}
# Ok(())
}
You'd then : (a) parse the path to identify service/method,
(b) accumulate length-prefixed protobuf frames from the body,
(c) decode using prost directly,
(d) emit response frames with trailers (grpc-status: 0 for OK).
This is the shape zero-grpc itself wraps. Use the codegen 99% of
the time ; this is for proxy / generic-tooling use cases.
Cargo.toml :
[dependencies]
zero-grpc = { version = "...", features = ["server"] }
[build-dependencies]
zero-grpc-build = "..."
use zero_grpc::ServerStream;
include!(concat!(env!("OUT_DIR"), "/chart.rs")); // generated
#[derive(Default)]
struct ChartSvc;
#[zero_grpc::async_trait]
impl chart_server::Chart for ChartSvc {
type StreamStream = ServerStream<Tick>;
async fn stream(&self, req: StreamReq) -> Result<Self::StreamStream, zero_grpc::Status> {
let (tx, rx) = ServerStream::channel(64);
tokio::spawn(async move {
for tick in your_tick_source(&req.symbol).await {
if tx.send(Ok(tick)).await.is_err() { break; }
}
});
Ok(rx)
}
}
#[tokio::main]
async fn main() -> std::io::Result<()> {
zero_grpc::Server::builder()
.add_service(chart_server::ChartServer::new(ChartSvc))
.serve("0.0.0.0:50051".parse().unwrap()).await
}
The generated chart_server::Chart trait makes the wire format
disappear : you write request → response logic, zero-grpc handles the
HTTP/2 framing, varint encoding, length prefixing, status codes.
If you're talking to a generic gRPC service whose proto you don't have
(transparent proxy, observability tooling), drop to native HTTP/2 :
use std::time::Duration;
use zero_io::{Io, Config, Event, MessageKind, HttpRequest};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
let conn = io.http_connect("https://example.com:50051")?;
// Build the gRPC request body : 1-byte compressed flag + 4-byte BE length
// + protobuf payload.
let proto_bytes: Vec<u8> = your_request.encode_to_vec();
let mut body = Vec::with_capacity(5 + proto_bytes.len());
body.push(0); // not compressed
body.extend_from_slice(&(proto_bytes.len() as u32).to_be_bytes());
body.extend_from_slice(&proto_bytes);
let req = HttpRequest::post("/chart.Chart/Stream")
.header("content-type", "application/grpc")
.header("te", "trailers")
.body(&body);
let request_id = io.http_request(conn, req)?;
loop {
io.poll(Duration::from_millis(50))?;
while let Some(ev) = io.next_event() {
match ev {
Event::HttpResponse { id, status, .. } if id == request_id => {
if status != 200 { eprintln!("http {status}"); return Ok(()); }
}
Event::StreamFrame { conn: c, kind: MessageKind::Data, data, .. } if c == conn => {
// Each gRPC frame here is `[1B flag][4B BE len][payload]`.
// Decode with prost (or your own) ; status arrives in trailers.
}
Event::HttpTrailers { id, headers, .. } if id == request_id => {
// grpc-status header, "0" = OK
return Ok(());
}
_ => {}
}
}
}
}
This is the bottom-of-the-stack form. Use codegen (next tab) unless
you genuinely don't want a proto dependency on the client.
The codegen client mirrors the server side — same .proto,
zero-grpc-build emits a *_client::*Client stub :
use zero_grpc::transport::Endpoint;
include!(concat!(env!("OUT_DIR"), "/chart.rs")); // generated
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let channel = Endpoint::from_static("https://example.com:50051")
.connect().await?;
let mut client = chart_client::ChartClient::new(channel);
// Server-streaming RPC
let req = StreamReq { symbol: "BTCUSDT".into() };
let mut stream = client.stream(req).await?.into_inner();
while let Some(tick) = stream.message().await? {
println!("tick {} @ {}", tick.price, tick.ts);
}
Ok(())
}
Endpoint::from_static parses the URL, connect() returns a Channel
(connection pool to the target). ChartClient exposes one method per
RPC in the proto. API parity with tonic — port via import swap.
mTLS via Endpoint::tls_config(...).
Streaming modes
Mode
Wire shape
gRPC trait
Unary
1 request frame, 1 response frame
async fn returning Result<Resp, Status>
Server streaming
1 request, N responses
returns ServerStream<Resp>
Client streaming
N requests, 1 response
takes ClientStream<Req>
Bidi streaming
N ↔ N
takes + returns streams
All four are codegen-supported. Backpressure is per-stream HTTP/2 flow
control ; zero-grpc surfaces that as mpsc channel send returning
Err when the client's window closes.
TLS / mTLS
Standard Server::builder().tls(cert, key) — same code path as plain
HTTP. For mTLS (verifying client certs), tls_with_client_ca(cert, key, client_ca). kTLS offload supported (Linux).
Comparison vs tonic
tonic
zero-grpc
Runtime
tokio
zero-io shard ; tokio for handlers via async bridge
Allocs / unary req hot path
several
aim for 0 in body path
HTTP/2 stack
h2 (vendored or upstream)
vendored h2 from quiche, with our zero-alloc HPACK patch
Source-Specific Multicast (SSM, RFC 4607) lets you join a multicast group
filtered by source — only packets from a specific sender land on your
socket. Useful for : market data feeds (one publisher, many subscribers),
discovery protocols, cluster heartbeat.
use std::net::{IpAddr, SocketAddr};
use std::time::Duration;
use zero_io::{Io, Config, Event, UdpEndpointConfig};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
let group: IpAddr = "232.1.2.3".parse().unwrap();
let source: IpAddr = "198.51.100.42".parse().unwrap();
let bind: SocketAddr = "0.0.0.0:5555".parse().unwrap();
io.udp_bind_with(
UdpEndpointConfig::new(bind)
.multicast_join(group, Some(source))
.so_rcvbuf(8 * 1024 * 1024) // 8 MiB recv buffer for bursts
)?;
println!("listening for {group} from {source} on udp/5555");
loop {
io.poll(Duration::from_millis(100))?;
while let Some(ev) = io.next_event() {
if let Event::UdpRecv { from, data, .. } = ev {
process_multicast_packet(from, data);
}
}
}
}
fn process_multicast_packet(from: SocketAddr, data: &[u8]) {
// ...
}
multicast_join(group, Some(source)) issues IGMPv3 with source filter
on Linux — the kernel rejects packets from any other sender before
they reach your socket. Multiple .multicast_join(...) calls are
additive ; one socket can subscribe to many groups.
use std::net::{IpAddr, SocketAddr};
use std::time::Duration;
use zero_io::{Io, Config, Event, UdpEndpointConfig};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
let group: SocketAddr = "232.1.2.3:5555".parse().unwrap();
let bind: SocketAddr = "0.0.0.0:0".parse().unwrap();
let endpoint = io.udp_bind_with(
UdpEndpointConfig::new(bind)
.multicast_ttl(8) // limit to 8 hops
.multicast_loopback(false) // don't receive own packets
.multicast_iface("eth0") // pick the egress NIC
)?;
println!("publishing to {group}");
let mut tick = 0u64;
loop {
io.poll(Duration::from_millis(50))?;
let payload = format!("tick {tick}");
let mut buf = io.send_buffer(payload.len())?;
buf.write(payload.as_bytes());
io.udp_send(endpoint, group, buf)?;
tick += 1;
}
}
multicast_iface pins the outgoing interface — important on
multi-homed boxes. multicast_ttl bounds the TTL (1 = link-local, 8 =
typical intra-DC, 32 = within a region).
Why SSM specifically (vs. ASM / IGMP v2)
Filtered at kernel layer — the kernel rejects packets from other
sources before they hit your socket. Less userspace work.
No PIM rendezvous point — IGMPv3 SSM joins are direct ; the network
doesn't need a rendezvous point. Operationally simpler.
Range : 232.0.0.0/8 (IPv4) and ff3x::/32 (IPv6) reserved for SSM.
Use any address in that range ; coordinate with your network team about
routing.
The Event::UdpRecv then carries pkt_info: Option<UdpPktInfo { dst_addr, .. }>, telling you which destination address (= which group) received the
packet.
ASM works but is operationally messier — anyone on the network can send to
the group, you filter at application layer. Avoid for production unless
you control the entire L2/L3 path.
Buffer sizing
Multicast feeds are bursty by nature (NTP-synchronized publishers send at
exact times, not jittered). Default SO_RCVBUF is too small for ≥ 1 Gbps
feeds. Bump it :
.so_rcvbuf(64 * 1024 * 1024) // 64 MiB
You may need sysctl -w net.core.rmem_max=67108864 to allow the
application to set 64 MiB. Without it, the kernel caps at rmem_default.
Cross-platform fallback
multicast_join returns IoError::NotSupportedOnPlatform { platform } on
macOS and Windows. Their multicast APIs are different enough that we don't
ship a parity shim ; if you need multicast on Tier 2 / Tier 3, file a
bug, but the answer is likely "use socket2 directly there and bridge to
zero-io for the data path."
Status
multicast_join SSM is a Tier 1 feature. Status follows step 174 (UDP
handler) ; the API is part of the UDP endpoint config. The IGMP / MLD
state is managed entirely by the kernel ; we just set the right socket
options.
The boring-but-essential mail submission. zero-smtp is a thin layer
over zero-io's TCP shard that handles the SMTP RFC 5321 grammar
(plus 5322 message format, 6376 DKIM, 4954 AUTH, 3207 STARTTLS).
Replaces lettre for zero-io-shaped applications.
A minimal MTA folded into a zero-io shard — accept connections, drive
the SMTP grammar yourself, surface received messages as events :
use std::time::Duration;
use zero_io::{Io, Config, Event};
use zero_io::smtp::{SmtpListener, SmtpAuth};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
io.smtp_listen("0.0.0.0:25".parse().unwrap())?;
io.smtp_listen_tls("0.0.0.0:465".parse().unwrap(), &cert, &key)?;
loop {
io.poll(Duration::from_millis(100))?;
while let Some(ev) = io.next_event() {
match ev {
Event::SmtpAuth { conn, user, pass, request_id } => {
if user == "alice" && pass == "secret" {
io.smtp_auth_accept(request_id)?;
} else {
io.smtp_auth_reject(request_id)?;
}
}
Event::SmtpMessage { from, to, body, .. } => {
println!("from={from} to={to:?} {} bytes", body.len());
// store / relay / bounce
}
_ => {}
}
}
}
}
The shard owns the listener, runs EHLO / STARTTLS / AUTH / MAIL FROM /
RCPT TO / DATA per-connection, and surfaces complete messages as
Event::SmtpMessage. Auth is delegated back to your code via
Event::SmtpAuth — accept or reject by request id.
Submission port (587) and SMTPS (465) supported ; STARTTLS on 25 if
you need it. The server handles the wire grammar ; you focus on auth
storage. Useful for transactional bounce processing or an embedded
relay — not a full Postfix replacement.
If you want SMTP without the zero-smtp ergonomics — for a custom
relay or a forwarder that mutates messages mid-flight — drop to
native zero-io TCP and drive the SMTP grammar yourself.
use std::time::Duration;
use zero_io::{Io, Config, Event, MessageKind};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
let conn = io.tcp_connect("smtp.example.com:587".parse().unwrap())?;
let mut state = SmtpState::Greeting;
loop {
io.poll(Duration::from_millis(100))?;
while let Some(ev) = io.next_event() {
match ev {
Event::Connected { conn, .. } => { /* wait for 220 banner */ }
Event::StreamFrame { conn, kind: MessageKind::Data, data, .. } => {
state = state.advance(&data, &mut io, conn)?;
if state == SmtpState::Done { return Ok(()); }
}
_ => {}
}
}
}
}
You'd hand-implement the EHLO / STARTTLS / AUTH / MAIL FROM / RCPT
TO / DATA state machine. ~300 lines for a complete client. Use the
zero-smtp crate (next tab) for everything except genuinely custom
flows.
use zero_smtp::{SmtpTransport, Mailbox, Message, Credentials};
#[tokio::main]
async fn main() -> Result<(), zero_smtp::SmtpError> {
let mailer = SmtpTransport::relay("smtp.example.com")?
.credentials(Credentials::new("user", "pass"))
.starttls(true)
.build();
let msg = Message::builder()
.from(Mailbox::parse("alice@example.com")?)
.to(Mailbox::parse("bob@example.com")?)
.subject("Hello from zero-smtp")
.body("Plain text body or MIME multipart.")?;
mailer.send(msg).await?;
Ok(())
}
The SmtpTransport builder is API-shaped after lettre — same
method names, same authentication knobs. Drop-in for code that's
currently using lettre.
DKIM signing :
let signer = zero_smtp::DkimSigner::ed25519_pem(&key_pem)?
.with_selector("zero")
.with_domain("example.com");
let mailer = SmtpTransport::relay(...).dkim(signer).build();
Ed25519 signing is O(80 µs per message) vs RSA-2048 ~6 ms. Use
Ed25519 for high-volume relays.
When zero-smtp, when something else
zero-smtp : you're sending transactional mail (alerts,
signups, password resets), embedding a relay, or replacing
lettre in an existing app.
External MTA (Postfix, Exim) : you have heavy mail volume,
queue persistence with millions of pending messages, or compliance
policy requires a battle-tested MTA.
Mailgun / SendGrid / SES : you don't want to run mail at all.
Use their HTTPS API via the HTTP client recipe.
Pitfalls
Sender reputation : direct send from a fresh IP usually lands
in spam. For non-relay use, route through an established relay
(Postfix on a warmed IP, or a transactional service).
DKIM + SPF + DMARC : DKIM signing alone isn't enough — your
sending domain needs aligned SPF, and a published DMARC record.
zero-smtp signs ; the rest is DNS configuration.
STARTTLS opportunistic : RFC 3207 STARTTLS is opportunistic
by default ; downgrade attacks are real on port 25. Enforce
STARTTLS for submission (587), require TLS for SMTPS (465).
Yes, FTP still exists. Mostly because it's wired into a generation of
internal-IT scripts, embedded device firmware, and printer / scanner
drop-folders. zero-ftp is the modest layer that lets you talk to it
without dragging in a heavy library.
Active and passive modes both supported. EPSV / EPRT for IPv6.
splice for large file transfers — files served at NIC line rate
without ever touching userspace memory beyond the control channel
grammar.
For a production-grade FTP server, you'd typically use vsftpd /
proftpd — but zero-ftp::server is fine for embedded contexts (a
device firmware that exposes a config drop-folder), CI artifact
servers, or replacing a legacy FTPlib-based Python script with
something safer.
use zero_ftp::{Ftp, AuthMode};
#[tokio::main]
async fn main() -> Result<(), zero_ftp::FtpError> {
let mut ftp = Ftp::connect("ftp.example.com:21").await?;
ftp.auth_tls().await?; // upgrade to FTPS via AUTH TLS
ftp.login("anonymous", "guest@example.com").await?;
ftp.cwd("/pub/dist").await?;
let bytes = ftp.retr("README.txt").await?;
println!("got {} bytes", bytes.len());
ftp.binary().await?;
ftp.stor("upload.bin", &my_data).await?;
let entries = ftp.list_machine_readable().await?; // MLSD
for e in entries { println!("{} {} {}", e.name, e.size, e.modify); }
ftp.quit().await?;
Ok(())
}
MLSD (RFC 3659 machine-listing) is preferred over LIST —
machine-parseable, no locale-specific timestamp formats.
Passive mode is automatic ; EPSV (RFC 2428, IPv6) is preferred when
both ends speak it. For large transfers, zero-ftp uses zero-io's TCP
shard with splice(2) from the data-channel socket directly to a
std::fs::File — line-rate throughput on Linux Tier 1.
When FTP at all
Legacy interop : the only honest reason. SCP / SFTP / HTTPS /
S3 / object storage are all better in 2026.
Embedded device firmware that has FTP and isn't going to grow
HTTPS.
Printer drop-folders with FTP-only firmware.
If you're building something new, use HTTPS or SFTP. zero-ftp is
here so you don't have to maintain a corner of legacy code on a
heavier stack.
What's NOT in zero-ftp
SFTP (SSH-based) — use russh-sftp or similar SSH stack.
SFTP is a different wire protocol entirely (it rides on top of an
SSH channel).
FTPS implicit ("FTPS" without "AUTH TLS") — supported via
listen_tls(...) on port 990, but explicit AUTH TLS over plain
FTP (port 21) is the default in Ftp::connect() because it's
what most servers are configured for.
Anonymous-only public mirror server — feasible but not the
ergonomic target. Use vsftpd for that.
Recipe : FIX text + SBE (zero-fix + zero-sbe)
🚧 Design preview — separate crates (steps 193 / 193b / 194).
FIX 4.4 text + FIX SBE (Simple Binary Encoding) for CME MDP 3.0,
Eurex T7. Session FSM, persistent WAL, multicast feed handling.
The financial protocols. FIX text is the workhorse of order entry
across most equity / FX exchanges ; SBE (Simple Binary Encoding) is
what high-volume market data feeds (CME, Eurex, ICE) use for tick
streams. zero-fix and zero-sbe are separate crates because their
wire formats and invariants are different — but they share zero-io's
TCP / multicast shards underneath.
Order entry as the initiating side — connect to the exchange, log on,
submit orders, consume execution reports.
use zero_fix::{FixClient, FixConfig, FixVersion, Side, OrdType};
use zero_fix::session::SessionStore;
#[tokio::main]
async fn main() -> Result<(), zero_fix::FixError> {
let store = SessionStore::file("./fix-sessions/sender-id.wal")?;
let cfg = FixConfig::new(FixVersion::Fix44)
.sender("SENDER-ID")
.target("EXCHANGE-ID")
.heartbeat_secs(30)
.session_store(store);
let mut client = FixClient::connect("fix.exchange.com:9876", cfg).await?;
client.logon().await?;
client.new_order_single(|b| b
.cl_ord_id("ORDER-001")
.symbol("ESH6")
.side(Side::Buy)
.ord_type(OrdType::Limit)
.price(4500.25)
.order_qty(1)
).await?;
while let Some(msg) = client.recv().await {
match msg.msg_type() {
"8" => println!("ExecReport : {}", msg.field("39").unwrap_or("")),
"3" => eprintln!("Reject : {}", msg.field("58").unwrap_or("")),
_ => {}
}
}
Ok(())
}
The session WAL is the load-bearing piece. FIX requires that on
disconnect/resume, both sides agree on MsgSeqNum ; SessionStore
persists every outbound message + last received seqnum, atomic
checkpoint per second + on logout. Append-only, CRC32C per record,
recovered on connect to resume cleanly.
The acceptor side — you run the FIX engine, counterparties connect to
you. Same session WAL discipline, applied per inbound session.
The acceptor handles the FIX session FSM (Logon → Heartbeat / TestRequest
/ ResendRequest → Logout), including SeqNum gap detection and resend.
Your business logic only sees application-layer messages
(NewOrderSingle, etc.) once the session is healthy.
zero-sbe is the consumer for binary market-data feeds. Messages are
flyweights — you don't decode into Rust structs, you index into
the wire bytes via a generated schema struct. Zero alloc per tick.
use zero_sbe::{FeedHandler, FeedConfig};
use zero_sbe::schemas::cme_mdp3::*; // generated from MDP3 schema XML
#[tokio::main]
async fn main() -> Result<(), zero_sbe::SbeError> {
let mut feed = FeedHandler::join(
FeedConfig::cme_mdp3()
.channel("310") // ES futures real-time
.multicast("224.0.31.0:14310".parse().unwrap())
.replay("history.cme.com:14311".parse().unwrap())
).await?;
while let Some(packet) = feed.next().await? {
for msg in packet.messages() {
match msg.template_id() {
MdIncrementalRefreshBook46::TEMPLATE_ID => {
let book = MdIncrementalRefreshBook46::wrap(msg.body());
for entry in book.entries() {
// entry.security_id(), entry.price(), entry.qty(), ...
}
}
_ => {}
}
}
}
Ok(())
}
FeedHandler joins the multicast group, accumulates packets, tracks
MsgSeqNum per channel, detects gaps, requests replay from the TCP
replay channel automatically.
We don't ship an SBE publisher — that's the exchange's role. If
you're running an internal SBE-style fan-out, see the
Multicast SSM sender recipe and frame your own
SBE messages on top.
Performance — what each is for
FIX text
FIX SBE
Use case
Order entry, cancel/replace, execution reports
Market data ticks, book refreshes
Throughput target
1k–10k msgs/sec per session
100k–1M+ msgs/sec
Latency budget
sub-ms (ack ↔ ack-receipt)
sub-µs (handler entry)
Wire
ASCII-delimited tags, parsed via zero-fix::Parser
Binary, indexed via schema flyweights
Persistence
Required (SessionWal)
None — feed is replayable from exchange
When zero-fix / zero-sbe, when something else
zero-fix : direct exchange connectivity, custom OMS,
back-testing harness with replay capability. Replaces QuickFIX or
hand-rolled FIX engines.
zero-sbe : low-latency market data ingestion for trading
systems, market-making, smart order routing. Replaces hand-rolled
flyweight decoders.
Financial-data SaaS (Polygon, IEX Cloud, Databento) : if you
don't need exchange-direct latency, use their REST / WebSocket APIs
via the HTTP client and
WebSocket recipes.
What's intentionally not in scope
Order management state across sessions — your business logic.
Smart order routing across venues — different concern, lives
above the FIX wire.
FIXP (FIX Performance) and other binary FIX successors — out
of scope for v1 ; SBE covers the high-volume binary case.
Recipe : NTP client
🚧 Design preview — gated on step 191. Built into
zero-io core (opt-in via ntp feature). Single-shot or daemon-style
sync against an NTP server.
The clock-synchronization protocol. Even on systems where chronyd /
ntpd does the heavy lifting at the OS level, applications sometimes
want to query an NTP server directly — e.g., a financial system that
needs to estimate clock skew vs the exchange's reference, or an
embedded board that doesn't run a real NTP daemon.
If you're already running an Io shard for other I/O, fold NTP into
the same poll loop — no extra thread, no separate runtime.
use std::time::Duration;
use zero_io::{Io, Config, Event};
use zero_io::ntp::NtpClient;
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
let ntp = io.ntp_client()?;
let req_id = ntp.query("pool.ntp.org:123".parse().unwrap())?;
loop {
io.poll(Duration::from_millis(100))?;
while let Some(ev) = io.next_event() {
match ev {
Event::NtpResponse { id, result } if id == req_id => {
let r = result?;
println!("offset : {} ms, RTT : {} ms",
r.offset_ms(), r.rtt_ms());
return Ok(());
}
_ => {}
}
}
}
}
Io::ntp_client() returns a handle that owns a UDP endpoint
internally. Queries return a request id ; results come back via
Event::NtpResponse { id, result }. No threads, no tokio, just the
shard.
use zero_io::ntp;
#[tokio::main]
async fn main() -> std::io::Result<()> {
let result = ntp::query("pool.ntp.org:123").await?;
println!("offset : {} ms", result.offset_ms());
println!("round-trip : {} ms", result.rtt_ms());
println!("stratum : {}", result.stratum());
println!("server time : {}", result.server_time());
Ok(())
}
Single UDP exchange : sends one NTPv4 (RFC 5905) request, parses the
response, computes offset and round-trip per the standard formulas.
For continuous skew estimation (a polling daemon) :
use zero_io::ntp::{NtpDaemon, NtpDaemonConfig};
#[tokio::main]
async fn main() -> std::io::Result<()> {
let mut daemon = NtpDaemon::new(NtpDaemonConfig::default()
.servers(&["pool.ntp.org:123", "time.cloudflare.com:123"])
.poll_interval(std::time::Duration::from_secs(64)))
.start().await?;
while let Some(snapshot) = daemon.next_snapshot().await {
println!("offset {} ms (stratum {}, {} samples)",
snapshot.offset_ms, snapshot.stratum, snapshot.samples);
}
Ok(())
}
The daemon does NOT modify the system clock. It computes offsets ;
your code decides whether to call clock_settime (jump),
adjtimex (slew), or just expose the offset to your own time-stamp
logic.
NTP vs PTP
NTP gets you to ~1-50 ms accuracy depending on network conditions.
For sub-millisecond and below, the answer is PTP (Precision
Time Protocol, IEEE 1588) which uses hardware timestamps in the NIC
driver. zero-io doesn't ship a PTP client — that's a different
beast (multicast, follow-up messages, BMCA, hardware support
required). Use linuxptp (ptp4l / phc2sys) at the OS level.
For application-level "what time does the exchange think it is",
NTP is plenty.
Clock model
The library exposes an offset relative to your local clock — it
doesn't try to wrap a MonotonicClock abstraction around it. You
have :
Your local std::time::SystemTime / std::time::Instant.
The NTP-reported offset.
The freedom to combine them as fits your app (often : timestamp
events with SystemTime and apply offset only when reporting to
external systems).
For high-frequency trading style "match exchange clock exactly",
you'd want PTP + hardware timestamps. NTP gets you within "the
exchange and I are within 30 ms agreement" which suffices for most
non-HFT workloads.
What zero-io NTP doesn't do
Authentication (NTPv4 keyed authentication, NTS) — not in
v1. Public NTP servers don't auth ; if you need auth, use a
private NTP daemon and read from it via Unix socket.
Server mode — zero-io doesn't ship an NTP server. Run
chrony or ntpd for that.
Leap second handling — pass-through ; the library reports
the leap-second flag in the response, your code decides what to
do with it.
Recipe : mDNS / DNS-SD
🚧 Design preview — gated on step 192. Built into
zero-io core (opt-in via mdns feature). RFC 6762 (mDNS) + 6763 (DNS-SD).
Multicast DNS is what Apple Bonjour, Avahi, and the IoT discovery
protocols use to find services on a local network without a central
DNS server. "What's on this LAN that speaks _chartingview._tcp ?"
The mDNS shard (browse + announce in one) folded into your existing
Io :
use std::time::Duration;
use zero_io::{Io, Config, Event};
use zero_io::mdns::{MdnsShard, ServiceInfo};
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
let mdns = io.mdns_shard()?;
mdns.announce(ServiceInfo::new("_chartingview._tcp.local")
.name("server-1")
.port(8080))?;
let browse_id = mdns.browse("_printer._tcp.local")?;
loop {
io.poll(Duration::from_millis(500))?;
while let Some(ev) = io.next_event() {
match ev {
Event::MdnsServiceFound { id, name, addrs, port, txt } if id == browse_id => {
println!("found {name} at {addrs:?}:{port}");
}
Event::MdnsServiceLost { id, name } if id == browse_id => {
println!("lost {name}");
}
_ => {}
}
}
}
}
The shard owns one UDP socket bound to the multicast address, joins
the group, drives both browse and announce concurrently.
use zero_io::mdns::{Browser, ServiceEvent};
#[tokio::main]
async fn main() -> std::io::Result<()> {
let mut browser = Browser::new("_chartingview._tcp.local").start().await?;
while let Some(event) = browser.next().await {
match event {
ServiceEvent::Found { name, addrs, port, txt } => {
println!("found '{name}' at {addrs:?}:{port}");
println!(" TXT records : {txt:?}");
}
ServiceEvent::Lost { name } => {
println!("lost '{name}'");
}
}
}
Ok(())
}
Browser joins the mDNS multicast group, sends the initial PTR
query, listens for responses. Found / Lost events delivered as
services come and go on the LAN.
use zero_io::mdns::{Announcer, ServiceInfo};
#[tokio::main]
async fn main() -> std::io::Result<()> {
let info = ServiceInfo::new("_chartingview._tcp.local")
.name("alice's chart server")
.port(8080)
.txt("version", "1.4")
.txt("api", "rest+ws");
let mut announcer = Announcer::new(info).start().await?;
tokio::signal::ctrl_c().await?;
announcer.stop().await?;
Ok(())
}
The announcer joins the multicast group, sends the initial
unsolicited announcement (3 packets at 1, 2, 4 second intervals per
RFC 6762), then responds to incoming PTR / SRV / TXT queries.
Service-type naming
DNS-SD service types follow _service._proto.local format :
_http._tcp.local — HTTP servers
_https._tcp.local — HTTPS
_ssh._tcp.local — SSH
_ipp._tcp.local — Internet Printing
_zero-io-app._tcp.local — your custom app
Pick a unique service type for your app. Apple maintains a registry ;
register if you ship a public protocol.
When mDNS, when something else
mDNS : zero-config local discovery on a LAN. Works without a
DNS server, without DHCP options, without manual config. Right
for IoT devices, dev tools, peer-to-peer apps.
DNS-SD over unicast (RFC 6763 unicast) : the same query
shape against a normal DNS server. Use when you can configure a
DNS server (corporate AD, internal DNS).
Service discovery in cloud (Consul, etcd, k8s services) :
different problem space — multi-host, authenticated, persistent.
mDNS is link-local only.
Pitfalls
Network stack interference : modern Linux distros run
systemd-resolved or avahi-daemon which already binds the
mDNS port (5353). Two processes can both join the multicast
group on Linux (SO_REUSEADDR + SO_REUSEPORT), but cooperation
matters. If you're embedded and own the box, disable the daemon ;
if you're sharing, use the daemon's API (D-Bus for Avahi).
Wireless link : mDNS multicast on Wi-Fi is famously flaky
(multicast packets get rate-limited, sometimes dropped silently).
Some clients fall back to SSDP / unicast probes.
Subnet boundary : mDNS is link-local. It doesn't traverse
routers. For cross-subnet, you need an mDNS reflector
(avahi-daemon with enable-reflector=yes) or a dedicated
service-discovery tier.
Recipe : SOCKS5 proxy
🚧 Design preview — gated on step 190. Built into
zero-io core (opt-in via socks5 feature). RFC 1928 + 1929 (auth)
1961 (GSS, optional).
The standard layer-4 generic proxy protocol. SOCKS5 lets a client
say "open a TCP connection to host:port via this proxy", with optional
username/password auth. Used by Tor, by corporate egress proxies, by
debugging chains (mitmproxy --upstream socks5://...).
A SOCKS5 server folded into an existing Io shard (no separate
runtime) :
use std::time::Duration;
use zero_io::{Io, Config, Event};
use zero_io::socks5::Socks5Listener;
fn main() -> std::io::Result<()> {
let mut io = Io::new(Config::default())?;
let listener = io.socks5_listen("127.0.0.1:1080".parse().unwrap())?;
loop {
io.poll(Duration::from_millis(100))?;
while let Some(ev) = io.next_event() {
match ev {
Event::Socks5ConnectRequest { peer, target, request_id } => {
if target.ip().is_loopback() {
io.socks5_accept(request_id)?; // splice the relay
} else {
io.socks5_reject(request_id, "rule X")?;
}
}
_ => {}
}
}
}
}
The shard owns the listener, surfaces connect requests as events,
your code applies policy, accepted connections automatically splice
(Linux Tier 1) between the client side and the upstream side.
Use a SOCKS5 proxy as the egress for an outbound TCP connection :
use zero_io_async::net::TcpStream;
use zero_io::socks5::{Socks5Connect, Socks5Auth};
#[tokio::main]
async fn main() -> std::io::Result<()> {
// Connect through a SOCKS5 proxy
let mut sock = TcpStream::connect_via_socks5(
"proxy.example.com:1080", // SOCKS5 server
"target.example.com:443", // ultimate destination
Socks5Auth::UserPass("alice", "secret"),
).await?;
// sock is now a regular TcpStream — read/write as usual ;
// the proxy is transparent.
sock.write_all(b"GET / HTTP/1.1\r\nHost: target.example.com\r\n\r\n").await?;
Ok(())
}
connect_via_socks5 opens TCP to the proxy, runs the SOCKS5
handshake (auth method negotiation, optional username/password,
CONNECT request), then hands you back a TcpStream wrapping the
post-handshake bytes. From your code's perspective, indistinguishable
from a direct TCP connection.
For HTTP / QUIC / WebSocket clients to route through SOCKS5,
HttpPool::with_socks5_proxy(...) and quic_connect_with(... .socks5(...)).
Run a SOCKS5 server. Useful for : a localhost-only debug proxy
(mitmproxy / Wireshark chain), an internal corporate egress, an
anonymizing relay (with explicit caveats).
use zero_io::socks5::server::{Socks5Server, AuthOutcome, ConnectPolicy};
#[tokio::main]
async fn main() -> std::io::Result<()> {
Socks5Server::builder()
.listen("127.0.0.1:1080")
.auth(|user, pass| async move {
// your auth ; return AuthOutcome::Accept or ::Reject.
// Or AuthOutcome::Anonymous to allow unauthenticated.
AuthOutcome::Accept
})
.connect_policy(|peer, target| async move {
// arbitrary policy : block private subnets, allowlists,
// rate limit per peer, etc.
if target.ip().is_private() {
ConnectPolicy::Reject("no internal targets".into())
} else {
ConnectPolicy::Allow
}
})
.run().await
}
The server forwards bytes between the client and the target via
zero-io's TCP shard, using splice on Linux for line-rate
throughput without userspace memory traffic.
ConnectPolicy is the policy hook — block private RFC 1918, enforce
allowlists, log every connection, rate-limit. The SOCKS5 wire grammar
is fully handled by the library ; you write business logic.
UDP ASSOCIATE (SOCKS5 UDP relay) is supported but disabled by
default (rare in practice ; opt in with .allow_udp_associate(true)).
When SOCKS5
Client side : the most common reason is "I need to route
outbound through a corporate proxy" or "I want to test Tor / SSH
-D / a tunnel". Drop in connect_via_socks5(...) for any TCP
destination ; HTTP / QUIC clients have explicit options too.
Server side : embedded debug proxy, internal egress with policy
enforcement, MitM forensics chain. Not an anonymizing public
relay — running an open SOCKS5 server publicly is asking for
abuse.
Pitfalls
Open SOCKS5 servers are abused for spam, scraping, and DDoS
reflection. Always require auth + IP allowlists. Better :
bind to 127.0.0.1 only.
DNS leaks — by default connect_via_socks5("host:443") resolves
the hostname locally before sending the IP to the proxy.
connect_via_socks5_remote_dns(...) lets the proxy do the DNS
(preferred for Tor, anonymizing chains).
UDP ASSOCIATE rarely works through firewalls — not all SOCKS5
servers/clients implement it, and middleboxes often drop UDP relay
packets. Stick to TCP CONNECT unless you specifically need UDP.
What's not in scope
HTTP CONNECT proxy (RFC 7231 §4.3.6) — a different shape ;
use the HTTP client's upstream-proxy config.
SOCKS4 / SOCKS4a — superseded by SOCKS5 since the 90s. We
don't ship support.
Transparent proxy (tproxy) — different mechanism (kernel
netfilter) ; out of scope.
Migration : from tokio
🚧 Design preview — gated on protocol implementations shipping. Shape is
stable ; details may shift with steps 174-188.
Most tokio users won't migrate everything to zero-io — and shouldn't.
Tokio is excellent for general-purpose async. The migration path is
selective : push the network hot path to zero-io, keep tokio for the
business logic that's genuinely async-shaped.
What you keep on tokio
Async DB drivers (sqlx, redis, tonic).
Async HTTP clients to outbound services (reqwest ; or migrate to
zero-io HTTP client per recipe).
async fn business logic, middleware, JWT validation, rate limiting.
The pool of pre-allocated buffers (replace bytes::Bytes round-tripping).
API mapping
tokio
zero-io
TcpListener::bind(...) + accept().await loop
Io::tcp_listen(...) + Event::Connected
TcpStream::read(buf).await
Event::StreamFrame { data, .. } (push, not pull)
TcpStream::write_all(buf).await
io.send_buffer + io.stream_write
tokio::net::UdpSocket::recv_from(buf).await
Event::UdpRecv { data, from, .. }
tokio::time::sleep(d)
io.timer(d, callback)
tokio::select!
match on Event variants in the next_event() loop
tokio::spawn(fut)
shard thread for I/O ; tokio runtime for business async (bridge via IoHandle + OwnedSlot)
Arc<Mutex<State>> shared across tasks
per-shard state, no Arc<Mutex<_>> ; cross-shard via channels
The mental shift
tokio is pull-based. You await the next thing. Your code reads like
sequential code with .await punctuation. The runtime drives polling under
the hood.
zero-io is push-based. You drive poll(). Events arrive in a queue.
Your code is a loop that drains the queue. Handlers don't await ;
they react to events synchronously and queue more work.
This is the same mental model as : Wayland event loops, kqueue +
event-driven C, embedded MCUs. It feels foreign on day 1, natural by day 3.
Migration shape
The pragmatic incremental path :
Identify the hot path — typically the request loop of one HTTP server.
Move just that — start a zero-io shard thread for HTTP, keep
tokio for the rest. Connect them via IoHandle + OwnedSlot.
Profile — verify the wins are real on your workload.
Migrate adjacent hot paths if profile shows they need it.
Stop — you don't have to migrate everything. The bridge is meant to
be permanent.
Example : minimal bridge
// Existing tokio app
let app = my_existing_axum_router();
// Wrap in zero-io HTTP shard via zero-io-axum (no code changes to handlers)
let mut io = zero_io::Io::new(Config::default())?;
let server = zero_io_axum::serve(app, "0.0.0.0:8080".parse().unwrap());
server.run(&mut io)?;
zero-io-axum::serve runs your axum app on the zero-io HTTP shard. Your
handlers still see axum::extract::*, your middleware stack works
unchanged. The cost : ~640 B alloc per request for axum's HeaderMap.
Wins : zero alloc on TLS handshake, kTLS offload, splice for static files.
For full perf, write zero-io-native handlers. But the axum bridge buys you
80% of the wins for ~10 lines of code change.
Don't try to migrate
Code that's heavily Stream-based (e.g., reactive pipelines using
futures::Stream). Different paradigm ; conversion is more rewrite than
migration.
Code that uses tokio::join! heavily for parallel sub-tasks. The
zero-io equivalent is "spawn N independent shard tasks and merge results"
which isn't a 1:1 mapping.
Code that's not on a hot path. The wins disappear ; you pay the
migration cost for no gain.
What's never lost
You can run tokio and zero-io in the same process indefinitely. They
coordinate via channels and IoHandle. There's no big-bang rewrite ;
there's an incremental drift toward whatever shape fits each part of your
program.
Migration : from mio
🚧 Design preview — gated on protocol implementations shipping.
mio is a thin layer over epoll / kqueue / IOCP. If you're using mio
directly, you're already comfortable with event-driven I/O — the migration
to zero-io is more a vocabulary change than a paradigm shift.
Mental model
Both are event-driven. Both have a poll loop. Both deliver events in
batches. The differences are in :
What counts as an event — mio gives you "this fd is readable", you
call recv yourself ; zero-io gives you Event::UdpRecv { data, .. },
the read already happened.
Buffer ownership — mio assumes you own a Vec<u8> buffer per
socket ; zero-io owns the buffer pool.
Lifetime of data — in mio, you recv into your buffer, you own it
until you drop ; in zero-io, Event<'poll>::data: &[u8] is borrowed
and dies at the next poll(). Detach via OwnedSlot if you need it
longer.
Io::udp_bind(addr) / tcp_listen(addr) ; the runtime arms reads
Token
EndpointId, ConnId (more typed than a generic Token)
Events (vec, you allocate)
next_event() returns one event at a time
poll.poll(&mut events, timeout)
io.poll(timeout) then while let Some(ev) = io.next_event()
socket.recv(buf) after readable event
already done, see Event::UdpRecv { data, .. }
socket.send(buf)
io.send_buffer + io.udp_send
Example : udp echo
In mio :
let mut poll = Poll::new()?;
let mut events = Events::with_capacity(128);
let mut sock = mio::net::UdpSocket::bind("0.0.0.0:8080".parse()?)?;
poll.registry().register(&mut sock, Token(0), Interest::READABLE)?;
let mut buf = vec![0u8; 1500];
loop {
poll.poll(&mut events, None)?;
for ev in &events {
if ev.token() == Token(0) {
loop {
match sock.recv_from(&mut buf) {
Ok((n, peer)) => { let _ = sock.send_to(&buf[..n], peer); }
Err(e) if e.kind() == ErrorKind::WouldBlock => break,
Err(e) => return Err(e),
}
}
}
}
}
In zero-io :
let mut io = Io::new(Config::default())?;
io.udp_bind("0.0.0.0:8080".parse().unwrap())?;
loop {
io.poll(Duration::from_millis(100))?;
while let Some(ev) = io.next_event() {
if let Event::UdpRecv { endpoint, from, data } = ev {
let mut buf = io.send_buffer(data.len())?;
buf.write(data);
io.udp_send(endpoint, from, buf)?;
}
}
}
The zero-io version is shorter because the recv-loop-until-WouldBlock and
the buffer management are inside the runtime. You don't see WouldBlock ;
you don't allocate the receive buffer ; you don't track which socket the
event came from (the endpoint is right there in the event).
What you give up
Custom Poll::register / deregister — zero-io doesn't expose raw
fd registration. If you need to register an FD you own (e.g., a Unix
domain socket from a third-party library, an inotify FD), you can use
the IoHandle::raw_fd_register(fd, watch_for, callback) escape hatch.
Sub-second-resolution custom events via Waker — zero-io has its
own cross-process wakeup story (__ulock / futex_wait / Named Events)
that's tighter than mio's Waker ; the mechanism is different.
Generic Token — zero-io uses typed EndpointId / ConnId /
StreamId. They're less flexible (no arbitrary usize) but they catch
bugs.
What you gain
Pool slots — zero-copy RX (mio always copies kernel→buffer).
Multi-shard primitive — IoCluster instead of running N mio
instances by hand.
Protocol handlers built in — QUIC, HTTP, WebSocket, TLS without
pulling in quiche / h2 / rustls separately and wiring them up.
io_uring on Linux — none of mio's optimizations beat BUF_RING +
SENDMSG_ZC + SQPOLL.
When to stay on mio
You're already shipping. mio is well-tested, stable for years. No need
to migrate "just because."
You're on a kernel < 6.7 and stuck there. mio's epoll works ; zero-io
rejects.
You want explicit control over every recv/send call. mio doesn't hide
anything.
Side-by-side cost
Cost
mio
zero-io
Allocations per packet (steady state)
~0 (you reuse your Vec)
0 (pool slot reused)
Userspace memcpy per RX packet
1 (kernel → your buffer)
0 (pool registered with BUF_RING)
Syscalls per TX packet (no SQPOLL)
1 (sendmsg)
1 (io_uring_enter)
Syscalls per TX packet (SQPOLL)
n/a
0 (kernel polls our SQ)
The biggest perf delta is the BUF_RING zero-copy RX, which mio cannot
provide because epoll is readiness-only.
quinn is the current state-of-the-art Rust QUIC library. It's mature, used
in production, async-first. zero-io's QUIC is a different shape, not a
strictly-better drop-in. The migration trade is real.
What you keep
rustls — both quinn and zero-io use it for TLS. Configuration
is similar.
Application protocol — your handler code that consumes streams /
datagrams ports over with mostly s/r/async/sync/.
Cert + key handling — file paths, PEM parsing, ALPN — all the same.
What's different
Async vs sync — quinn is async-first. You accept().await,
recv_datagram().await, read().await. zero-io is event-driven sync.
See why no async for the rationale.
Pool slots vs Bytes — quinn returns Bytes (heap-backed,
refcounted). zero-io returns &[u8] slices into pool slots, with
OwnedSlot for cross-thread.
Connection migration — both support it. zero-io exposes
Event::PathMigration ; quinn has it via the Connection API. Same
capability, different surface.
0-RTT — both support it. Configuration is similar.
HTTP/3 — quinn doesn't ship h3 ; you pair it with h3-quinn.
zero-io ships HTTP/3 via vendored quiche/h3 (vendor patches in §8).
let endpoint = Endpoint::server(server_config, addr)?;
while let Some(incoming) = endpoint.accept().await {
let conn = incoming.await?;
tokio::spawn(handle_connection(conn));
}
async fn handle_connection(conn: Connection) {
while let Ok(stream) = conn.accept_bi().await {
let (mut send, mut recv) = stream;
let buf = recv.read_to_end(usize::MAX).await.unwrap();
send.write_all(&buf).await.unwrap();
send.finish().unwrap();
}
}
zero-io :
let mut io = Io::new(Config::default())?;
io.quic_listen(addr, &cert, &key)?;
loop {
io.poll(Duration::from_millis(10))?;
while let Some(ev) = io.next_event() {
match ev {
Event::Connected { conn, .. } => { /* track conn if you want */ }
Event::StreamFrame { conn, stream, kind, data } => {
let mut buf = io.send_buffer(data.len())?;
buf.write(data);
io.stream_write(conn, stream, buf)?;
if kind == MessageKind::End {
io.stream_shutdown(conn, stream)?;
}
}
Event::Disconnected { conn, .. } => { /* cleanup */ }
_ => {}
}
}
}
The shape changes : no tokio::spawn per connection, no async fn, no
accept().await. Connections multiplex through the same poll loop. State
machine state lives where you put it (a HashMap<ConnId, MyState> is
fine).
Async parity via zero-io-async
If you want to keep some async code (e.g., accept().await ergonomics),
the bridge wraps Io :
// from zero-io-async
let mut async_io = AsyncIo::new(Config::default())?;
async_io.quic_listen(addr, &cert, &key).await?;
loop {
let ev = async_io.next_event().await;
match ev { /* ... */ }
}
The async surface costs ~200 ns per event (future poll + waker), but
gives you the await-shape for code that doesn't need every nanosecond.
Performance comparison (target)
When the benchmark CI gate (step 172.9b) lands, we'll publish actual
numbers. The targets we're tracking (will replace these placeholders) :
Scenario
quinn 0.11
zero-io
QUIC 1-RTT handshake
reference
within ±10%
QUIC datagram round-trip (lo)
reference
-20% to -40% latency
QUIC stream 1 MB throughput (lo)
reference
+30% to +50%
Allocs / 100 ticks (steady state)
several hundred
0
Memcpy / packet TX
2 (Bytes copies)
0 (with SENDMSG_ZC)
Memory per 10k connections
reference
-30% to -50%
These are projections from our internal benchmarks against the design ;
they'll be measured and published.
When not to migrate
You ship quinn already. quinn is good. "It's faster" is rarely
worth the risk of a migration.
You depend on the quinn ecosystem (h3-quinn, quinn-proto extensions,
custom crypto provider). zero-io's QUIC stack is built differently ;
ports are not 1:1.
Your app is mostly async. The shape change is the dominant cost ; if
the rest of your app is async, going sync at the QUIC layer creates a
bridge that may eat the wins.
Roadmap
zero-io is in active design and partial implementation. This section is the
single live tracker — accurate as of build time. Each plan file in the repo
(PLAN-STEP*.md) is the source of truth ; this is just the index.
Separate companion crates : zero-rest (180), zero-smtp (184),
zero-ftp (185), zero-grpc (186), zero-mqtt (188b), zero-redis (189),
zero-fix / zero-sbe (193 / 194). Their docs live with their own crate.
179 + 188 are all merged with green CI, the zero_alloc_proof test passes
across the protocol matrix, and benchmarks are within ±15% of nginx (HTTP)
and quinn (QUIC) on the reference machine.
FAQ
The questions that keep coming up. Answers are short on purpose ; deeper
material lives in the linked sections.
"Can I use zero-io with tokio?"
Yes. Two paths :
Native : the shard thread runs sync, async work goes through IoHandle
Bridge : zero-io-async (step 181) wraps the shard with AsyncIo so
you can .await events. ~200 ns overhead per event for the future
machinery.
You don't have to pick one or the other ; pick per use case.
"Why isn't it async by default?"
Because async allocates, indirects, and adds scheduling latency. Those
costs are tiny in absolute terms ; on a 1M-pps hot path, they're the
bottleneck. Long version : Why no async.
"Does it work on macOS / Windows?"
Yes, with explicit perf differences. macOS = Tier 2 (kqueue), Windows = Tier 3
(RIO + IOCP). Linux is Tier 1 with all features. Cross-platform parity is
API-level, not perf-level. See Tier 1 / 2 / 3.
"How many connections can it handle?"
Default Config accepts ~1k concurrent connections per shard. Tunable via
Config.max_connections and the connection table size. With multi-shard at
the cap (16384 for QUIC) and per-shard 64k connections, you're at ~1B
total ; you'll hit kernel limits, NIC limits, and ulimit before you hit
zero-io's limits.
"Is it production-ready?"
Honest answer : not yet. §172a foundations are stable and tested. §172
io_uring backend is substantial but evolving. Higher-level protocols (UDP
174, QUIC client 175, TCP 176, HTTP 179, WebSocket 179b, …) are in design
or partial implementation. The first production target is a real-time
market data system pushing ~100k msg/s, validating the library on a real
workload before broader release. See Roadmap for the live
tracker.
"When does 1.0 ship?"
When : (a) the protocols listed in Roadmap are merged with
green CI, (b) zero_alloc_proof and memcpy_proof pass across the matrix,
(c) benchmarks land within ±15% of nginx (HTTP) and quinn (QUIC) on the
reference machine, (d) docs.rs published, (e) we've run a non-trivial
production workload for 30+ days. No date.
"What's the license?"
TBD. Likely permissive (MIT or Apache-2.0), to be decided at first public
release.
"How does it compare to quinn / tokio / mio / hyper / nginx?"
quinn
tokio
mio
hyper
nginx
zero-io
Allocations / req hot path
several
several
a few
many
0
0
Locks / req
few
many
none on hot
few
none
none
Memcpy / pkt RX
1-2
1+
1
2+
0-1
0
Memcpy / pkt TX
1-2
1+
1
2+
0
0 (with SENDMSG_ZC)
Async
yes
core
low-level
yes
no
no, optional bridge
Cross-platform
yes
yes
yes
yes
Linux + others
Tier 1 Linux, 2/3 others
"Who's building this?"
Initially built to push the latency floor of standard Rust async
runtimes for a real-time market data + trading backend, then extracted
as a generalist transport library. The landing's footer has the author
/ team info ; this docs page is the technical detail.
"How do I contribute?"
Source repo and contribution guide will be published with 1.0. Until then,
the design is happening in PLAN-STEP*.md files, reviewed iteratively. If
you're early-curious, file a GitHub issue when the repo is public.
"Why the name?"
Three zeros. Zero allocations, zero locks, zero copies on the hot path.
That's the entire pitch ; the name says it on the tin.
Resources
Where to find more, in increasing order of detail.
Inside this site
Home — the marketing pitch, three zeros, comparison table, daemon
vs embedded modes, protocol matrix.
API reference — every public type, every method, every error
variant. Equivalent to docs.rs once 1.0 ships.
Roadmap — live status of each plan in the project.
External (when published)
crates.io — cargo add zero-io once we ship.
docs.rs/zero-io — auto-generated rustdoc HTML, the canonical API
reference for any version.
GitHub repository — source, issues, PRs. Published with 1.0.
Plan files (current source of truth, in-tree)
The library is being designed in PLAN-STEP*.md files. Each plan covers one
step ; review iterations are tracked in their R-logs. If you want to read the
design as it's being decided, these are the primary documents :
PLAN-STEP172a.5-QUICHE-PIPELINE-SOTA.md — quiche integration cleanup,
GRO fanout via share_into.
PLAN-STEP172b-QUICHE-H3-ZERO-ALLOC.md — zero-alloc HPACK encode in our
vendored quiche/h3/.
PLAN-STEP172d-MEMFD-TYPESTATE-SOTA.md — memfd-backed buffers, typestate
on Mapped<S> regions.
PLAN-STEP173-ZERO-IO-RENAME-API.md — public API surface, multi-shard
routing R3.7.
PLAN-STEP174-UDP-HANDLER.md — UdpHandler impl over io_uring.
PLAN-STEP174b-AF-XDP.md — AF_XDP backend (in active review R184+).
PLAN-STEP174c-FSTACK-TCP-USERSPACE.md — userspace TCP stack
(FreeBSD's network code ported to userspace) over AF_XDP (design only).
Plan filename keeps the historical short tag ; the user-facing name is
"userspace TCP stack".
CLAUDE.md — the project specification. Coding rules, architectural
invariants, the philosophy.
How this docs site is built
This page is rendered from docs-src/*.md files in the project repository
by scripts/build-docs/ (a small Rust binary using pulldown-cmark). The
output is injected between <!-- DOCS:START --> and <!-- DOCS:END -->
markers in index.html, which gets deployed to 0io.xmit.dev via the xmit
CLI.
When the zero-io crate is published, the docs-src/*.md files migrate
into zero-io/docs/ and become the canonical book. The same content,
multiple rendering targets : this site, GitHub markdown, eventually
mdbook with cross-page search.
Contact / questions
Until the public repo opens : email the team listed in the home page footer.
File issues for bugs and feature requests once the GitHub URL is published.
Benchmarks methodology
🚧 Numbers pending. The benchmark CI gate (step 172.9b)
isn't fully wired yet. This page documents how we'll measure once it
lands, so the numbers we publish later are reproducible and can be
challenged.
The "zero-io is faster than X" claim has to be backed by reproducible numbers
on a documented machine, with documented commands, against documented peers.
Everything else is marketing.
What we measure
Three classes of metrics, each with its own gate :
Throughput
UDP echo pps — single shard, 1500 B payload, loopback. We compare
against : raw recvmsg / sendmsg C code (the kernel ceiling), tokio:: net::UdpSocket, mio directly.
QUIC datagram round-trip — single connection, single dgram, RTT
measured ns-precision via TSC. Peer : quinn 0.11.
QUIC stream 1 MB transfer — single bidi stream, 1 MiB payload, time
from open_bi() to recv complete. Peer : quinn 0.11.
HTTP "Hello world" — req/sec on a 1-byte response over keep-alive.
Peer : nginx, hyper, actix-web. nginx is the reference.
HTTP 1 KB JSON — typical API workload. Same peers.
HTTP 1 MB file download — both cleartext and HTTPS (with kTLS where
applicable). Tests splice + zero-copy file serving. Peer : nginx.
WebSocket fan-out — N subscribers receive M messages/sec from one
publisher. Tests pool capacity and share_into GRO. Peer : tokio- tungstenite-based ws server.
Latency
p50 / p99 / p99.9 / p99.99 of every throughput scenario, measured
on the receiving side at server-handler entry, not at the wire.
Tail latency under load — same scenarios at 80%, 90%, 95% of the
max throughput. Tail behavior is where async runtimes typically lose.
Cold-start latency — connection establishment time (TLS handshake
included). Peer same as throughput.
Resource
Allocs / 100 ticks — counting allocator wrapped around a 10-second
steady-state run. Target : exactly 0 for the protocol matrix.
Gating in CI.
Memcpy / packet — eBPF tracepoint on __memcpy / memmove,
filtered to the application's threads. Target : exactly 0 for RX,
exactly 1 for TX (the unavoidable user→pool slot copy). Gating in CI.
RSS / 1k connections — process memory after 1k idle connections,
steady state. Peer same as throughput.
CPU per Mbps — userspace + softirq + kernel time per Mbps of
throughput. Lower is better. Tells us whether wins come from less work
or just batching better.
Reference machine
CPU : AMD EPYC 9554 (64 cores Zen 4) or Intel Xeon Gold 6438Y+ (32
cores Sapphire Rapids).
RAM : 256 GiB DDR5-4800.
NIC : 100GbE Mellanox ConnectX-6 Dx (XDP native mode).
OS : Ubuntu 24.04 LTS (kernel 6.8 with our patched 6.7+ feature
matrix).
rustc : pinned (will publish exact version with each benchmark
release).
The hardware spec is published with every benchmark run. Numbers from a
different machine are different numbers ; we don't claim universality.
How to reproduce
# Clone the repo
$ git clone <repo>
$ cd zero-io
# Set up the kernel
$ uname -r # must be ≥ 6.7
$ sysctl net.core.rmem_max=67108864 net.core.wmem_max=67108864
# Run the bench suite
$ cargo bench --bench full_matrix
# Each scenario emits criterion output + a JSON dump.
# Compare against baselines :
$ scripts/bench-regression.sh
bench-regression.sh checks each scenario against the baseline (per
scenario, e.g., 15% margin for UDP echo, 20% for QUIC streams) and fails
the gate if regression > margin.
What the gates assert
Two CI gates :
Numerical regression — each benchmark must be within X% of its
committed baseline (margin per-scenario). PRs that regress block.
Zero-alloc / zero-memcpy proof — zero_alloc_proof test asserts
exactly 0 allocs over a steady-state workload ; memcpy_proof asserts
exactly the structural minimum memcpys. Either nonzero blocks the PR.
The numerical regression gate prevents perf rot. The zero-alloc / zero-
memcpy gate prevents structural perf rot — even if the new code is
1 ns faster on average, it can't introduce an alloc or a memcpy.
What we don't measure
Synthetic micro-benchmarks of the runtime alone — they don't tell
you anything about real workloads. We bench against real peers.
"How fast is my hello world" — nginx will always look great
there because hello-world is sendfile of a static file. We bench
scenarios that hit actual network code paths.
GC pauses — there are none. (No GC.) But we still measure tail
latency, because allocator behavior on tokio causes equivalent pauses.
What gets published
When the gate is green :
A markdown table per scenario, with median + p99 + p99.9 + std-dev
across N runs.
Don't trust any specific number we publish in headlines or marketing. The
qualitative claims (zero alloc / zero copy / 0-lock) are structurally
true today (CI gates them on synthetic benchmarks). The quantitative
claims (≥ ±X% vs Y) come with the gate.
Glossary
Vocabulary that recurs across the docs. Definitions are short and link to
the section that goes deeper.
AF_XDP — Linux kernel mechanism that bypasses the network stack and
delivers packets directly to userspace via shared memory rings (UMEM).
Required for the linux-af-xdp Cargo feature. See
AF_XDP runtime.
Brand ('b lifetime) — phantom lifetime parameter on Slot<'p, 'b, S>
that tags a slot to a specific pool. Mixing slots from different pools is a
compile error. See why typestate.
BUF_RING — io_uring feature where the kernel writes incoming packets
directly into pre-registered userspace buffers (our pool slots). No memcpy,
no syscall to provide a buffer per recv. Linux 5.19+. See
zero-copy.
Cap (shard cap) — maximum number of shards a routing strategy supports.
16384 for QUIC ReusePortCbpf, 65536 for UDP/TCP kernel, 1024 for AF_XDP.
See single shard vs cluster.
Cascade (backpressure cascade) — four-level overload-handling
escalation : pool → connection → listener → cluster ingress. Each level
has a default policy and a hook to override. The signal propagates up
when a level can't absorb the pressure locally. See
backpressure cascade.
ConnId — typed identifier for a connection. Replaces an opaque usize
or generic Token. Returned by quic_listen / tcp_connect etc.
DcidDispatch — multi-shard routing strategy where one dispatcher thread
reads all packets from a single socket and transfers ownership of pool
slots to per-shard SPSC rings. Used when central admission control matters.
See multi-shard routing.
Endpoint / EndpointId — a bound network address (typically a UDP
socket) opened via udp_bind. Multiple endpoints per shard supported.
Event<'poll> — borrowed event delivered by next_event(). The 'poll
lifetime ties it to &mut Io ; events die at the next poll(). See
event lifetimes.
Userspace TCP — FreeBSD's TCP/IP network code, ported to userspace
and running on AF_XDP. Bypasses Linux kernel TCP entirely. Opt-in via
linux-userspace-tcp Cargo feature. See
userspace TCP stack.
Hot path — the per-tick code that runs millions of times per second :
RX, route, handler, TX. Where the three zeros are gated.
Io — single-shard handle. !Send. Owns sockets, io_uring, pool,
connection table. See mental model.
IoCluster — multi-shard launcher. Spawns N shards, distributes
incoming connections via SO_REUSEPORT or BPF. See
single shard vs cluster.
IoHandle — Send + Sync + Clone cross-thread reference back into a
shard. Used to ship TX work from async tasks. See
async integration.
kTLS — kernel-side TLS encryption (Linux 5.10+). Userspace negotiates
the handshake, kernel encrypts the data path. Enables splice for
zero-copy HTTPS file serving. Linux-only. See Tier 1 / 2 / 3.
MessageKind — variant on StreamFrame distinguishing Text / Binary /
Ping / Pong / Close (WebSocket) or End-of-stream (QUIC).
OwnedSlot — Send + Sync refcounted handle to a pool slot, returned by
detach_event_data(). Survives across poll() cycles. The cross-thread
zero-copy primitive. See zero-copy.
PathMigration — Event variant fired when a QUIC connection's peer
address changes (Wi-Fi → cellular, NAT rebinding). The DCID stays the same
so the shard mapping is unchanged.
Pool — pre-allocated array of fixed-size slots for packet buffers.
Sized at boot, never grows. The reason zero-alloc holds. See
pool system.
poll() cycle — one tick of the shard. Six phases : release guards →
io_uring submit/wait → drain CQEs → process dirty handlers → flush TX →
fire timers. See the poll cycle.
ReusePortCbpf — multi-shard routing strategy using SO_REUSEPORT + a
kernel BPF program that picks the destination shard from packet contents.
The default. See multi-shard routing.
SendBuffer — typed wrapper around a Reserved pool slot. Returned by
io.send_buffer(min_size). Use buf.write(&[u8]) or buf.as_mut_slice()
to fill, then udp_send / stream_write to ship.
Shard — one OS thread + one Io instance + one io_uring + one pool +
one connection table. Single-threaded by design. See mental model.
share_into<N> — pool slot primitive for GRO fan-out. Splits one
Borrowed slot into N refcounted SharedRead siblings, all reading the
same buffer. Zero alloc, N atomic refcount bumps. See pool system.
Slot / Slot<S> — single buffer in the pool, parameterized by typestate
S. 6 states : Reserved, Committed, InFlight, Completed,
Borrowed, SharedRead. See why typestate.
SPSC ring / MPSC ring — lock-free queue primitives in
base/ring.rs (SpscRingRepr<T,N>, MpscRingRepr<T,N>). SPSC is
wait-free (Lamport pattern, two AtomicU32 cache-line-isolated heads,
no CAS). MPSC is lock-free but not wait-free (CAS-reservation +
2-phase publish gate, ~10-50 ns / op typical). Used internally by
IoHandle (MPSC : many async tasks → one shard, CAS reservation +
ShardWakeupHandle futex bump) and the DcidDispatch cross-shard
relay (SPSC : one dispatcher thread → one shard). Re-exported as
pub in charting-transport::base for layer-3 users ; most
applications use IoHandle instead. See
cross-thread channels.
SQPOLL — io_uring mode where a kernel thread polls our SQ ring.
Userspace produces SQEs by writing to memory ; no enter() syscall on the
TX hot path. Linux 5.5+. See why Linux first.
StreamFrame — Event variant for stream data (QUIC, TCP, WebSocket
text/binary). Carries conn, stream, kind: MessageKind, data: &[u8].
Typestate — encoding state machine state as a generic type parameter,
making illegal transitions uncompilable. The Slot<S> design. See why
typestate.
XdpDcid — multi-shard routing strategy using AF_XDP + an XDP-attached
BPF program. Highest-pps option, capped at 1024 shards by xsk_map BPF
verifier. See AF_XDP runtime.
Zero IO — the project's pitch : zero allocations, zero locks, zero
copies on the hot path. Each measurable, each gated by CI. See
The Three Zeros.
ZeroRuntime — async bridge wrapper that runs Io alongside a tokio
runtime. Four modes (Single-thread, Pool, Dedicated, Hybrid). See
async integration.
Last built : 2026-05-05 · source : docs-src/*.md in repo