The Latency Tax: Internet Polling vs Process Births
I moved CI onto my own Ryzen 7 7730U — no containers, no queues, host binaries, one-second runner polling. Jobs still wait. Here’s the itemized bill, because “fast machine” and “fast pipeline” are different currencies.
Tax 1: the illusion of hardware speed
My runner sits behind a home NAT. GitLab.com cannot reach in
— no inbound connections through the firewall — so my runner must poll
out: HTTPS long-poll, “any jobs for me?”, every second
(check_interval = 1, down from the 3–30s default backoff).
Each poll is a round trip across the public internet to gitlab.com and
back.
| Hop | Cost |
|---|---|
| Poll interval floor | ~1s per pickup (was up to 30s) |
| NAT + TLS + HTTPS per poll | tens of ms, forever |
Single job slot (concurrent = 1) |
superseded pipelines queue behind the running one — minutes, the real killer |
The CPU was never the bottleneck. Reachability was. A beefy box behind NAT still pays the internet polling tax; killing the defaults (1s poll, cancel superseded runs) recovers most of it. The remainder is physics — packets, not gigahertz.
Tax 2: process births, not work
Checking ~200 posts means three gates per post — and each gate spawns a fresh Babashka JVM (~0.3–0.5s startup) to do milliseconds of actual work. Roughly 600 process births: minutes of overhead, seconds of checking. My pre-push hook (same gates, two of them) runs in 0.35s — the difference is entirely birth rate.
The real fix isn’t faster hardware, it’s fewer births: one long-lived process looping posts in-process instead of 600 short-lived ones. Same checks, one startup. That’s a post-Monday project; until then, changed-posts-only scoping on merge requests pays most of the savings.
Push vs poll (realpolitik)
Local runners behind home firewalls can never receive event pushes — GitLab can’t open connections inward. Polling is not a design flaw to fix; it’s the permanent price of self-hosting behind NAT. You budget for it (1s interval), you cancel stale queue entries aggressively, and you stop blaming the CPU for what the network owns.
Total latency equation for my setup: queue position + 1s poll + process births + whole-site build. Three of the four are now tunable. The fourth — the full build and 1,180-file link check — is the irreducible cost of actually verifying the site. That one you don’t optimize. You respect it.