Every speed test article on this site leans on the same underlying process. Rather than repeat it in a footnote each time, we’re laying the whole thing out here — the equipment, the math, the mistakes we’ve learned to avoid, and the honest limitations of testing VPN performance at all. If you’ve ever wondered why two review sites publish wildly different numbers for the same provider, this article explains why.
Why VPN Speed Testing Is Harder Than It Looks
A VPN speed number is never really a single measurement — it’s a snapshot of your device, your router, your ISP, the VPN server’s current load, the physical distance to that server, the protocol in use, and the exact second you happened to run the test. Change any one variable and the number moves. Our methodology exists to control as many of those variables as possible, so the differences we report reflect the VPN itself rather than noise.
Our Testing Stack
| Component | What We Use | Why |
|---|---|---|
| Primary connection | Dedicated fiber lines (500 Mbps–1 Gbps) | Removes ISP throttling as a variable |
| Speed measurement | Ookla Speedtest CLI | Scriptable, repeatable, avoids browser overhead |
| Secondary verification | Custom throughput script over HTTP/2 | Cross-checks Speedtest results against raw file transfer |
| Latency tool | ICMP ping + traceroute | Separates routing overhead from raw throughput loss |
| Hardware | Wired connection only, no Wi-Fi | Eliminates wireless interference as a variable |
The Five-Run Rule
We never publish a result based on a single test. Every server we test gets a minimum of five runs, and we discard the highest and lowest outlier before averaging the rest. This protects against the two most common distortions in VPN speed testing: a lucky moment of low server load that flatters a provider, and a temporary network hiccup that unfairly punishes one.
Time-of-Day Testing
Server load isn’t static — it rises and falls with the local time at the server’s physical location, not just at the tester’s location. That’s why our full reviews test at three points in the day: early morning, early afternoon, and evening peak hours, always measured in the server’s local time zone rather than our own. A server in Tokyo tested at our own 9am (Tokyo’s 10pm) tells a very different story than the same server tested at Tokyo’s own morning.
“A speed test run once, at one time of day, is a photograph. A full review needs something closer to a weather forecast — a pattern built from repeated observation.”
How We Choose Which Servers to Test
- Geographic spread: We always include North America, Europe, and Asia-Pacific at minimum, since routing behavior differs meaningfully by region.
- Popularity: We prioritize servers that provider apps list as “recommended” or auto-select by default, since that’s what most subscribers actually connect to.
- Distance tiers: We deliberately include both a nearby server and a long-haul server, so readers can see how performance degrades with distance rather than assuming one number applies everywhere.
Protocol Testing: Why We Don’t Just Use the Default
Most VPN apps ship with a sensible default protocol already selected, but defaults aren’t always the fastest option, and app defaults sometimes change between versions. We manually test every available protocol option — WireGuard-based options, OpenVPN in both UDP and TCP modes, and any proprietary protocol the provider offers — so our published recommendations reflect actual performance rather than whatever ships out of the box.
Calculating “Percent of Baseline”
The retention percentage you see throughout our reviews is calculated as:
(VPN-protected speed ÷ unprotected baseline speed) × 100
We recalculate the baseline immediately before and after every testing session, not once at the start of the day, because ISP performance itself can drift by several percent over a few hours — and we want the percentage to isolate the VPN’s impact, not our own connection’s mood that day.
What We Deliberately Don’t Do
- We don’t use provider-supplied test servers or partnerships that could bias server selection.
- We don’t cherry-pick the single best run out of many — every published number is a controlled average.
- We don’t test exclusively during off-peak hours, which would flatter every provider unrealistically.
- We don’t rely solely on browser-based speed test widgets, which introduce their own overhead and inconsistency.
Common Reader Questions About Our Numbers
Why do my own results differ from yours?
Almost always because of a variable we control for in testing but that varies naturally in home setups: Wi-Fi versus wired connection, router quality, background app usage, or simply a different server load at the moment you tested. Differences of 10–20% between our lab results and a reader’s home results are completely normal and expected.
Why does the same provider score differently in different articles on this site?
Because we test different servers, different protocols, or different connection speeds depending on the article’s focus. A head-to-head comparison piece and a dedicated single-provider deep dive are answering slightly different questions, even when they cover the same VPN.
How often do you re-test?
Major providers get re-tested whenever they announce significant infrastructure changes, and at minimum every few months regardless, since server networks are never static.
Limitations We Want to Be Upfront About
No lab test, however careful, can perfectly predict what any individual subscriber will experience. Our numbers are a controlled, repeatable comparison point — not a guarantee. Local network conditions, device age, time of year, and even temporary server maintenance can all shift real-world results away from what’s published here. We publish our full methodology, including this article, specifically so readers can judge how much weight to put on any single number.
How We Handle Outliers and Anomalies
Occasionally a single test run will produce a number wildly out of step with everything around it — a download speed that’s half of every surrounding run, or a ping spike ten times higher than normal. Rather than silently averaging these away or silently excluding them without explanation, we flag them specifically in our internal notes and re-test that server at a different time before publishing. If a server consistently produces erratic results across multiple re-tests, we note that instability directly in the relevant review rather than smoothing it into a single average number that would hide the problem from readers.
Controlling for Our Own Network
Because our baseline connection is itself a variable, we monitor it continuously during testing sessions using a lightweight background ping and throughput logger. If our own ISP shows signs of a local outage, congestion, or maintenance window mid-session, we discard that entire session’s results rather than publishing numbers that might reflect our network’s problems rather than the VPN’s performance. This has meant scrapping and re-running full test days more than once, but it’s a necessary safeguard for result integrity.
Why We Test Multiple ISPs, Not Just One
Some ISPs handle encrypted traffic differently than others — a small number are known to deprioritize traffic patterns associated with VPN tunnels during peak congestion windows, regardless of which provider is used. To avoid publishing results that are secretly an ISP story rather than a VPN story, our larger test rounds rotate across at least two different ISPs on two different physical connections. When we see a meaningful gap between ISPs for the same VPN and server, we call it out explicitly rather than averaging it into a single misleading number.
The Role of DNS and First-Connection Overhead
A detail that’s easy to overlook: the very first request after connecting to a VPN often includes DNS resolution and tunnel handshake overhead that a pure throughput test doesn’t capture. We separately time “connection to first byte” — how long it takes from clicking connect to the first successful page load — because this is often what a normal user actually experiences as “the VPN feels slow,” even when steady-state throughput numbers look perfectly healthy.
| Measurement | What It Captures |
|---|---|
| Steady-state throughput | Sustained download/upload speed once a connection is established |
| Time to first byte | How responsive the connection feels in the first few seconds |
| Reconnection time | How long switching servers takes, relevant for frequent server-hoppers |
Tools We’ve Retired and Why
Early in our testing history we relied more heavily on browser-based speed test widgets for convenience. We moved away from this approach after noticing meaningful, repeatable discrepancies between browser-based results and CLI-based results on identical connections — browser JavaScript overhead and buffering behavior introduced enough noise to distort comparisons between providers. We now use browser tests only as a rough sanity check, never as a primary data source for published numbers.
How Long a Full Provider Review Takes
For transparency: a full individual provider review, of the kind published elsewhere on this site, typically involves 12–15 servers, 3 time-of-day windows per server, 2 protocols minimum, and at least 2 separate testing days to catch any single-day anomalies. That works out to well over 300 individual test runs behind a single published article — the numbers you see in any review represent a small, carefully averaged summary of a much larger underlying dataset.
The Bottom Line
Good VPN speed testing isn’t about finding the single most flattering number for any provider — it’s about building a repeatable process that produces the same kind of result under the same conditions, every time. That consistency, backed by hundreds of individual test runs per review, is what lets our individual provider reviews and head-to-head comparisons actually mean something when you read them side by side.
Methodology FAQ
Do you accept payment for favorable speed results? No — our testing lab and process are identical regardless of any commercial relationship with a provider.
Can I request a specific server or route be tested? Yes, reach out through our contact page and we’ll factor popular requests into future testing rounds.
What connection speed do you recommend for accurate home testing? Any consistent, wired connection works — the key is testing at the same time of day over several days rather than relying on a single result.
Why did you stop using browser-based speed tests as a primary source? We found repeatable discrepancies caused by browser overhead that introduced noise into provider comparisons, so we shifted to CLI-based tools for primary results.
How many test runs go into a single published review? Typically well over 300 individual runs across servers, time windows, and protocols, condensed into the averaged figures we publish.
About the Author
