# What to measure in a proxy benchmark

Source: https://ipvolt.com/blog/what-a-proxy-benchmark-should-measure
Markdown: https://ipvolt.com/blog/what-a-proxy-benchmark-should-measure.md

[Home](https://ipvolt.com/index.md) / [Blog](https://ipvolt.com/blog.md) / What to measure in a proxy benchmark

Category: Analysis
Published: 2026-09-11
Updated: 2026-09-11
Author: ipvolt team
Reading time: 7 minutes
Tags: proxies, choosing, troubleshooting

Compare proxies using content success, response times and session behaviour, with a curl-based Python harness that records failures and calculates results.

Compare proxies using application success rate, response-time distributions, connection setup and session behaviour. Define success for each destination before collecting results. The examples below include an observed direct baseline and a small harness that checks a response marker and calculates success rate and latency. Comparable residential and datacenter results have not yet been published by us.

## Define success before measuring speed

Low response times can hide a high cost per successful request when failures trigger retries. Count completed work as well as elapsed time, and include any retry traffic in your cost estimate.

Success needs a definition per destination: the expected status, the required content and an acceptable redirect outcome. An HTTP 200 response containing a CAPTCHA challenge can fail that definition. A missing field or an incomplete download can fail it too. A simple text marker is useful for a starter check; a production workload may need parsed records, schema validation or a complete multi-step flow.

Record successful responses and each failure category separately, then investigate the cause. A timeout can occur on several parts of the path. A 403 alone does not identify which intermediary or origin enforced a restriction. Neither response time nor status is enough to assign blame.

## Record time to first byte and total time

curl reports timing milestones measured from the start of the transfer. Time to first byte (TTFB) tells you when the response began; total time includes receiving its body. Record both, because a prompt first byte can still precede a slow or incomplete download.

On 11 September 2026, three consecutive direct requests to `https://example.com/` from a VPS in Helsinki using curl 8.18 produced the following values. No proxy was used. The labels below make the cumulative meaning explicit:

```text
http=200 tcp_complete=0.009780s tls_complete=0.023224s ttfb=0.044821s total=0.044983s
http=200 tcp_complete=0.008289s tls_complete=0.023851s ttfb=0.039902s total=0.039966s
http=200 tcp_complete=0.008149s tls_complete=0.031935s ttfb=0.061614s total=0.061688s
```

That is an observed baseline of approximately 40–62 ms to first byte, not a lower bound on future requests. Three observations do not establish a stable distribution. Collect a fresh baseline on the same machine and destination close to the proxy comparison, and retain the run date, client version and configuration.

This bounded command collects the same fields for a new direct request:

```sh
curl --disable --silent --show-error --noproxy '*' \
  --connect-timeout 5 --max-time 20 --retry 0 \
  --output /dev/null \
  --write-out 'http=%{http_code} tcp_complete=%{time_connect}s tls_complete=%{time_appconnect}s ttfb=%{time_starttransfer}s total=%{time_total}s exit=%{exitcode}\n' \
  'https://example.com/'
```

For the same request through an HTTP(S) proxy, use curl 8.3 or later. Set `PROXY_URL` to the provider's credential-free gateway, including its supported scheme and port. Have your secret manager supply `PROXY_USERNAME` and `PROXY_PASSWORD` in the environment. This example follows the [curl setup guide](/guides/curl-proxy-setup): curl imports credentials itself so the expanded password is not a shell argument. Do not enable shell tracing.

```sh
: "${PROXY_URL:?Set a credential-free HTTP(S) gateway URL}"
curl --disable --silent --show-error --noproxy '' \
  --proxy "$PROXY_URL" --proxy-basic \
  --variable %PROXY_USERNAME --variable %PROXY_PASSWORD \
  --expand-proxy-user '{{PROXY_USERNAME}}:{{PROXY_PASSWORD}}' \
  --connect-timeout 5 --max-time 20 --retry 0 \
  --output /dev/null \
  --write-out 'http=%{http_code} tunnel=%{http_connect} tcp_complete=%{time_connect}s tls_complete=%{time_appconnect}s ttfb=%{time_starttransfer}s total=%{time_total}s exit=%{exitcode}\n' \
  'https://example.com/'
```

`--disable` stays first to ignore personal curl configuration. The direct command's `--noproxy '*'` bypasses inherited proxies; the proxied command's empty bypass list prevents `NO_PROXY` from excluding the destination. These commands use one fresh connection per curl process, do not follow redirects and do not retry. An HTTP error can still have curl exit code zero because the commands deliberately omit `--fail`; inspect the HTTP status separately. See the [curl manual](https://curl.se/docs/manpage.html#--noproxy) for route selection and the [timeout guide](/guides/proxy-timeout-troubleshooting) for deadlines.

## Understand what connection timings include

For an HTTPS destination through an HTTP proxy, curl establishes a connection to the proxy, requests a CONNECT tunnel, then negotiates TLS to the destination through that tunnel. [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110#name-connect) defines CONNECT.

The interval between `time_connect` and `time_appconnect` includes tunnel setup and TLS establishment. Subtracting the corresponding direct interval does not isolate CONNECT overhead: routing and handshake conditions change too. An HTTPS proxy adds another TLS handshake. A static CDN object does not remove those differences.

Use the milestones to locate where delay appears, then investigate with controlled endpoints or suitably instrumented client and proxy logs. Only subtract milestones that completed in the same simple transfer. Failed requests, reused connections and redirects need separate interpretation; a zero value may mean a phase was not reached.

`%{http_connect}` reports the HTTP CONNECT response code separately from the transfer's HTTP status. `--proxy-header` sends a header to the proxy; it does not report tunnel status. Prefer selected status and timing fields over verbose traces that may contain authentication data. These interpretations follow curl's [documented timing fields](https://curl.se/docs/manpage.html#-w).

## Check rotation against the promised policy

The exit IP is the public address observed by the destination. Query a provider-supported diagnostic endpoint or an endpoint you control that reports the caller's address. [ipify](https://www.ipify.org/) is one public option; check its current usage guidance before automating requests.

Keep a timestamped sequence of results before reducing them to unique-address counts. For each attempt, record the selected region, session setting, transport result and a validated IPv4 or IPv6 address. Exclude failed requests and invalid bodies from address counts, while reporting those failures separately. In Python, `ipaddress.ip_address(value.strip())` can validate a plain-text address response.

Compare that sequence with the provider's rotation policy. Rotation may occur per request, connection or time interval, and the pool may reuse an address. Ten requests do not require ten distinct exits unless the product explicitly promises that. A sticky session is intended to retain an exit for its documented duration; check its expiry and disconnection behaviour before treating a reassignment as a fault.

Use the country or region and session settings your workload needs. Repeat the test across the relevant session window. The [rotating vs sticky guide](/guides/rotating-vs-sticky-proxies) explains which behaviours to evaluate. The content benchmark below does not measure rotation.

## Choose a sample and observation window

Ten requests can uncover basic failures, but they offer little evidence about a latency distribution. Slow requests can hold up a queue; a mean alone can hide slow outliers.

Our proposed starting point is 200 requests per destination per configuration, spread over at least an hour. This is a test-design choice, not a universal reliability threshold. The 95th percentile of a small sample depends on very few observations. Report sample size, failure counts and the observation window, then repeat across times and conditions relevant to the workload.

When comparing providers, interleave their requests during the same window. This reduces differences caused by changing destination load without guaranteeing identical conditions. Keep request content, client settings and success criteria consistent. Test connection reuse and production concurrency separately from the sequential, fresh-process example below.

## A small content benchmark with a summary

Save this as `proxy-bench.py` and run `python3 proxy-bench.py`. It requires Python 3 and curl 8.3+. It uses the same private proxy environment variables as above. Set `BENCH_MODE=direct` for a direct run or leave it at `proxy`. Each run creates a new result directory and prints its path.

The default destination is a small illustrative page. For an actual comparison, set `BENCH_URL` to a permitted HTTPS endpoint and replace `BENCH_MARKER` with a meaningful expected string. Do not use URLs containing credentials or sensitive parameters. Adapt the content check for your real response format before quoting application success.

```python
import collections
import datetime
import json
import math
import os
from pathlib import Path
import statistics
import subprocess
import tempfile
import time

url = os.environ.get("BENCH_URL", "https://example.com/")
marker = os.environ.get("BENCH_MARKER", "<h1>Example Domain</h1>").encode()
mode = os.environ.get("BENCH_MODE", "proxy")
if not url.startswith("https://") or not marker or mode not in {"direct", "proxy"}:
    raise SystemExit("Use an HTTPS URL, a nonempty marker and direct or proxy mode")
route = ["--noproxy", "*"]
if mode == "proxy":
    route = ["--noproxy", "", "--proxy", os.environ["PROXY_URL"], "--proxy-basic",
             "--variable", "%PROXY_USERNAME", "--variable", "%PROXY_PASSWORD",
             "--expand-proxy-user", "{{PROXY_USERNAME}}:{{PROXY_PASSWORD}}"]
fields = ["http", "tunnel", "tcp_complete", "tls_complete", "ttfb", "total"]
write_out = "%{http_code} %{http_connect} %{time_connect} %{time_appconnect} %{time_starttransfer} %{time_total}"
count, window = 200, 3600.0
directory = Path(tempfile.mkdtemp(prefix="proxy-bench-", dir="."))
rows = []
started = time.monotonic()
with (directory / "results.jsonl").open("x") as log:
    for i in range(count):
        due = started + i * window / (count - 1)
        time.sleep(max(0, due - time.monotonic()))
        timestamp = datetime.datetime.now(datetime.timezone.utc).isoformat()
        with tempfile.TemporaryDirectory() as scratch:
            body = Path(scratch) / "body"
            result = subprocess.run(
                ["curl", "--disable", "--silent", "--connect-timeout", "5",
                 "--max-time", "20", "--retry", "0", *route,
                 "--output", str(body), "--write-out", write_out, "--url", url],
                capture_output=True, text=True)
            values = result.stdout.split()
            timings = dict(zip(fields, values)) if len(values) == len(fields) else {}
            if result.returncode:
                outcome = f"curl_{result.returncode}"
            elif not timings:
                outcome = "missing_measurements"
            elif timings["http"] != "200":
                outcome = "http_" + timings["http"]
            elif not body.exists() or marker not in body.read_bytes():
                outcome = "content_mismatch"
            else:
                outcome = "success"
        row = dict(attempt=i + 1, timestamp=timestamp, mode=mode,
                   exit_code=result.returncode, outcome=outcome, **timings)
        rows.append(row)
        log.write(json.dumps(row) + "\n")
        log.flush()

successful = [row for row in rows if row["outcome"] == "success"]
summary = {"attempts": len(rows), "successes": len(successful),
           "success_rate": len(successful) / len(rows),
           "outcomes": dict(collections.Counter(row["outcome"] for row in rows))}
for metric in ("ttfb", "total"):
    samples = sorted(float(row[metric]) for row in successful)
    summary[metric] = ({"median_seconds": statistics.median(samples),
                        "p95_seconds": samples[math.ceil(0.95 * len(samples)) - 1]}
                       if samples else None)
(directory / "summary.json").write_text(json.dumps(summary, indent=2) + "\n")
print(directory)
print(json.dumps(summary, indent=2))
```

The first request starts immediately; the 200th is scheduled 3,600 seconds after the first. Requests remain sequential. If a request takes longer than the spacing, later starts slip and the run takes longer. Actual start timestamps are retained in the log. There are no automatic retries or followed redirects, so a 3xx response is an HTTP failure in this example. If your workload requires redirects, define which destinations and final content are acceptable before extending the harness.

Each log row contains the attempt number, UTC start timestamp, route mode, curl exit code, outcome, HTTP and CONNECT status, and four cumulative timing milestones in seconds. The harness retains neither response bodies nor credentials in its result files. Keep a separate private run note for the provider, pool, target, client version and session configuration.

The success-rate denominator is **all attempted requests**, including timeouts and content failures. curl exit codes such as 28 (timeout), 7 (connection failure) and 56 (receive failure) describe broad failure categories, not a confirmed faulty component. Use the [official exit-code reference](https://everything.curl.dev/cmdline/exitcode.html) when investigating.

Latency statistics use only successful responses, and the summary keeps the failure counts beside them. The median is the middle value, averaging the middle pair when necessary. The 95th percentile uses the nearest-rank method: sort the successful samples and select rank `ceil(0.95 × sample_count)`, counting from one. With no successes, both latency summaries are `null`.

For example, four successes from five attempts give an 80% success rate. If their TTFB values are 0.10, 0.12, 0.20 and 0.40 seconds, the median is 0.16 seconds and the nearest-rank p95 is 0.40 seconds. These are invented values to explain the calculation, not measured proxy results.

Start with a short controlled check of the harness and your content rule, then run the agreed observation window. Publish the configuration, sample size, failures and success definition alongside any latency comparison. The direct baseline above is illustrative; it does not establish a proxy ranking.

## Sources

- [curl manual: --write-out variables and proxy configuration](https://curl.se/docs/manpage.html#-w)
- [everything curl: exit codes](https://everything.curl.dev/cmdline/exitcode.html)
- [RFC 9110: HTTP Semantics, section 9.3.6 CONNECT](https://www.rfc-editor.org/rfc/rfc9110#name-connect)
- [ipify: a simple public IP address API](https://www.ipify.org/)

## Know when ipvolt access opens.

ipvolt is in development. Leave your email and we’ll notify you once when access opens.

Consent: One email when access opens. Nothing else.

[Notify me](https://ipvolt.com/blog/what-a-proxy-benchmark-should-measure#waitlist-blog-end). Use the email form on this page to join the interest list.

[Privacy](https://ipvolt.com/privacy)

## Related posts

- [Amazon listing updates: accepted is not live](https://ipvolt.com/blog/amazon-listing-update-reconciliation.md) (Analysis, Sep 14, 2026, 8 min read): Diagnose accepted Amazon listing updates by separating submitted attributes, live offers, inventory and buyability, with a practical reconciliation matrix.
- [Bookmaker odds feeds: validate before comparing](https://ipvolt.com/blog/bookmaker-odds-feed-validation.md) (Analysis, Sep 14, 2026, 7 min read): Compare bookmaker odds only after checking market identity, settlement rules, timestamps and status. Use a local validator to expose false comparisons.
- [Cloudflare AI Agent Blocks: What to Fix First](https://ipvolt.com/blog/cloudflare-ai-agent-proxy-setup.md) (Analysis, Sep 14, 2026, 9 min read): Diagnose Cloudflare failures in AI-agent jobs, configure a stable browser proxy, and reject challenges or invalid content before RAG ingestion.

## Related guides

- [Use a proxy with curl](https://ipvolt.com/guides/curl-proxy-setup.md): Test an HTTP proxy with curl, separate proxy authentication from destination authentication, and read connection failures without exposing credentials.
- [Troubleshoot proxy timeouts one stage at a time](https://ipvolt.com/guides/proxy-timeout-troubleshooting.md): Separate proxy DNS, TCP, CONNECT, TLS and response delays with curl timings, then set request deadlines and decide whether a retry is safe.
- [Rotating vs sticky proxies: plan for session continuity](https://ipvolt.com/guides/rotating-vs-sticky-proxies.md): Choose rotation around the smallest complete workflow, distinguish exit affinity from cookies, and test what happens when a sticky session ends early.

## About ipvolt

Technical analysis from the ipvolt team.

ipvolt access is not open yet.