Before buying more proxy traffic for a content monitor, inspect the headers your client actually sends. On our live website, adding Cache-Control: no-cache changed unchanged-page checks from body-free 304 responses to full 200 downloads, even though the requests carried the matching ETag.
In a controlled comparison, 12 conditional requests without that override transferred zero response-body bytes; 12 with it transferred 271,968 bytes of unchanged HTML. The six initial downloads needed to obtain the ETags cost another 135,984 bytes and are included in the published records.
That is a finding about one deployed Next.js 16.3.4 site and a particular request policy. It is not advice to remove freshness requirements from a monitor. The useful decision is whether to change the client policy, choose a smaller representation, or investigate the server's revalidation behavior.
The task: monitor known content without downloading it unnecessarily
We tested two existing ipvolt articles: the proxy benchmark guide and proxy error explanation, each as HTML and Markdown. Our monitor watched the article title and two specified section headings. It also checked media types and, for HTML, the page title and canonical URL.
Those checks establish that the watched information is present. They do not make Markdown a substitute for layout, JavaScript behavior or a working signup journey. The exact assertions are in the study configuration.
Collection used authenticated third-party proxy connections targeting the US, UK and Netherlands. The initial experiment also included a direct control. Country-check responses reported the requested countries at the checkpoints; an exit address changed during the initial US run, so these are route observations rather than fixed-IP or country-wide reliability results. The connection provider is omitted; this was not a test of an available ipvolt proxy service.
All three HTTP experiments ran on 11 September 2026. They made 234 requests and transferred 2,975,066 response-body bytes, including initial downloads, verification downloads and unsuccessful hypotheses. Every attempted request completed without a curl transport error. Additional geographic probes and browser QA are recorded separately in the method note; they are outside that HTTP-experiment total.
The smaller first download did not answer the caching question
The initial experiment requested gzip consistently and measured received body bytes. Across twelve initial observations per representation, the sizes were:
| Article representation | Received body bytes per initial request |
|---|---|
| Benchmark HTML | 26,521 |
| Benchmark Markdown | 7,056–7,061 |
| Error guide HTML | 18,807 |
| Error guide Markdown | 5,081 |
Both HTML endpoints supplied ETags. Neither Markdown endpoint supplied an ETag or Last-Modified validator in this experiment. HTML conditional checks without a Cache-Control override returned 304; the runner reused the matching, previously accepted body instead of interpreting the empty response as a new document.
The exploratory run made 144 requests: 120 full 200 responses and 24 accepted 304 responses. It skipped 36 conditional slots because the Markdown had no validator or the homepage was marked no-store. Skipped requests are not counted as successes or zero-byte network observations.
We then ran five successive checks per representation through each of the three proxy routes. Each check explicitly sent Cache-Control: no-cache, and the client retained eligible validators between checks. All 60 requests returned 200, including the 24 HTML rechecks carrying a matching ETag. The expectation that repeated HTML validation would avoid downloading the body did not hold under this policy.
Isolating the header that changed the response
To investigate, we held the URL, route configuration, Accept, language, gzip preference and exact seed ETag constant. For each of the six route/article combinations, we made one seed request and four conditional requests. The order was A/B/B/A or B/A/A/B, where A omitted a Cache-Control override and B sent no-cache.
| Conditional request policy | Attempts | Responses | Received body bytes |
|---|---|---|---|
| Matching If-None-Match; no Cache-Control override | 12 | 12 × 304 | 0 |
| Same If-None-Match; Cache-Control: no-cache | 12 | 12 × 200 | 271,968 |
All 24 conditional observations matched the seed's watched values and decoded content. Including the six seeds, this controlled experiment transferred 407,952 body bytes across 30 requests. It used balanced ordering, not random sampling, and establishes the observed behavior of these endpoints during this short window.
The implementation explains a plausible mechanism. We checked the actual deployed dependency files against the local Next.js 16.3.4 installation: the relevant source hashes matched. Its ETag response helper calls fresh before returning 304; the bundled freshness check returns false when the request contains no-cache, before evaluating the matching ETag. A separate local function test reproduced the same branch. Next.js response helper, fresh implementation.
This is stack-specific behavior consistent with the live results. HTTP does not define no-cache as a universal instruction to download a complete body. Nor does receiving 304 prove that the request reached the origin rather than a responding cache. HTTP caching.
The useful optimization depends on the freshness you need
For the five-check experiment that explicitly used no-cache, the actual totals were:
| Representation | Requests, including initial downloads | Received body bytes |
|---|---|---|
| HTML | 30 | 679,920 |
| Markdown | 30 | 182,070 |
| Difference | Same request count | 497,850 |
Markdown transferred 73.2% fewer response-body bytes for the watched content under that policy. It did not reduce the request count. These are measured payload differences, not a measured invoice saving or a forecast of monthly traffic. curl's download-size metric excludes headers and other network overhead. curl payload accounting.
There is another option when freshness requirements allow it: the Markdown responses advertised public, max-age=300, must-revalidate. A cache may reuse a still-fresh stored representation without making a request. Missing validators do not make it uncacheable. Our runners deliberately checked over the network; they did not implement that freshness-based scheduling policy. Freshness and reuse.
Use the result to choose the next action:
| Your monitoring requirement | Action supported by this case |
|---|---|
| Reuse within the server's freshness window is acceptable | Evaluate a local freshness-aware cache before scheduling another download. |
| Ask the responding server or cache to validate every check | Test the final request headers against the actual endpoint; do not assume an ETag guarantees a body-free response. |
| Need the watched text, and each check returns a full body | Compare compressed HTML with a text or Markdown representation that contains those same facts. |
| Need rendered layout or an interactive journey | Keep browser checks; the smaller text representation does not cover that job. |
Do not strip no-cache solely to reproduce the zero-byte row. The two header policies can have different freshness consequences. Establish what your monitor must detect, then measure the permitted implementation.
Reproduce the result and adapt the check
Download the offline analysis script and the sanitized recordings into the same directory. Python 3.10 or newer is sufficient; this command makes no network requests:
python3 reproduce.py recordings.zipIt recomputes the attempt counts, body totals, header comparison and Markdown percentage. Compare its output with the recorded result. The dataset includes timestamps, request policies, selected response headers, content checks and hashes. Credentials, exit IPs, unfiltered headers and raw body captures remain private; replay verifies the published arithmetic rather than independently authenticating historical responses.
The download instructions link the exact collection programs, configurations and 27 passing offline tests. The live programs require curl 8.4 or newer, preserve TLS verification, bound requests and body transfer, and pass proxy authentication privately through curl's input. Their allowlist is restricted to this case's public site; adapt both the allowlist and content assertions for a property you operate.
We did not observe a real content update or measure detection delay. Decoded content stayed stable for every tested endpoint; a few gzip byte streams differed despite identical decoded text. That is a reason to compare the content your monitor cares about, not evidence that this study eliminated operational false alerts. This is one site, two articles and a short observation window—not a provider ranking or a universal caching benchmark.
Method: agent-assisted collection and analysis, with separate evidence, editorial and search reviews. All reported network observations were actually executed; the offline tests exercise controlled fixtures and are labeled separately.
Notify me when access opens. One email when access opens. Nothing else.