curl --compressed: which clients skip compression

curl sends no Accept-Encoding header unless you pass --compressed. GNU Wget 1.x and Python's urllib.request send Accept-Encoding: identity, and Node's node:http, PHP's ext-curl, Guzzle and Java's java.net.http.HttpClient send no header at all. A server that compresses only when asked then returns the full, uncompressed text, and every one of those bytes crosses your proxy. Requests, httpx, aiohttp, Scrapy, Node's built-in fetch, axios, got, Go and browsers already ask for compression: for them there is nothing to switch on, and at most an optional br decoder trims a little more from some pages.

Each default below was recorded by a local lab that ipvolt ran on 27 September 2026, with the versions listed under each table. The results are synthetic: a server and a proxy on 127.0.0.1 served a local test page, so no byte count here comes from a public site or a provider bill.

Who asks for compression by default

Sends no Accept-Encoding, or identity

A server that compresses only when asked sends these clients uncompressed text. No header: curl, node:http, node:https, PHP ext-curl, Guzzle, Laravel Http, Java HttpClient. identity: Wget, urllib, http.client.

Versions run: curl 8.7.1, 8.22.0; Wget 1.25.0; urllib.request, http.client (Python 3.13.15, 3.14.7); node:http, node:https (Node.js 22.23.3, 24.21.0, 26.10.0); PHP 8.5.8 ext-curl (libcurl 8.20.0); Guzzle 8.2.0, Laravel Http 13.33.0; Java 25.0.4.1 HttpClient.

• curl: a hand-set -H 'Accept-Encoding: gzip' is sent, but the body stays compressed.
• Wget: with --compression=auto it sent gzip and decoded the reply.
• urllib.request and http.client return compressed bodies as they arrived.
• node:http and node:https return compressed bodies as they arrived.
• PHP ext-curl: with CURLOPT_ACCEPT_ENCODING set to '', this libcurl sent deflate, gzip, br, zstd. A hand-set header is sent but not decoded.
• Laravel Http: withOptions(['decode_content' => 'gzip']).
• Java HttpClient never decodes: wrap the body in GZIPInputStream when the reply is gzip.

Already asks for compression

Nothing to change: these clients ask for compression and decode the reply.

Versions run: Requests 2.34.2; httpx 0.28.1; aiohttp 3.14.3; Scrapy 2.19.0; built-in fetch (Node.js 22.23.3, 24.21.0, 26.10.0); Go 1.27.1 net/http.

• Requests: brotli adds br; on Python 3.13, backports.zstd adds zstd.
• httpx: brotli adds br and zstandard adds zstd.
• aiohttp: Brotli adds br; on Python 3.13, backports.zstd adds zstd.
• Built-in fetch over http://: gzip, deflate.
• Go: setting the header yourself turns decoding off.

If your client already asks for compression, confirm it with the byte check below, and read the section on copied browser headers before you add any header by hand. Compression is worth nothing for images, video and other payloads that are already compressed, whichever client you use.

Every row is a lab run, not a reading of the source: 567 recorded cells, none marked source-read. tables.md lists the records behind each row. "Wget" here means GNU Wget 1.x. GNU Wget2 asks for every coding it was built with by default (wget.c), and Fedora 40 and later install wget2 as the wget command (Fedora change); that is source-read, and the lab did not run Wget2.

curl: no header until you pass --compressed

Plain curl URL sends no Accept-Encoding at all. In libcurl, CURLOPT_ACCEPT_ENCODING defaults to NULL, which "makes libcurl not send an Accept-Encoding: header and not decompress received contents automatically", and the curl tool changes it only for --compressed. Neither curl build in the lab sent the header, directly, through an http:// proxy or through CONNECT. curl#11091 shows the same thing with an nc capture of curl 7.88.1.

--compressed asks for every coding your build can decode, then decodes the reply:

• The macOS system curl 8.7.1, built with zlib only, sent deflate, gzip.
• Homebrew curl 8.22.0, built with brotli and zstd, sent deflate, gzip, br, zstd.

To see what yours will ask for, read the Features: line of curl -V: libz means gzip and deflate, brotli adds br, and zstd adds zstd.

Three details decide whether it helps:

• It is "a request, not an order; the server may or may not deliver data compressed" (man page). curl reports no error when a server ignores it and has no switch to fail in that case (curl#7516), so check the bytes as shown below.
• A hand-set -H 'Accept-Encoding: gzip' is sent, but curl does not decode the reply. In the lab, curl output the 17,750-byte gzip stream instead of the 100,129-byte page. Use --compressed, not a header.
• --no-compressed turns it off again, for example after a --compressed line in .curlrc.

When the lab server sent br or zstd that the macOS build cannot decode, curl --compressed stopped with exit code 61:

curl: (61) Unrecognized content encoding type. libcurl understands deflate, gzip content encodings.

Decompression has a cost of its own. The man page warns that "even tiny transfers might be expanded and generate a huge amount of bytes" and suggests --max-filesize. That option stops a transfer that grows through --compressed decompression only "Since 8.20.0" (man page), so it does not protect the macOS system curl 8.7.1. That limit is documented; the lab did not test it.

Without --compressed, a server that compresses anyway leaves you with raw gzip bytes, as in curl#2836; that is the familiar "binary output" case, and --compressed fixes it. For the proxy flags themselves, see use a proxy with curl.

Python: urllib sends identity; Requests, httpx, aiohttp and Scrapy compress

• urllib.request and http.client send Accept-Encoding: identity on CPython 3.13.15 and 3.14.7. http.client adds it unless you set your own, with the comment "we don't support encodings such as x-gzip or x-deflate" (client.py), and neither module decodes a reply. In the lab, urllib.request returned gzip, br and zstd replies as they arrived; http.client, which urllib.request sends its requests through, ran only with its default header, so for http.client this rests on the source. Setting the header yourself only gets you compressed bytes to decode by hand, so switch clients instead. A script that must stay on the standard library can send Accept-Encoding: gzip and, when the reply carries Content-Encoding: gzip, decode the body with gzip.decompress(); that route was not run in the lab.
• Requests 2.34.2 (with urllib3 2.8.0) takes its default from urllib3 (utils.py): gzip, deflate on Python 3.13.15 and gzip, deflate, zstd on 3.14.7, where urllib3 uses the standard library's compression.zstd. Installing brotli adds br; on 3.13, backports.zstd adds zstd. Since urllib3 2.6.0, the separate zstandard package no longer enables zstd (request.py; source-read, not run). The proxy side is in configure Python Requests.
• urllib3 on its own, through urllib3.request() or a PoolManager, adds no compression header. Its request path passes through to http.client (connection.py), which by the source means identity. This is source-read; the lab ran Requests, not urllib3 alone.
• aiohttp 3.14.3 sends gzip, deflate on 3.13.15 and gzip, deflate, zstd on 3.14.7; Brotli adds br, and on 3.13 backports.zstd adds zstd. It is the only Python client here that raises an exception on a coding it cannot decode; the exact message is below.
• Scrapy 2.19.0 sends gzip, deflate, br, zstd on both. Since Scrapy 2.18.0, brotli and Zstandard support are required, "so br and zstd are always included in the Accept-Encoding header of requests" (release notes).

httpx and zstd on Python 3.14

httpx 0.28.1 sends gzip, deflate on both Pythons. It adds br with brotli and zstd only with the third-party zstandard package; unlike Requests and aiohttp, it does not use the standard-library zstd of Python 3.14 (_decoders.py). The HTTPX documentation installs both decoders with pip install "httpx[brotli,zstd]". None of that is needed for compression itself, since gzip is always requested.

It matters when a server sends a coding httpx cannot decode: httpx returns the compressed bytes and raises nothing. On Python 3.14.7 without zstandard, a zstd reply reached the caller as the 19,298-byte zstd stream. Proxy configuration is in the HTTPX async proxy guide.

Node.js: built-in fetch compresses, node:http does not

Built-in fetch (undici) chooses the header by URL scheme and release line (source, v26.10.0; v24.21.0):

• https:// URLs: br, gzip, deflate on Node.js 22.23.3 and 24.21.0, and br, gzip, deflate, zstd on 26.10.0.
• http:// URLs: gzip, deflate on all three.
• Any request with a Range header: identity on all three.

All three decoded gzip and br. zstd is where they differ, including zstd a server sends unasked:

• Node.js 22.23.3 fetch has no zstd decoder and returned the compressed bytes with no error (source).
• Node.js 24.21.0 fetch decoded zstd it had not asked for. A body made of four concatenated zstd frames came back as the first frame only: 25,033 of 100,129 bytes, with no error. axios 1.20.0 and got 16.0.0 also returned only the first frame on Node.js 22.23.3 and 24.21.0.
• Node.js 26.10.0 decoded all four frames with fetch, axios and got.

node:http and node:https send no Accept-Encoding on any of the three releases and never decode; Node's core HTTP client does not set the header (_http_client.js). Node's zlib documentation shows the manual route: set the header yourself and pipe the response through a decompressor. Switching to fetch, got or axios is less code.

Built-in fetch vs the node-fetch package

The npm package node-fetch is a separate client with its own defaults (node-fetch#1556). On Node.js 24.21.0, node-fetch 3.3.2 sent gzip, deflate, br and node-fetch 2.7.0 sent gzip,deflate, and both decoded the reply. axios 1.20.0 sends gzip, compress, deflate, br and adds zstd only with transitional: { advertiseZstdAcceptEncoding: true } (http.js). got 16.0.0 sent gzip, deflate, br, zstd on all three releases (index.ts). None of them needs a change. The proxy dispatcher for built-in fetch is covered in use a proxy with Node.js fetch.

Go, Wget, PHP and Java

Go net/http asks for gzip only

Go 1.27.1 sends Accept-Encoding: gzip, decodes the reply and removes Content-Encoding and Content-Length from the response it gives you (transport.go). It sends no header for HEAD requests, for requests with a Range header, or with DisableCompression: true. Two consequences:

• Setting Accept-Encoding yourself turns the decoding off: "if the user explicitly requested gzip it is not automatically uncompressed". With req.Header.Set("Accept-Encoding", "gzip"), the lab's Go client received the 17,750-byte gzip stream.
• net/http has no br or zstd decoder. A server that sent them unasked delivered raw bytes.

GNU Wget 1.x sends identity

Wget 1.25.0 sends Accept-Encoding: identity. For --compression, the manual says of "none": "This is the default" (manual); the same page's .wgetrc list still calls auto the default, but the lab run matches none. With --compression=auto, Wget sent gzip and decoded gzip, and nothing else: br or zstd sent unasked arrived raw. Wget2 differs, as noted above.

PHP: Guzzle, Laravel Http and ext-curl send no Accept-Encoding

• PHP ext-curl (PHP 8.5.8 with libcurl 8.20.0) sends nothing, because CURLOPT_ACCEPT_ENCODING "Defaults to null" (PHP manual). CURLOPT_ACCEPT_ENCODING => '' asks for every coding the linked libcurl supports, deflate, gzip, br, zstd in this build, and decodes. An Accept-Encoding line in CURLOPT_HTTPHEADER is sent but not decoded.
• Guzzle 8.2.0 removes the header on purpose. Its cURL handler turns on every decoder, then adds an empty Accept-Encoding: line to stop curl sending one, with the comment that this "will be interpreted as 'Accept-Encoding: *'" (CurlFactory.php). The same code is in every tag checked from 6.5.0 to 8.2.0 (source-read). Give decode_content a string, such as 'decode_content' => 'gzip', and Guzzle sends it as the header (Client.php); 'gzip, deflate, br, zstd' also worked with the cURL handler. By default the cURL handler still decodes gzip, br or zstd that a server sends unasked. Guzzle's StreamHandler, which uses PHP streams instead of ext-curl, decoded gzip but returned br and zstd raw.
• Laravel Http (illuminate/http 13.33.0) builds a plain Guzzle client and inherits all of this (PendingRequest.php). Both withOptions(['decode_content' => 'gzip']) and withHeaders(['Accept-Encoding' => 'gzip']) sent gzip and decoded the reply.

Java HttpClient never decodes

java.net.http.HttpClient on Temurin 25.0.4.1 sent no Accept-Encoding and did not decode a gzip reply that the server sent unasked. To get compression, send Accept-Encoding: gzip yourself and read the body through java.util.zip.GZIPInputStream when the response carries Content-Encoding: gzip, as the lab's Java client does. No other Java client was run.

Accept-Encoding: identity, no header, and garbled output

RFC 9110 separates three request states:

• No header (curl, node:http, ext-curl, Guzzle, Java): "any content coding is considered acceptable by the user agent".
• identity (urllib, Wget 1.x): "a synonym for 'no encoding'".
• An empty value: "the user agent does not want any content coding in response".

So the standard allows a server to compress a reply to a request with no header, and "compress only when asked" is server practice, not an RFC rule. Server and CDN documentation describes that practice. Cloudflare picks gzip, Brotli, Zstandard or no compression "depending on" the values in the request's accept-encoding header, the plan and any matching compression rule (Cloudflare, page updated 17 April 2026). Apache's mod_deflate sends Vary: Accept-Encoding so that compressed content is not "sent to a client that will not understand it" (mod_deflate). Guzzle's comment relies on the RFC reading; with servers that follow that practice, a missing header gets the uncompressed body. The lab server followed the same practice. How often real servers compress unasked was not measured.

Some do. nginx's gzip_static always serves the gzipped file "without checking if the client supports it" (nginx), and curl#2836 reports a real host that did it. When the lab server sent gzip unasked, curl without --compressed, Wget, urllib, node:http, PHP ext-curl, Java and Go with DisableCompression all handed the caller the 17,750-byte gzip stream; Guzzle and Laravel Http decoded it. If a response from one of those clients looks like binary garbage, check its Content-Encoding first.

Verify the bytes on your own target

For a client without a byte-count snippet here, run the curl comparison below against the same URL through the same proxy, and compare its Content-Encoding with your client's. This measures curl's response; a different coding or server response is not a byte measurement of your client.

The curl, Requests, httpx and node-fetch-bytes.mjs snippets below print wire body bytes: the response body as it crossed the network, before any decoding. Response headers, TLS records, the CONNECT exchange and retries are not included, so this is not what a provider bills. In eight lab pairs (tables.md lists them), the same client fetched the test page over CONNECT with and without compression, and those extras added 2,343 to 3,520 bytes per response on the proxy's client leg. They lowered the ratio, for example from 5.19× on the body to 4.56× on the proxy leg for curl 8.22.0, but not the saving: in each of those pairs, the proxy leg shrank by the body difference plus 87 bytes. To meter a whole run through your proxy and convert it at your own rate, use Scrapescope.

The Go snippet prints the decoded size and resp.Uncompressed, which says whether Go decoded the reply, and the linked node-fetch-verify.mjs prints the header fetch sent; neither prints wire bytes. Each snippet ran unchanged in the lab except that https://example.com/ became a loopback URL, and the outputs shown are from those runs; the Go snippet was also compiled with one lab file that trusts the lab's CA, described in the method section. Set PROXY_URL to your proxy. If you need to trust a private CA, use the client's own setting, such as --cacert, NODE_EXTRA_CA_CERTS or Go's tls.Config.RootCAs; never turn certificate verification off.

Verify with curl

# Wire body bytes and the Content-Encoding the server chose (curl 7.84.0+ for %header{}).
curl -sS -o /dev/null -x "$PROXY_URL" \
  -w '%{size_download} %header{content-encoding}\n' https://example.com/
curl -sS -o /dev/null -x "$PROXY_URL" --compressed \
  -w '%{size_download} %header{content-encoding}\n' https://example.com/

With Homebrew curl 8.22.0 against the lab's test page, the first line printed 100129 and no coding, and the second 19298 zstd; the macOS system curl 8.7.1 printed 17750 gzip on the second line. curl's size_download is "the size of the body/data that was transferred, excluding headers", and %header{} needs curl 7.84.0 or later (write-out). The ratio for that page is the first number divided by the second.

Verify with Requests and httpx

import os

import requests

proxies = {"http": os.environ["PROXY_URL"], "https": os.environ["PROXY_URL"]}
with requests.get("https://example.com/", proxies=proxies, stream=True) as r:
    wire = r.raw.read(decode_content=False)  # body bytes as sent, still encoded
    print(r.headers.get("content-encoding"), len(wire))

Requests printed gzip 17750 on Python 3.13.15 and zstd 19298 on 3.14.7. For httpx, r.num_bytes_downloaded counts the bytes as read and len(r.content) the decoded body:

import os

import httpx

with httpx.Client(proxy=os.environ["PROXY_URL"]) as client:
    r = client.get("https://example.com/")
    # wire body bytes (as received) vs decoded bytes your code sees
    print(r.headers.get("content-encoding"), r.num_bytes_downloaded, len(r.content))

It printed gzip 17750 100129 on Python 3.14.7, and zstd 19298 100129 with zstandard installed. urllib never decodes, so the length of what you read from it is already the wire body.

Verify with Node.js

For built-in fetch, Node's resource timing entry for the URL gives the wire body size:

// Wire body bytes of a built-in fetch, from Node's resource timing entry for the URL.
// Run: NODE_USE_ENV_PROXY=1 HTTPS_PROXY="$PROXY_URL" node node-fetch-bytes.mjs
import { setImmediate } from 'node:timers/promises';

const url = new URL('https://example.com/');
const res = await fetch(url);
const body = await res.arrayBuffer(); // already decoded by fetch

// Node adds the entry after the body has been read, so give the event loop a turn.
let entry;
for (let turn = 0; !entry && turn < 100; turn++) {
  await setImmediate();
  entry = performance.getEntriesByType('resource').findLast((e) => e.name === url.href);
}
performance.clearResourceTimings(); // the buffer stops at 250 entries unless you clear it

// encodedBodySize: the body as it crossed the network, before decoding (no headers, TLS or CONNECT)
console.log('content-encoding', res.headers.get('content-encoding'),
  'wire body bytes', entry?.encodedBodySize, 'decoded bytes', body.byteLength);

Through the lab's CONNECT proxy it printed content-encoding br wire body bytes 18451 decoded bytes 100129 on Node.js 22.23.3 and 24.21.0, and content-encoding zstd wire body bytes 19298 decoded bytes 100129 on 26.10.0, each matching what the lab server sent. In 18 further lab cells on the three releases (direct http://, direct https:// and through the CONNECT proxy, each with and without compression), encodedBodySize equalled the body bytes the server sent every time; tables.md lists the readings. The entry never existed straight after the body was read, only after one event-loop turn, which is why the snippet waits. The snippet then clears the entries, because Node's resource timing buffer holds only 250 by default (observe.js). Do not use transferSize instead: it was encodedBodySize plus a fixed 300 bytes in every cell, while through the proxy the client leg carried 3,431 to 3,520 bytes more than the body.

node-fetch-verify.mjs prints the header fetch sent (sent accept-encoding: br, gzip, deflate, zstd on Node.js 26.10.0). For node:https, which never decodes, node-http-verify.mjs counts the body as it arrives, which is the wire body.

Verify with Go

// Did Go ask for gzip and decode it for you? resp.Uncompressed says so.
// Run: go run go-verify.go   (with PROXY_URL set in the environment)
package main

import (
	"fmt"
	"io"
	"net/http"
	"net/url"
	"os"
)

func main() {
	proxy, err := url.Parse(os.Getenv("PROXY_URL"))
	if err != nil {
		panic(err)
	}
	t := http.DefaultTransport.(*http.Transport).Clone()
	t.Proxy = http.ProxyURL(proxy)
	resp, err := (&http.Client{Transport: t}).Get("https://example.com/")
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()
	body, err := io.ReadAll(resp.Body)
	if err != nil {
		panic(err)
	}
	// true: Go sent "Accept-Encoding: gzip" itself, received gzip and decoded it,
	// then removed Content-Encoding and set ContentLength to -1.
	fmt.Printf("uncompressed=%v content-encoding=%q content-length=%d decoded-bytes=%d\n",
		resp.Uncompressed, resp.Header.Get("Content-Encoding"), resp.ContentLength, len(body))
}

With Go 1.27.1 it printed uncompressed=true content-encoding="" content-length=-1 decoded-bytes=100129. resp.Uncompressed only says that Go decoded the reply (response.go); the response no longer carries its wire size. To count wire bytes, set the header yourself, which leaves the body compressed, measure it, then decode it with compress/gzip.

Through a proxy: CONNECT vs absolute-form

For an https:// URL, the client opens a CONNECT tunnel and sends its request inside TLS. With end-to-end TLS and no TLS interception, the proxy cannot read or change Accept-Encoding; it relays the encrypted response. The lab recorder did not parse headers inside CONNECT tunnels. For an http:// URL, most clients send the request itself to the proxy in absolute form, as in GET http://host/path, so the proxy reads every header: the lab proxy saw curl 8.22.0's deflate, gzip, br, zstd with --compressed, Wget's identity and Go's gzip. An intermediary on that path could also rewrite the header or the body. The lab proxy did neither, and no real proxy was tested.

Two clients behaved differently with http:// URLs:

• Node.js 22.23.3 and 24.21.0 built-in fetch, with NODE_USE_ENV_PROXY=1, tunnelled http:// URLs through CONNECT. The lab recorder did not parse the inner header, but that HTTP request was still plaintext and inspectable: CONNECT itself does not encrypt traffic. Node.js 26.10.0 sent absolute form.
• Guzzle's StreamHandler sent an origin-form request line to the proxy unless request_fulluri was set, and the lab proxy rejected it with a 400.

What counts toward a bill is each provider's rule. Decodo, for example, documents download traffic as "every byte received from the target website, including response headers and cookies", says that for HTTPS "the bytes tunneled through the proxy also count toward proxy traffic usage", and gives an example in which its own test endpoint took 0.85 kB of download over http:// against 3.96 kB over https:// (Decodo help, read 26 September 2026, page modified 23 April 2026). Decodo does not say whether it counts compressed or decoded bytes. For HTTPS with end-to-end TLS inside CONNECT, the proxy counts the encrypted bytes crossing the tunnel, including the compressed response and TLS overhead; that is an inference, not Decodo's statement. Scraping APIs and unblockers that send the request to the target for you choose their own headers and meter in their own way, so check your provider's documentation.

Copying a browser's Accept-Encoding

You may be tempted to paste a browser's Accept-Encoding: gzip, deflate, br, zstd into a scraper. That asks the server for codings your client may not decode, and in several clients a hand-set header also switches decoding off. With exactly that header, the lab server chose zstd:

• curl (both builds), urllib, node:http, Go, PHP ext-curl, Java HttpClient and Guzzle's StreamHandler returned the compressed bytes: a normal response with an unreadable body.
• Requests and httpx did the same whenever the matching decoder was missing. On Python 3.13, bare Requests returned both zstd and br raw. On 3.14 it decoded zstd but returned br raw when the server sent br. httpx without zstandard returned zstd raw on both Pythons, with no exception.
• Node.js 22.23.3 fetch returned zstd raw; 24.21.0 and 26.10.0 decoded it.
• Guzzle's cURL handler decoded both zstd and br.
• aiohttp raised an exception whenever it lacked the decoder: br without Brotli, and zstd on Python 3.13 without backports.zstd. On 3.14 it decoded zstd.

The fix is to let the client write the header. If you want br or zstd, install the decoder package your client uses, such as brotli, backports.zstd on Python 3.13 or httpx[brotli,zstd]; the client then adds the token itself.

aiohttp 3.14.3: Can not decode content-encoding: brotli (br). Please install `Brotli`

When a server sends br and the Brotli package is missing, aiohttp 3.14.3 raises the first of these, on Python 3.13.15 and 3.14.7 alike. The second is the zstd version, raised on Python 3.13.15 without backports.zstd (loopback URLs shortened):

aiohttp.client_exceptions.ClientResponseError: 400, message='Can not decode content-encoding: brotli (br). Please install `Brotli`', url='…'
aiohttp.client_exceptions.ClientResponseError: 400, message='Can not decode content-encoding: zstandard (zstd). Please install `backports.zstd`', url='…'

The 400 is aiohttp's own status for a response it cannot decode; the lab server had answered 200 (http_parser.py). Installing Brotli, or backports.zstd for the second, fixed it in the lab: aiohttp then decoded br, and zstd including a body of four zstd frames. The shorter Can not decode content-encoding: br comes from a different path in the same file: a decoder is installed, but the body failed to decompress.

Servers that mishandle advertised zstd

Advertising a coding can itself break a request. OpenSearch 2.19.0 hung when a request's Accept-Encoding included zstd, as it does from curl builds with zstd support. It was fixed for 2.19.1 and 3.0.0 (OpenSearch#17339). If one host times out only for zstd-capable clients, a request that asks for gzip only will show whether zstd negotiation is the problem.

Where compression does not help

• Payloads that are already compressed, such as images, video and archives. "Media files such as images that are already compressed do not benefit from HTTP compression" (Web Almanac 2021).
• Responses a server or CDN leaves uncompressed. Cloudflare, for example, documents compressing only 200, 403 and 404 responses, with a minimum size of 48 bytes for gzip and 50 bytes for Brotli and Zstandard (Cloudflare).
• Range and HEAD requests. Go sends no header for either, and Node.js fetch sends identity whenever a Range header is set.
• Uploads. --compressed and its equivalents work on downloads; for request bodies "there is no standard way to do compression" (Everything curl).
• Browser automation. Browsers negotiate compression themselves: MDN gives gzip, deflate, br, zstd as the typical browser value (MDN), and Chrome has decoded zstd by default since version 123 (Chrome Platform Status), so in Playwright or Puppeteer the bytes to cut are in what the browser downloads, not in this header.

For pages you fetch again and again, conditional requests are a separate lever: a 304 Not Modified response carries no body at all (RFC 9110; ETag monitoring).

What it is worth

Only clients in the first table have compression to switch on. For them:

saved $ ≈ pages × uncompressed text bytes per page × (1 − 1/ratio) ÷ 10⁹ × your $ per GB

Take the ratio from the verify step on your own pages, as bytes without compression divided by bytes with it, and the rate from your own plan.

Compression lowers the bill only when you pay per byte. On a flat plan priced by Mbps or threads the price stays the same, though more compressed pages fit through the channel. What unlimited residential proxies cost per GB compares the two ways of billing.

An invented example, not a measurement: 100,000 pages of 100 KB (100,000 bytes) of HTML are 10 GB uncompressed. At a ratio of 4, compression saves 7.5 GB; at a ratio of 10, it saves 9.0 GB. At an invented rate of $3 to $8 per GB, that is about $22.50 to $72 for one crawl of those pages.

For a client that already asks for compression, switching it on saves nothing, because it is already on. An optional br decoder for Requests, httpx or aiohttp can trim a little more on some pages: on the lab's 23 documentation pages, br bodies totalled 277,665 bytes against 325,873 for gzip, about 15% fewer, but on the synthetic page br was larger than gzip (18,451 against 17,750 bytes).

The lab's own ratios are not a forecast. The synthetic test page shrank 5.64× with gzip and 5.19× with zstd. In this corpus of 23 CPython 3.14.7 documentation pages, the uncompressed bodies were 6.67× their gzip size, 6.61× their zstd size and 7.83× their br size. Those numbers describe those files only.

The formula counts body bytes; in the lab, headers, TLS and the CONNECT exchange barely changed with compression, as the verify section shows. How a provider turns bytes into a bill (GB or GiB, minimums, failed requests) is covered by Scrapescope. The result is an estimate, not a provider bill.

Method, limits and downloads

ipvolt ran the recorded matrix of 567 cells once on 27 September 2026 on macOS 15.7.4 (Apple M4 Pro, arm64): setup from 02:43 to 02:44 UTC, then the whole matrix from 02:44 to 02:45 UTC. A Python server on 127.0.0.1 served a 100,129-byte synthetic HTML page, and for some cells 23 pages of the CPython 3.14.7 documentation, behind a counting forward proxy on 127.0.0.1 that handled both absolute-form and CONNECT requests. With no header, the server sent the page uncompressed; otherwise it picked from the codings the client listed, preferring zstd, then br, gzip and deflate. Forced modes sent gzip, br, zstd, four zstd frames or no compression regardless of the request. Every HTTPS cell trusted a throwaway local CA through the client's own setting, and 21 control cells without that CA all failed certificate verification. The Go snippet was compiled with one lab file that put that CA in http.DefaultTransport's RootCAs before the snippet cloned it (its text is LAB_TRUST_GO in snippets.py), so trust did not depend on SSL_CERT_FILE: go1.27.1 on macOS honours that variable unless GODEBUG=x509sslcertoverrideplatform=0 is set, as it is by default in a module declaring go 1.26. A second run from the published archive, in an empty directory with every download fetched afresh, matched all 567 records on every compared field (compare.py --strict).

Versions: curl 8.7.1 (macOS, LibreSSL) and 8.22.0 (Homebrew, OpenSSL 3.6.4, brotli, zstd); GNU Wget 1.25.0; CPython 3.13.15 and 3.14.7 with Requests 2.34.2 (urllib3 2.8.0), httpx 0.28.1, aiohttp 3.14.3 and Scrapy 2.19.0; Node.js 22.23.3, 24.21.0 and 26.10.0 from nodejs.org, checksums verified, with axios 1.20.0, got 16.0.0 and node-fetch 3.3.2 and 2.7.0; Go 1.27.1; PHP 8.5.8 (static build, libcurl 8.20.0) with Guzzle 8.2.0 and illuminate/http 13.33.0; Temurin JDK 25.0.4.1.

The lab did not cover HTTP/2 or HTTP/3 (the server offered HTTP/1.1 only), Linux or Windows builds, browsers, Wget2, urllib3 on its own, curl_cffi, Apache HttpClient, OkHttp, Symfony HttpClient, .NET HttpClient, Rust reqwest, Ruby Net::HTTP or Faraday, other PHP or libcurl builds, or Java releases after 25. Defaults change between releases. These were the versions installed or current when the lab ran: the macOS system curl 8.7.1 is older than curl 8.22.0, PHP's static build was three patch releases behind php.net's 8.5.11, and Java 25 is the newest LTS release, not the newest feature release. Rerun the lab before you rely on a row.

All files are under https://ipvolt.com/downloads/accept-encoding-defaults/:

• accept-encoding-lab.zip holds everything below; its checksum is in accept-encoding-lab.zip.sha256.
• README.md explains how to run the lab, compare a rerun and read a record.
• results.json and results.csv hold all 567 records; tables.md has every table, including the decode matrix and the exact error text.
• environment.txt records the OS and every version output, corpus-manifest.txt the documentation pages, and SHA256SUMS the checksum of every other file in the archive.
• The verify snippets: curl-verify.sh, requests-verify.py, httpx-verify.py, node-fetch-bytes.mjs, node-fetch-verify.mjs, node-http-verify.mjs and go-verify.go.
• render_tables.py regenerates the tables from results.json or from your own run's harness/out/results.json, and compare.py compares your rerun with the published one.

To reproduce the whole run, unzip the archive, run ./setup.sh and then ./run.sh in harness/, then python3 ../compare.py ../results.json out/results.json. The README lists what setup downloads and what each group of cells needs. No proxy account is needed.

ipvolt, which publishes this guide, is a proxy service in development with per-GB launch pricing. The measurements use a local test proxy; the advice applies to any provider or none.

If you want to hear when ipvolt access opens, join the early-access list. One email when access opens. Nothing else.