# curl --compressed: which clients skip compression

Source: https://ipvolt.com/guides/curl-compressed-accept-encoding
Markdown: https://ipvolt.com/guides/curl-compressed-accept-encoding.md

[Home](https://ipvolt.com/index.md) / [Guides](https://ipvolt.com/guides.md) / [curl --compressed: which clients skip compression](https://ipvolt.com/guides/curl-compressed-accept-encoding.md)

Category: Integration
Reviewed: 2026-09-27
Published: 2026-09-27
Reading time: 20 minutes
Author: ipvolt

curl, Wget 1.x, urllib, node:http, PHP curl, Guzzle and java.net.http.HttpClient send no Accept-Encoding (or identity) by default. Tested fixes and byte checks.

curl sends no `Accept-Encoding` header unless you pass `--compressed`. GNU Wget 1.x and Python's `urllib.request` send `Accept-Encoding: identity`, and Node's `node:http`, PHP's ext-curl, Guzzle and Java's `java.net.http.HttpClient` send no header at all. A server that compresses only when asked then returns the full, uncompressed text, and every one of those bytes crosses your proxy. Requests, httpx, aiohttp, Scrapy, Node's built-in `fetch`, axios, got, Go and browsers already ask for compression: for them there is nothing to switch on, and at most an optional br decoder trims a little more from some pages.

Each default below was recorded by a local lab that ipvolt ran on 27 September 2026, with the versions listed under each table. The results are synthetic: a server and a proxy on 127.0.0.1 served a local test page, so no byte count here comes from a public site or a provider bill.

## Who asks for compression by default

### Sends no Accept-Encoding, or identity

A server that compresses only when asked sends these clients uncompressed text. No header: curl, node:http, node:https, PHP ext-curl, Guzzle, Laravel Http, Java HttpClient. `identity`: Wget, urllib, http.client.

| Client | Fix |
|---|---|
| curl | Add `--compressed` |
| Wget | Add `--compression=auto` |
| urllib, http.client | Use Requests or httpx |
| node:http, node:https | Use `fetch`, got or axios |
| PHP ext-curl | Set `CURLOPT_ACCEPT_ENCODING` to `''` |
| Guzzle, Laravel Http | Set `decode_content` to `'gzip'` |
| Java HttpClient | Send `Accept-Encoding: gzip`, unzip with `GZIPInputStream` |

Versions run: curl 8.7.1, 8.22.0; Wget 1.25.0; urllib.request, http.client (Python 3.13.15, 3.14.7); node:http, node:https (Node.js 22.23.3, 24.21.0, 26.10.0); PHP 8.5.8 ext-curl (libcurl 8.20.0); Guzzle 8.2.0, Laravel Http 13.33.0; Java 25.0.4.1 HttpClient.

- curl: a hand-set `-H 'Accept-Encoding: gzip'` is sent, but the body stays compressed.
- Wget: with `--compression=auto` it sent `gzip` and decoded the reply.
- urllib.request and http.client return compressed bodies as they arrived.
- node:http and node:https return compressed bodies as they arrived.
- PHP ext-curl: with `CURLOPT_ACCEPT_ENCODING` set to `''`, this libcurl sent `deflate, gzip, br, zstd`. A hand-set header is sent but not decoded.
- Laravel Http: `withOptions(['decode_content' => 'gzip'])`.
- Java HttpClient never decodes: wrap the body in `GZIPInputStream` when the reply is gzip.

### Already asks for compression

Nothing to change: these clients ask for compression and decode the reply.

| Client | Header sent |
|---|---|
| Requests | Python 3.13.15: `gzip, deflate`; Python 3.14.7: `gzip, deflate, zstd` |
| httpx | `gzip, deflate` |
| aiohttp | Python 3.13.15: `gzip, deflate`; Python 3.14.7: `gzip, deflate, zstd` |
| Scrapy | `gzip, deflate, br, zstd` |
| Node.js built-in fetch | Node.js 22.23.3/24.21.0: `br, gzip, deflate`; Node.js 26.10.0: `br, gzip, deflate, zstd` |
| Go net/http | `gzip` |

Versions run: Requests 2.34.2; httpx 0.28.1; aiohttp 3.14.3; Scrapy 2.19.0; built-in fetch (Node.js 22.23.3, 24.21.0, 26.10.0); Go 1.27.1 net/http.

- Requests: `brotli` adds br; on Python 3.13, `backports.zstd` adds zstd.
- httpx: `brotli` adds br and `zstandard` adds zstd.
- aiohttp: `Brotli` adds br; on Python 3.13, `backports.zstd` adds zstd.
- Built-in fetch over `http://`: `gzip, deflate`.
- Go: setting the header yourself turns decoding off.

If your client already asks for compression, confirm it with the byte check below, and read the section on copied browser headers before you add any header by hand. Compression is worth nothing for images, video and other payloads that are already compressed, whichever client you use.

Every row is a lab run, not a reading of the source: 567 recorded cells, none marked source-read. [tables.md](https://ipvolt.com/downloads/accept-encoding-defaults/tables.md) lists the records behind each row. "Wget" here means GNU Wget 1.x. GNU Wget2 asks for every coding it was built with by default ([wget.c](https://gitlab.com/gnuwget/wget2/-/blob/v2.3.0/src/wget.c#L4306-4331)), and Fedora 40 and later install wget2 as the `wget` command ([Fedora change](https://fedoraproject.org/wiki/Changes/Wget2asWget)); that is source-read, and the lab did not run Wget2.

## curl: no header until you pass --compressed

Plain `curl URL` sends no `Accept-Encoding` at all. In libcurl, [`CURLOPT_ACCEPT_ENCODING`](https://curl.se/libcurl/c/CURLOPT_ACCEPT_ENCODING.html) defaults to NULL, which "makes libcurl not send an `Accept-Encoding:` header and not decompress received contents automatically", and the curl tool changes it only for `--compressed`. Neither curl build in the lab sent the header, directly, through an `http://` proxy or through CONNECT. [curl#11091](https://github.com/curl/curl/issues/11091) shows the same thing with an `nc` capture of curl 7.88.1.

`--compressed` asks for every coding your build can decode, then decodes the reply:

- The macOS system curl 8.7.1, built with zlib only, sent `deflate, gzip`.
- Homebrew curl 8.22.0, built with brotli and zstd, sent `deflate, gzip, br, zstd`.

To see what yours will ask for, read the `Features:` line of `curl -V`: `libz` means gzip and deflate, `brotli` adds br, and `zstd` adds zstd.

Three details decide whether it helps:

- It is "a request, not an order; the server may or may not deliver data compressed" ([man page](https://curl.se/docs/manpage.html#--compressed)). curl reports no error when a server ignores it and has no switch to fail in that case ([curl#7516](https://github.com/curl/curl/issues/7516)), so check the bytes as shown below.
- A hand-set `-H 'Accept-Encoding: gzip'` is sent, but curl does not decode the reply. In the lab, curl output the 17,750-byte gzip stream instead of the 100,129-byte page. Use `--compressed`, not a header.
- `--no-compressed` turns it off again, for example after a `--compressed` line in `.curlrc`.

When the lab server sent br or zstd that the macOS build cannot decode, `curl --compressed` stopped with exit code 61:

```text
curl: (61) Unrecognized content encoding type. libcurl understands deflate, gzip content encodings.
```

Decompression has a cost of its own. The man page warns that "even tiny transfers might be expanded and generate a huge amount of bytes" and suggests `--max-filesize`. That option stops a transfer that grows through `--compressed` decompression only "Since 8.20.0" ([man page](https://curl.se/docs/manpage.html#--max-filesize)), so it does not protect the macOS system curl 8.7.1. That limit is documented; the lab did not test it.

Without `--compressed`, a server that compresses anyway leaves you with raw gzip bytes, as in [curl#2836](https://github.com/curl/curl/issues/2836); that is the familiar "binary output" case, and `--compressed` fixes it. For the proxy flags themselves, see [use a proxy with curl](/guides/curl-proxy-setup).

## Python: urllib sends identity; Requests, httpx, aiohttp and Scrapy compress

- **urllib.request and http.client** send `Accept-Encoding: identity` on CPython 3.13.15 and 3.14.7. http.client adds it unless you set your own, with the comment "we don't support encodings such as x-gzip or x-deflate" ([client.py](https://github.com/python/cpython/blob/v3.14.7/Lib/http/client.py#L1296-L1299)), and neither module decodes a reply. In the lab, urllib.request returned gzip, br and zstd replies as they arrived; http.client, which urllib.request sends its requests through, ran only with its default header, so for http.client this rests on the source. Setting the header yourself only gets you compressed bytes to decode by hand, so switch clients instead. A script that must stay on the standard library can send `Accept-Encoding: gzip` and, when the reply carries `Content-Encoding: gzip`, decode the body with [`gzip.decompress()`](https://docs.python.org/3.14/library/gzip.html#gzip.decompress); that route was not run in the lab.
- **Requests 2.34.2** (with urllib3 2.8.0) takes its default from urllib3 ([utils.py](https://github.com/psf/requests/blob/v2.34.2/src/requests/utils.py#L91-L93)): `gzip, deflate` on Python 3.13.15 and `gzip, deflate, zstd` on 3.14.7, where urllib3 uses the standard library's [`compression.zstd`](https://docs.python.org/3.14/library/compression.zstd.html). Installing `brotli` adds br; on 3.13, `backports.zstd` adds zstd. Since urllib3 2.6.0, the separate `zstandard` package no longer enables zstd ([request.py](https://github.com/urllib3/urllib3/blob/2.8.0/src/urllib3/util/request.py#L22-L41); source-read, not run). The proxy side is in [configure Python Requests](/guides/python-requests-proxy).
- **urllib3 on its own**, through `urllib3.request()` or a `PoolManager`, adds no compression header. Its request path passes through to http.client ([connection.py](https://github.com/urllib3/urllib3/blob/2.8.0/src/urllib3/connection.py#L525-L531)), which by the source means `identity`. This is source-read; the lab ran Requests, not urllib3 alone.
- **aiohttp 3.14.3** sends `gzip, deflate` on 3.13.15 and `gzip, deflate, zstd` on 3.14.7; `Brotli` adds br, and on 3.13 `backports.zstd` adds zstd. It is the only Python client here that raises an exception on a coding it cannot decode; the exact message is below.
- **Scrapy 2.19.0** sends `gzip, deflate, br, zstd` on both. Since Scrapy 2.18.0, brotli and Zstandard support are required, "so `br` and `zstd` are always included in the `Accept-Encoding` header of requests" ([release notes](https://github.com/scrapy/scrapy/blob/2.19.0/docs/news.rst)).

### httpx and zstd on Python 3.14

httpx 0.28.1 sends `gzip, deflate` on both Pythons. It adds br with `brotli` and zstd only with the third-party `zstandard` package; unlike Requests and aiohttp, it does not use the standard-library zstd of Python 3.14 ([_decoders.py](https://github.com/encode/httpx/blob/0.28.1/httpx/_decoders.py#L381-L393)). The [HTTPX documentation](https://www.python-httpx.org/) installs both decoders with `pip install "httpx[brotli,zstd]"`. None of that is needed for compression itself, since gzip is always requested.

It matters when a server sends a coding httpx cannot decode: httpx returns the compressed bytes and raises nothing. On Python 3.14.7 without `zstandard`, a zstd reply reached the caller as the 19,298-byte zstd stream. Proxy configuration is in the [HTTPX async proxy guide](/guides/httpx-async-proxy).

## Node.js: built-in fetch compresses, node:http does not

Built-in `fetch` (undici) chooses the header by URL scheme and release line ([source, v26.10.0](https://github.com/nodejs/node/blob/v26.10.0/deps/undici/src/lib/web/fetch/index.js#L1562-L1576); [v24.21.0](https://github.com/nodejs/node/blob/v24.21.0/deps/undici/src/lib/web/fetch/index.js#L1514-L1528)):

- `https://` URLs: `br, gzip, deflate` on Node.js 22.23.3 and 24.21.0, and `br, gzip, deflate, zstd` on 26.10.0.
- `http://` URLs: `gzip, deflate` on all three.
- Any request with a `Range` header: `identity` on all three.

All three decoded gzip and br. zstd is where they differ, including zstd a server sends unasked:

- Node.js 22.23.3 `fetch` has no zstd decoder and returned the compressed bytes with no error ([source](https://github.com/nodejs/node/blob/v22.23.3/deps/undici/src/lib/web/fetch/index.js#L2149-L2170)).
- Node.js 24.21.0 `fetch` decoded zstd it had not asked for. A body made of four concatenated zstd frames came back as the first frame only: 25,033 of 100,129 bytes, with no error. axios 1.20.0 and got 16.0.0 also returned only the first frame on Node.js 22.23.3 and 24.21.0.
- Node.js 26.10.0 decoded all four frames with `fetch`, axios and got.

`node:http` and `node:https` send no `Accept-Encoding` on any of the three releases and never decode; Node's core HTTP client does not set the header ([_http_client.js](https://github.com/nodejs/node/blob/v26.10.0/lib/_http_client.js)). Node's [zlib documentation](https://nodejs.org/api/zlib.html#compressing-http-requests-and-responses) shows the manual route: set the header yourself and pipe the response through a decompressor. Switching to `fetch`, got or axios is less code.

### Built-in fetch vs the node-fetch package

The npm package `node-fetch` is a separate client with its own defaults ([node-fetch#1556](https://github.com/node-fetch/node-fetch/discussions/1556)). On Node.js 24.21.0, node-fetch 3.3.2 sent `gzip, deflate, br` and node-fetch 2.7.0 sent `gzip,deflate`, and both decoded the reply. axios 1.20.0 sends `gzip, compress, deflate, br` and adds zstd only with `transitional: { advertiseZstdAcceptEncoding: true }` ([http.js](https://github.com/axios/axios/blob/v1.20.0/lib/adapters/http.js#L54-L57)). got 16.0.0 sent `gzip, deflate, br, zstd` on all three releases ([index.ts](https://github.com/sindresorhus/got/blob/v16.0.0/source/core/index.ts#L2248-L2258)). None of them needs a change. The proxy dispatcher for built-in `fetch` is covered in [use a proxy with Node.js fetch](/guides/nodejs-fetch-proxy).

## Go, Wget, PHP and Java

### Go net/http asks for gzip only

Go 1.27.1 sends `Accept-Encoding: gzip`, decodes the reply and removes `Content-Encoding` and `Content-Length` from the response it gives you ([transport.go](https://github.com/golang/go/blob/go1.27.1/src/net/http/transport.go#L199-L207)). It sends no header for `HEAD` requests, for requests with a `Range` header, or with `DisableCompression: true`. Two consequences:

- Setting `Accept-Encoding` yourself turns the decoding off: "if the user explicitly requested gzip it is not automatically uncompressed". With `req.Header.Set("Accept-Encoding", "gzip")`, the lab's Go client received the 17,750-byte gzip stream.
- net/http has no br or zstd decoder. A server that sent them unasked delivered raw bytes.

### GNU Wget 1.x sends identity

Wget 1.25.0 sends `Accept-Encoding: identity`. For `--compression`, the manual says of "none": "This is the default" ([manual](https://www.gnu.org/software/wget/manual/wget.html#HTTP-Options)); the same page's `.wgetrc` list still calls `auto` the default, but the lab run matches `none`. With `--compression=auto`, Wget sent `gzip` and decoded gzip, and nothing else: br or zstd sent unasked arrived raw. Wget2 differs, as noted above.

### PHP: Guzzle, Laravel Http and ext-curl send no Accept-Encoding

- **PHP ext-curl** (PHP 8.5.8 with libcurl 8.20.0) sends nothing, because `CURLOPT_ACCEPT_ENCODING` "Defaults to null" ([PHP manual](https://www.php.net/manual/en/curl.constants.php)). `CURLOPT_ACCEPT_ENCODING => ''` asks for every coding the linked libcurl supports, `deflate, gzip, br, zstd` in this build, and decodes. An `Accept-Encoding` line in `CURLOPT_HTTPHEADER` is sent but not decoded.
- **Guzzle 8.2.0** removes the header on purpose. Its cURL handler turns on every decoder, then adds an empty `Accept-Encoding:` line to stop curl sending one, with the comment that this "will be interpreted as 'Accept-Encoding: *'" ([CurlFactory.php](https://github.com/guzzle/guzzle/blob/8.2.0/src/Handler/CurlFactory.php#L2609-L2622)). The same code is in every tag checked from 6.5.0 to 8.2.0 (source-read). Give [`decode_content`](https://docs.guzzlephp.org/en/stable/request-options.html#decode-content) a string, such as `'decode_content' => 'gzip'`, and Guzzle sends it as the header ([Client.php](https://github.com/guzzle/guzzle/blob/8.2.0/src/Client.php#L1582-L1586)); `'gzip, deflate, br, zstd'` also worked with the cURL handler. By default the cURL handler still decodes gzip, br or zstd that a server sends unasked. Guzzle's `StreamHandler`, which uses PHP streams instead of ext-curl, decoded gzip but returned br and zstd raw.
- **Laravel Http** (illuminate/http 13.33.0) builds a plain Guzzle client and inherits all of this ([PendingRequest.php](https://github.com/laravel/framework/blob/v13.33.0/src/Illuminate/Http/Client/PendingRequest.php)). Both `withOptions(['decode_content' => 'gzip'])` and `withHeaders(['Accept-Encoding' => 'gzip'])` sent `gzip` and decoded the reply.

### Java HttpClient never decodes

`java.net.http.HttpClient` on Temurin 25.0.4.1 sent no `Accept-Encoding` and did not decode a gzip reply that the server sent unasked. To get compression, send `Accept-Encoding: gzip` yourself and read the body through `java.util.zip.GZIPInputStream` when the response carries `Content-Encoding: gzip`, as the lab's [Java client](https://ipvolt.com/downloads/accept-encoding-defaults/harness/clients/php-java/JavaHttpClientLab.java) does. No other Java client was run.

## Accept-Encoding: identity, no header, and garbled output

[RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-accept-encoding) separates three request states:

- **No header** (curl, node:http, ext-curl, Guzzle, Java): "any content coding is considered acceptable by the user agent".
- **`identity`** (urllib, Wget 1.x): "a synonym for 'no encoding'".
- **An empty value**: "the user agent does not want any content coding in response".

So the standard allows a server to compress a reply to a request with no header, and "compress only when asked" is server practice, not an RFC rule. Server and CDN documentation describes that practice. Cloudflare picks gzip, Brotli, Zstandard or no compression "depending on" the values in the request's `accept-encoding` header, the plan and any matching compression rule ([Cloudflare](https://developers.cloudflare.com/speed/optimization/content/compression/), page updated 17 April 2026). Apache's mod_deflate sends `Vary: Accept-Encoding` so that compressed content is not "sent to a client that will not understand it" ([mod_deflate](https://httpd.apache.org/docs/2.4/mod/mod_deflate.html)). Guzzle's comment relies on the RFC reading; with servers that follow that practice, a missing header gets the uncompressed body. The lab server followed the same practice. How often real servers compress unasked was not measured.

Some do. nginx's `gzip_static always` serves the gzipped file "without checking if the client supports it" ([nginx](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html)), and curl#2836 reports a real host that did it. When the lab server sent gzip unasked, curl without `--compressed`, Wget, urllib, `node:http`, PHP ext-curl, Java and Go with `DisableCompression` all handed the caller the 17,750-byte gzip stream; Guzzle and Laravel Http decoded it. If a response from one of those clients looks like binary garbage, check its `Content-Encoding` first.

## Verify the bytes on your own target

For a client without a byte-count snippet here, run the curl comparison below against the same URL through the same proxy, and compare its `Content-Encoding` with your client's. This measures curl's response; a different coding or server response is not a byte measurement of your client.

The curl, Requests, httpx and `node-fetch-bytes.mjs` snippets below print wire body bytes: the response body as it crossed the network, before any decoding. Response headers, TLS records, the CONNECT exchange and retries are not included, so this is not what a provider bills. In eight lab pairs ([tables.md](https://ipvolt.com/downloads/accept-encoding-defaults/tables.md) lists them), the same client fetched the test page over CONNECT with and without compression, and those extras added 2,343 to 3,520 bytes per response on the proxy's client leg. They lowered the ratio, for example from 5.19× on the body to 4.56× on the proxy leg for curl 8.22.0, but not the saving: in each of those pairs, the proxy leg shrank by the body difference plus 87 bytes. To meter a whole run through your proxy and convert it at your own rate, use [Scrapescope](/guides/scraper-bandwidth-report).

The Go snippet prints the decoded size and `resp.Uncompressed`, which says whether Go decoded the reply, and the linked `node-fetch-verify.mjs` prints the header `fetch` sent; neither prints wire bytes. Each snippet ran unchanged in the lab except that `https://example.com/` became a loopback URL, and the outputs shown are from those runs; the Go snippet was also compiled with one lab file that trusts the lab's CA, described in the method section. Set `PROXY_URL` to your proxy. If you need to trust a private CA, use the client's own setting, such as `--cacert`, `NODE_EXTRA_CA_CERTS` or Go's `tls.Config.RootCAs`; never turn certificate verification off.

### Verify with curl

```sh
# Wire body bytes and the Content-Encoding the server chose (curl 7.84.0+ for %header{}).
curl -sS -o /dev/null -x "$PROXY_URL" \
  -w '%{size_download} %header{content-encoding}\n' https://example.com/
curl -sS -o /dev/null -x "$PROXY_URL" --compressed \
  -w '%{size_download} %header{content-encoding}\n' https://example.com/
```

With Homebrew curl 8.22.0 against the lab's test page, the first line printed `100129` and no coding, and the second `19298 zstd`; the macOS system curl 8.7.1 printed `17750 gzip` on the second line. curl's `size_download` is "the size of the body/data that was transferred, excluding headers", and `%header{}` needs curl 7.84.0 or later ([write-out](https://curl.se/docs/manpage.html#-w)). The ratio for that page is the first number divided by the second.

### Verify with Requests and httpx

```python
import os

import requests

proxies = {"http": os.environ["PROXY_URL"], "https": os.environ["PROXY_URL"]}
with requests.get("https://example.com/", proxies=proxies, stream=True) as r:
    wire = r.raw.read(decode_content=False)  # body bytes as sent, still encoded
    print(r.headers.get("content-encoding"), len(wire))
```

Requests printed `gzip 17750` on Python 3.13.15 and `zstd 19298` on 3.14.7. For httpx, `r.num_bytes_downloaded` counts the bytes as read and `len(r.content)` the decoded body:

```python
import os

import httpx

with httpx.Client(proxy=os.environ["PROXY_URL"]) as client:
    r = client.get("https://example.com/")
    # wire body bytes (as received) vs decoded bytes your code sees
    print(r.headers.get("content-encoding"), r.num_bytes_downloaded, len(r.content))
```

It printed `gzip 17750 100129` on Python 3.14.7, and `zstd 19298 100129` with `zstandard` installed. urllib never decodes, so the length of what you read from it is already the wire body.

### Verify with Node.js

For built-in `fetch`, Node's resource timing entry for the URL gives the wire body size:

```js
// Wire body bytes of a built-in fetch, from Node's resource timing entry for the URL.
// Run: NODE_USE_ENV_PROXY=1 HTTPS_PROXY="$PROXY_URL" node node-fetch-bytes.mjs
import { setImmediate } from 'node:timers/promises';

const url = new URL('https://example.com/');
const res = await fetch(url);
const body = await res.arrayBuffer(); // already decoded by fetch

// Node adds the entry after the body has been read, so give the event loop a turn.
let entry;
for (let turn = 0; !entry && turn < 100; turn++) {
  await setImmediate();
  entry = performance.getEntriesByType('resource').findLast((e) => e.name === url.href);
}
performance.clearResourceTimings(); // the buffer stops at 250 entries unless you clear it

// encodedBodySize: the body as it crossed the network, before decoding (no headers, TLS or CONNECT)
console.log('content-encoding', res.headers.get('content-encoding'),
  'wire body bytes', entry?.encodedBodySize, 'decoded bytes', body.byteLength);
```

Through the lab's CONNECT proxy it printed `content-encoding br wire body bytes 18451 decoded bytes 100129` on Node.js 22.23.3 and 24.21.0, and `content-encoding zstd wire body bytes 19298 decoded bytes 100129` on 26.10.0, each matching what the lab server sent. In 18 further lab cells on the three releases (direct `http://`, direct `https://` and through the CONNECT proxy, each with and without compression), `encodedBodySize` equalled the body bytes the server sent every time; [tables.md](https://ipvolt.com/downloads/accept-encoding-defaults/tables.md) lists the readings. The entry never existed straight after the body was read, only after one event-loop turn, which is why the snippet waits. The snippet then clears the entries, because Node's resource timing buffer holds only 250 by default ([observe.js](https://github.com/nodejs/node/blob/v26.10.0/lib/internal/perf/observe.js#L103)). Do not use `transferSize` instead: it was `encodedBodySize` plus a fixed 300 bytes in every cell, while through the proxy the client leg carried 3,431 to 3,520 bytes more than the body.

[node-fetch-verify.mjs](https://ipvolt.com/downloads/accept-encoding-defaults/harness/snippets/node-fetch-verify.mjs) prints the header `fetch` sent (`sent accept-encoding: br, gzip, deflate, zstd` on Node.js 26.10.0). For `node:https`, which never decodes, [node-http-verify.mjs](https://ipvolt.com/downloads/accept-encoding-defaults/harness/snippets/node-http-verify.mjs) counts the body as it arrives, which is the wire body.

### Verify with Go

```go
// Did Go ask for gzip and decode it for you? resp.Uncompressed says so.
// Run: go run go-verify.go   (with PROXY_URL set in the environment)
package main

import (
	"fmt"
	"io"
	"net/http"
	"net/url"
	"os"
)

func main() {
	proxy, err := url.Parse(os.Getenv("PROXY_URL"))
	if err != nil {
		panic(err)
	}
	t := http.DefaultTransport.(*http.Transport).Clone()
	t.Proxy = http.ProxyURL(proxy)
	resp, err := (&http.Client{Transport: t}).Get("https://example.com/")
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()
	body, err := io.ReadAll(resp.Body)
	if err != nil {
		panic(err)
	}
	// true: Go sent "Accept-Encoding: gzip" itself, received gzip and decoded it,
	// then removed Content-Encoding and set ContentLength to -1.
	fmt.Printf("uncompressed=%v content-encoding=%q content-length=%d decoded-bytes=%d\n",
		resp.Uncompressed, resp.Header.Get("Content-Encoding"), resp.ContentLength, len(body))
}
```

With Go 1.27.1 it printed `uncompressed=true content-encoding="" content-length=-1 decoded-bytes=100129`. `resp.Uncompressed` only says that Go decoded the reply ([response.go](https://github.com/golang/go/blob/go1.27.1/src/net/http/response.go#L93-L100)); the response no longer carries its wire size. To count wire bytes, set the header yourself, which leaves the body compressed, measure it, then decode it with `compress/gzip`.

## Through a proxy: CONNECT vs absolute-form

For an `https://` URL, the client opens a CONNECT tunnel and sends its request inside TLS. With end-to-end TLS and no TLS interception, the proxy cannot read or change `Accept-Encoding`; it relays the encrypted response. The lab recorder did not parse headers inside CONNECT tunnels. For an `http://` URL, most clients send the request itself to the proxy in absolute form, as in `GET http://host/path`, so the proxy reads every header: the lab proxy saw curl 8.22.0's `deflate, gzip, br, zstd` with `--compressed`, Wget's `identity` and Go's `gzip`. An intermediary on that path could also rewrite the header or the body. The lab proxy did neither, and no real proxy was tested.

Two clients behaved differently with `http://` URLs:

- Node.js 22.23.3 and 24.21.0 built-in `fetch`, with `NODE_USE_ENV_PROXY=1`, tunnelled `http://` URLs through CONNECT. The lab recorder did not parse the inner header, but that HTTP request was still plaintext and inspectable: [CONNECT itself does not encrypt traffic](https://www.rfc-editor.org/rfc/rfc9110.html#name-connect). Node.js 26.10.0 sent absolute form.
- Guzzle's `StreamHandler` sent an origin-form request line to the proxy unless `request_fulluri` was set, and the lab proxy rejected it with a 400.

What counts toward a bill is each provider's rule. Decodo, for example, documents download traffic as "every byte received from the target website, including response headers and cookies", says that for HTTPS "the bytes tunneled through the proxy also count toward proxy traffic usage", and gives an example in which its own test endpoint took 0.85 kB of download over `http://` against 3.96 kB over `https://` ([Decodo help](https://help.decodo.com/docs/proxy-traffic), read 26 September 2026, page modified 23 April 2026). Decodo does not say whether it counts compressed or decoded bytes. For HTTPS with end-to-end TLS inside CONNECT, the proxy counts the encrypted bytes crossing the tunnel, including the compressed response and TLS overhead; that is an inference, not Decodo's statement. Scraping APIs and unblockers that send the request to the target for you choose their own headers and meter in their own way, so check your provider's documentation.

## Copying a browser's Accept-Encoding

You may be tempted to paste a browser's `Accept-Encoding: gzip, deflate, br, zstd` into a scraper. That asks the server for codings your client may not decode, and in several clients a hand-set header also switches decoding off. With exactly that header, the lab server chose zstd:

- curl (both builds), urllib, `node:http`, Go, PHP ext-curl, Java `HttpClient` and Guzzle's `StreamHandler` returned the compressed bytes: a normal response with an unreadable body.
- Requests and httpx did the same whenever the matching decoder was missing. On Python 3.13, bare Requests returned both zstd and br raw. On 3.14 it decoded zstd but returned br raw when the server sent br. httpx without `zstandard` returned zstd raw on both Pythons, with no exception.
- Node.js 22.23.3 `fetch` returned zstd raw; 24.21.0 and 26.10.0 decoded it.
- Guzzle's cURL handler decoded both zstd and br.
- aiohttp raised an exception whenever it lacked the decoder: br without `Brotli`, and zstd on Python 3.13 without `backports.zstd`. On 3.14 it decoded zstd.

The fix is to let the client write the header. If you want br or zstd, install the decoder package your client uses, such as `brotli`, `backports.zstd` on Python 3.13 or `httpx[brotli,zstd]`; the client then adds the token itself.

### aiohttp 3.14.3: `` Can not decode content-encoding: brotli (br). Please install `Brotli` ``

When a server sends br and the `Brotli` package is missing, aiohttp 3.14.3 raises the first of these, on Python 3.13.15 and 3.14.7 alike. The second is the zstd version, raised on Python 3.13.15 without `backports.zstd` (loopback URLs shortened):

```text
aiohttp.client_exceptions.ClientResponseError: 400, message='Can not decode content-encoding: brotli (br). Please install `Brotli`', url='…'
aiohttp.client_exceptions.ClientResponseError: 400, message='Can not decode content-encoding: zstandard (zstd). Please install `backports.zstd`', url='…'
```

The 400 is aiohttp's own status for a response it cannot decode; the lab server had answered 200 ([http_parser.py](https://github.com/aio-libs/aiohttp/blob/v3.14.3/aiohttp/http_parser.py#L1147-L1159)). Installing `Brotli`, or `backports.zstd` for the second, fixed it in the lab: aiohttp then decoded br, and zstd including a body of four zstd frames. The shorter `Can not decode content-encoding: br` comes from a different path in the same file: a decoder is installed, but the body failed to decompress.

### Servers that mishandle advertised zstd

Advertising a coding can itself break a request. OpenSearch 2.19.0 hung when a request's `Accept-Encoding` included zstd, as it does from curl builds with zstd support. It was fixed for 2.19.1 and 3.0.0 ([OpenSearch#17339](https://github.com/opensearch-project/OpenSearch/issues/17339)). If one host times out only for zstd-capable clients, a request that asks for gzip only will show whether zstd negotiation is the problem.

## Where compression does not help

- **Payloads that are already compressed**, such as images, video and archives. "Media files such as images that are already compressed do not benefit from HTTP compression" ([Web Almanac 2021](https://almanac.httparchive.org/en/2021/compression)).
- **Responses a server or CDN leaves uncompressed.** Cloudflare, for example, documents compressing only 200, 403 and 404 responses, with a minimum size of 48 bytes for gzip and 50 bytes for Brotli and Zstandard ([Cloudflare](https://developers.cloudflare.com/speed/optimization/content/compression/)).
- **Range and HEAD requests.** Go sends no header for either, and Node.js `fetch` sends `identity` whenever a `Range` header is set.
- **Uploads.** `--compressed` and its equivalents work on downloads; for request bodies "there is no standard way to do compression" ([Everything curl](https://everything.curl.dev/http/modify/compression.html)).
- **Browser automation.** Browsers negotiate compression themselves: MDN gives `gzip, deflate, br, zstd` as the typical browser value ([MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Accept-Encoding)), and Chrome has decoded zstd by default since version 123 ([Chrome Platform Status](https://chromestatus.com/feature/6186023867908096)), so in Playwright or Puppeteer the bytes to cut are in what the browser downloads, not in this header.

For pages you fetch again and again, conditional requests are a separate lever: a `304 Not Modified` response carries no body at all ([RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-304-not-modified); [ETag monitoring](/blog/etag-monitoring-cache-control)).

## What it is worth

Only clients in the first table have compression to switch on. For them:

saved $ ≈ pages × uncompressed text bytes per page × (1 − 1/ratio) ÷ 10⁹ × your $ per GB

Take the ratio from the verify step on your own pages, as bytes without compression divided by bytes with it, and the rate from your own plan.

Compression lowers the bill only when you pay per byte. On a flat plan priced by Mbps or threads the price stays the same, though more compressed pages fit through the channel. [What unlimited residential proxies cost per GB](/blog/unlimited-residential-proxies-cost-per-gb) compares the two ways of billing.

An invented example, not a measurement: 100,000 pages of 100 KB (100,000 bytes) of HTML are 10 GB uncompressed. At a ratio of 4, compression saves 7.5 GB; at a ratio of 10, it saves 9.0 GB. At an invented rate of $3 to $8 per GB, that is about $22.50 to $72 for one crawl of those pages.

For a client that already asks for compression, switching it on saves nothing, because it is already on. An optional br decoder for Requests, httpx or aiohttp can trim a little more on some pages: on the lab's 23 documentation pages, br bodies totalled 277,665 bytes against 325,873 for gzip, about 15% fewer, but on the synthetic page br was larger than gzip (18,451 against 17,750 bytes).

The lab's own ratios are not a forecast. The synthetic test page shrank 5.64× with gzip and 5.19× with zstd. In this corpus of 23 CPython 3.14.7 documentation pages, the uncompressed bodies were 6.67× their gzip size, 6.61× their zstd size and 7.83× their br size. Those numbers describe those files only.

The formula counts body bytes; in the lab, headers, TLS and the CONNECT exchange barely changed with compression, as the verify section shows. How a provider turns bytes into a bill (GB or GiB, minimums, failed requests) is covered by [Scrapescope](/guides/scraper-bandwidth-report). The result is an estimate, not a provider bill.

## Method, limits and downloads

ipvolt ran the recorded matrix of 567 cells once on 27 September 2026 on macOS 15.7.4 (Apple M4 Pro, arm64): setup from 02:43 to 02:44 UTC, then the whole matrix from 02:44 to 02:45 UTC. A Python server on 127.0.0.1 served a 100,129-byte synthetic HTML page, and for some cells 23 pages of the CPython 3.14.7 documentation, behind a counting forward proxy on 127.0.0.1 that handled both absolute-form and CONNECT requests. With no header, the server sent the page uncompressed; otherwise it picked from the codings the client listed, preferring zstd, then br, gzip and deflate. Forced modes sent gzip, br, zstd, four zstd frames or no compression regardless of the request. Every HTTPS cell trusted a throwaway local CA through the client's own setting, and 21 control cells without that CA all failed certificate verification. The Go snippet was compiled with one lab file that put that CA in `http.DefaultTransport`'s `RootCAs` before the snippet cloned it (its text is `LAB_TRUST_GO` in [snippets.py](https://ipvolt.com/downloads/accept-encoding-defaults/harness/nodego/snippets.py)), so trust did not depend on `SSL_CERT_FILE`: go1.27.1 on macOS honours that variable unless `GODEBUG=x509sslcertoverrideplatform=0` is set, as it is by default in a module declaring go 1.26. A second run from the published archive, in an empty directory with every download fetched afresh, matched all 567 records on every compared field (`compare.py --strict`).

Versions: curl 8.7.1 (macOS, LibreSSL) and 8.22.0 (Homebrew, OpenSSL 3.6.4, brotli, zstd); GNU Wget 1.25.0; CPython 3.13.15 and 3.14.7 with Requests 2.34.2 (urllib3 2.8.0), httpx 0.28.1, aiohttp 3.14.3 and Scrapy 2.19.0; Node.js 22.23.3, 24.21.0 and 26.10.0 from nodejs.org, checksums verified, with axios 1.20.0, got 16.0.0 and node-fetch 3.3.2 and 2.7.0; Go 1.27.1; PHP 8.5.8 (static build, libcurl 8.20.0) with Guzzle 8.2.0 and illuminate/http 13.33.0; Temurin JDK 25.0.4.1.

The lab did not cover HTTP/2 or HTTP/3 (the server offered HTTP/1.1 only), Linux or Windows builds, browsers, Wget2, urllib3 on its own, curl_cffi, Apache HttpClient, OkHttp, Symfony HttpClient, .NET HttpClient, Rust reqwest, Ruby Net::HTTP or Faraday, other PHP or libcurl builds, or Java releases after 25. Defaults change between releases. These were the versions installed or current when the lab ran: the macOS system curl 8.7.1 is older than curl 8.22.0, PHP's static build was three patch releases behind php.net's 8.5.11, and Java 25 is the newest LTS release, not the newest feature release. Rerun the lab before you rely on a row.

All files are under `https://ipvolt.com/downloads/accept-encoding-defaults/`:

- [accept-encoding-lab.zip](https://ipvolt.com/downloads/accept-encoding-defaults/accept-encoding-lab.zip) holds everything below; its checksum is in [accept-encoding-lab.zip.sha256](https://ipvolt.com/downloads/accept-encoding-defaults/accept-encoding-lab.zip.sha256).
- [README.md](https://ipvolt.com/downloads/accept-encoding-defaults/README.md) explains how to run the lab, compare a rerun and read a record.
- [results.json](https://ipvolt.com/downloads/accept-encoding-defaults/results.json) and [results.csv](https://ipvolt.com/downloads/accept-encoding-defaults/results.csv) hold all 567 records; [tables.md](https://ipvolt.com/downloads/accept-encoding-defaults/tables.md) has every table, including the decode matrix and the exact error text.
- [environment.txt](https://ipvolt.com/downloads/accept-encoding-defaults/environment.txt) records the OS and every version output, [corpus-manifest.txt](https://ipvolt.com/downloads/accept-encoding-defaults/corpus-manifest.txt) the documentation pages, and [SHA256SUMS](https://ipvolt.com/downloads/accept-encoding-defaults/SHA256SUMS) the checksum of every other file in the archive.
- The verify snippets: [curl-verify.sh](https://ipvolt.com/downloads/accept-encoding-defaults/harness/snippets/curl-verify.sh), [requests-verify.py](https://ipvolt.com/downloads/accept-encoding-defaults/harness/snippets/requests-verify.py), [httpx-verify.py](https://ipvolt.com/downloads/accept-encoding-defaults/harness/snippets/httpx-verify.py), [node-fetch-bytes.mjs](https://ipvolt.com/downloads/accept-encoding-defaults/harness/snippets/node-fetch-bytes.mjs), [node-fetch-verify.mjs](https://ipvolt.com/downloads/accept-encoding-defaults/harness/snippets/node-fetch-verify.mjs), [node-http-verify.mjs](https://ipvolt.com/downloads/accept-encoding-defaults/harness/snippets/node-http-verify.mjs) and [go-verify.go](https://ipvolt.com/downloads/accept-encoding-defaults/harness/snippets/go-verify.go).
- [render_tables.py](https://ipvolt.com/downloads/accept-encoding-defaults/render_tables.py) regenerates the tables from `results.json` or from your own run's `harness/out/results.json`, and [compare.py](https://ipvolt.com/downloads/accept-encoding-defaults/compare.py) compares your rerun with the published one.

To reproduce the whole run, unzip the archive, run `./setup.sh` and then `./run.sh` in `harness/`, then `python3 ../compare.py ../results.json out/results.json`. The README lists what setup downloads and what each group of cells needs. No proxy account is needed.

ipvolt, which publishes this guide, is a proxy service in development with per-GB launch pricing. The measurements use a local test proxy; the advice applies to any provider or none.

If you want to hear when ipvolt access opens, [join the early-access list](https://ipvolt.com/#waitlist-closing). One email when access opens. Nothing else.

## Sources & further reading

- [RFC 9110: CONNECT tunnels and the HTTPS scheme](https://www.rfc-editor.org/rfc/rfc9110.html#name-connect)
- [Python 3.14 gzip: gzip.decompress and multi-member streams](https://docs.python.org/3.14/library/gzip.html#gzip.decompress)
- [Node.js 26.10.0: resource timing buffer limit](https://github.com/nodejs/node/blob/v26.10.0/lib/internal/perf/observe.js#L103)
- [RFC 9110 section 12.5.3: Accept-Encoding (no header, identity, empty value)](https://www.rfc-editor.org/rfc/rfc9110.html#name-accept-encoding)
- [RFC 9110 section 15.4.5: 304 Not Modified (no content)](https://www.rfc-editor.org/rfc/rfc9110.html#name-304-not-modified)
- [curl man page: --compressed (the page describes 8.23.0; this text is unchanged since the tested 8.22.0)](https://curl.se/docs/manpage.html#--compressed)
- [curl man page: --max-filesize and decompression (since 8.20.0)](https://curl.se/docs/manpage.html#--max-filesize)
- [curl man page: -w, %{size_download} and %header{} (added in 7.84.0)](https://curl.se/docs/manpage.html#-w)
- [libcurl: CURLOPT_ACCEPT_ENCODING (default NULL, no header and no decoding)](https://curl.se/libcurl/c/CURLOPT_ACCEPT_ENCODING.html)
- [curl issue #11091: no Accept-Encoding by default, captured with nc](https://github.com/curl/curl/issues/11091)
- [curl issue #7516: curl can only ask for compression](https://github.com/curl/curl/issues/7516)
- [curl issue #2836: raw gzip output when a server compresses unasked](https://github.com/curl/curl/issues/2836)
- [Everything curl: HTTP compression applies to downloads](https://everything.curl.dev/http/modify/compression.html)
- [CPython 3.14.7 http.client: Accept-Encoding identity](https://github.com/python/cpython/blob/v3.14.7/Lib/http/client.py#L1296-L1299)
- [urllib3 2.8.0 util/request.py: gzip, deflate, br and zstd tokens](https://github.com/urllib3/urllib3/blob/2.8.0/src/urllib3/util/request.py#L22-L41)
- [urllib3 2.8.0 connection.py: the request path adds no compression header](https://github.com/urllib3/urllib3/blob/2.8.0/src/urllib3/connection.py#L525-L531)
- [Requests 2.34.2 utils.py: default Accept-Encoding](https://github.com/psf/requests/blob/v2.34.2/src/requests/utils.py#L91-L93)
- [httpx 0.28.1 _decoders.py: br needs brotli, zstd needs zstandard](https://github.com/encode/httpx/blob/0.28.1/httpx/_decoders.py#L381-L393)
- [HTTPX documentation: optional brotli and zstd decoders](https://www.python-httpx.org/)
- [aiohttp 3.14.3 http_parser.py: Can not decode content-encoding](https://github.com/aio-libs/aiohttp/blob/v3.14.3/aiohttp/http_parser.py#L1147-L1159)
- [Scrapy 2.19.0 release notes: br and zstd required since 2.18.0](https://github.com/scrapy/scrapy/blob/2.19.0/docs/news.rst)
- [Python 3.14: compression.zstd](https://docs.python.org/3.14/library/compression.zstd.html)
- [Node.js 26.10.0 bundled undici fetch: Accept-Encoding by scheme and Range](https://github.com/nodejs/node/blob/v26.10.0/deps/undici/src/lib/web/fetch/index.js#L1562-L1576)
- [Node.js 24.21.0 bundled undici fetch](https://github.com/nodejs/node/blob/v24.21.0/deps/undici/src/lib/web/fetch/index.js#L1514-L1528)
- [Node.js 22.23.3 bundled undici fetch: no zstd decoder](https://github.com/nodejs/node/blob/v22.23.3/deps/undici/src/lib/web/fetch/index.js#L2149-L2170)
- [Node.js 26.10.0 core HTTP client (no Accept-Encoding)](https://github.com/nodejs/node/blob/v26.10.0/lib/_http_client.js)
- [Node.js zlib: compressing HTTP requests and responses](https://nodejs.org/api/zlib.html#compressing-http-requests-and-responses)
- [node-fetch discussion #1556: default gzip, deflate, br](https://github.com/node-fetch/node-fetch/discussions/1556)
- [axios 1.20.0 http adapter: default Accept-Encoding and zstd opt-in](https://github.com/axios/axios/blob/v1.20.0/lib/adapters/http.js#L54-L57)
- [got 16.0.0 core: default accept-encoding](https://github.com/sindresorhus/got/blob/v16.0.0/source/core/index.ts#L2248-L2258)
- [Go 1.27.1 net/http transport.go: DisableCompression and automatic gzip](https://github.com/golang/go/blob/go1.27.1/src/net/http/transport.go#L199-L207)
- [Go 1.27.1 net/http response.go: Response.Uncompressed](https://github.com/golang/go/blob/go1.27.1/src/net/http/response.go#L93-L100)
- [GNU Wget manual: --compression (none is the default)](https://www.gnu.org/software/wget/manual/wget.html#HTTP-Options)
- [GNU Wget2 2.3.0 wget.c: default Accept-Encoding](https://gitlab.com/gnuwget/wget2/-/blob/v2.3.0/src/wget.c#L4306-4331)
- [Fedora change: Wget2 as wget](https://fedoraproject.org/wiki/Changes/Wget2asWget)
- [PHP manual: cURL constants, CURLOPT_ACCEPT_ENCODING](https://www.php.net/manual/en/curl.constants.php)
- [Guzzle 8.2.0 CurlFactory.php: Accept-Encoding suppressed by default](https://github.com/guzzle/guzzle/blob/8.2.0/src/Handler/CurlFactory.php#L2609-L2622)
- [Guzzle 8.2.0 Client.php: a string decode_content becomes the header](https://github.com/guzzle/guzzle/blob/8.2.0/src/Client.php#L1582-L1586)
- [Guzzle request options: decode_content](https://docs.guzzlephp.org/en/stable/request-options.html#decode-content)
- [Laravel 13.33.0 PendingRequest.php: a plain Guzzle client](https://github.com/laravel/framework/blob/v13.33.0/src/Illuminate/Http/Client/PendingRequest.php)
- [Cloudflare: Content compression](https://developers.cloudflare.com/speed/optimization/content/compression/)
- [Apache HTTP Server 2.4: mod_deflate and proxy servers](https://httpd.apache.org/docs/2.4/mod/mod_deflate.html)
- [nginx: ngx_http_gzip_static_module (gzip_static always)](https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html)
- [OpenSearch issue #17339: REST API hangs with Accept-Encoding zstd](https://github.com/opensearch-project/OpenSearch/issues/17339)
- [MDN: Accept-Encoding](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Accept-Encoding)
- [Chrome Platform Status: Zstd Content-Encoding (Chrome 123)](https://chromestatus.com/feature/6186023867908096)
- [Web Almanac 2021: Compression](https://almanac.httparchive.org/en/2021/compression)
- [Decodo help: how proxy traffic is calculated (vendor documentation)](https://help.decodo.com/docs/proxy-traffic)

## Related guides

- [Use a proxy with curl: -x, env vars, SOCKS5, auth](https://ipvolt.com/guides/curl-proxy-setup.md)
- [Python Requests proxy: proxies dict, auth, SOCKS5](https://ipvolt.com/guides/python-requests-proxy.md)
- [Use a proxy with Node.js fetch](https://ipvolt.com/guides/nodejs-fetch-proxy.md)
- [HTTPX async proxies: setup and PoolTimeout diagnosis](https://ipvolt.com/guides/httpx-async-proxy.md)

## About ipvolt

Examples use generic proxy settings, with links to the original technical documentation. Product-specific behavior must be checked with your provider. ipvolt is still in development.

## Know when access opens.

ipvolt · In development

We’re building proxy infrastructure for developers and data teams. Join the interest list for a heads-up when ipvolt is ready.

Consent: One email when access opens. Nothing else.

[Get early access](https://ipvolt.com/guides/curl-compressed-accept-encoding#waitlist-closing). Use the email form on this page to join the interest list.

[Privacy](https://ipvolt.com/privacy)