# SSL EOF behind a proxy: loopback lab

This download reproduces `EOF occurred in violation of protocol`, `[SSL: UNEXPECTED_EOF_WHILE_READING]`,
`SSLEOFError`, curl's `error:0A000126` and `ProxyError('Unable to connect to proxy', RemoteDisconnected(...))`,
and counts what each failure and each retry setting really sends. It backs the guide at
https://ipvolt.com/guides/fix-ssl-unexpected-eof-proxy. The same questions through named open-source proxy software on
loopback are in `ssl-eof-real-proxy.zip`, at the same address as this archive.

Everything listens on `127.0.0.1`. No proxy account is needed and no packet leaves the machine. The results
show how clients behave against this fixture. They say nothing about what any proxy provider does.

Target names end in `.test` and do not resolve. Only the fixture's proxies know them and they dial loopback
listeners only, so a request that reaches a target by name went through a proxy. Each origin looks up its peer
port in the proxies' tunnel table and logs which proxy carried the request. The proxy login is `labuser` /
`dummy-pass` everywhere. The certificates are created for each run and deleted afterwards.

| Listener | Port | Role |
| --- | --- | --- |
| `origin` | 28950 | TLS target. Logs every request, then answers or drops it according to its path |
| `close_target` | 28951 | reads what should be a ClientHello, then closes (also the `https://` proxy that closes in its handshake) |
| `rst_target` | 28952 | reads the ClientHello, then RST |
| `block_target` | 28953 | serves direct clients, closes clients that arrive through a tunnel |
| `relay` | 28954 | honest CONNECT proxy: 200 only after the target answers, 502 or 504 otherwise. Also forwards plain `http://` requests |
| `close_early` | 28955 | 200, then closes without reading the ClientHello |
| `close_hello` | 28956 | 200, reads the ClientHello, then closes |
| `optimistic` | 28957 | 200 first, then dials the target, and closes if that fails |
| `rst` | 28958 | 200, reads the ClientHello, then RST |
| `silent` | 28959 | 200, then silence. Its socket times out after 15 s and closes, so a curl with a longer `--max-time` sees exit 35, not 28 |
| `noreply` | 28960 | closes with no reply to CONNECT |
| `tls_proxy` | 28961 | working `https://` proxy |
| `socks` | 28962 | SOCKS5 relay with username and password, names resolved at the proxy |
| `mixed` | 28963 | the first tunnel relays, later tunnels answer 200 and close |
| `cut` | 28964 | relays, then cuts the tunnel after about 300 kB of upload |
| `blackhole` | 28965 | relays the handshake, swallows the request, then closes |
| `plain_origin` | 28966 | plain HTTP target behind `relay` |
| `h2_origin` | 28967 | HTTP/2 target (only if the `h2` package is installed) |
| `dead` | 28968 | never bound, so nothing answers there |
| `outage` | 28969 | new tunnels answer 200 and close during a set outage, then relay again |
| `earlydata` | 28990 | the first tunnel relays. Later tunnels pass the client's bytes on, swallow the server's flight, then close |
| `earlydata_target` | 28991 | `openssl s_server` with early data on. Section `X` starts it for one cell |
| `tls_proxy_rst` | 28992 | `https://` proxy: 200, reads the ClientHello, then RST |
| `hello_limit` | 28993 | 200, then closes if the client's first read is above 1,000 bytes, and relays otherwise |
| `delayed` | 28994 | relays the handshake and holds the request. It closes the client 50 ms later and delivers the request 400 ms after it first held it |
| `connect_rst` | 28995 | reads `CONNECT`, then RST |
| `first_served` | 28996 | TLS target. Serves the first connection, and later ones close on the ClientHello. It resets after 3 s idle |

`control.test` is relayed honestly by every misbehaving proxy port, so each behaviour has a control on the same
port, with the same client and the same trust settings.

Target paths choose the failure: `/ok`, `/drop-fin` and `/drop-rst` (the request arrives, then the tunnel ends
with no response), `/notify` (TLS `close_notify` with no response), `/mid-cl-fin`, `/mid-cl-rst`, `/mid-cl-notify`,
`/mid-chunked-fin`, `/mid-close-fin` (the body stops partway), `/flip` (a corrupted record), `/hang` (no answer)
and `/slow/MS`. Section `X` adds these:

- `/redir/CODE?to=URL` answers CODE with that `Location`, `Content-Length: 0` and `Connection: close`, then closes, so a next hop
  on the same host needs a new tunnel.
- `/garble` reads the request, writes a bad TLS record straight on the raw socket, then closes.
- `/close-after-head` closes right after the request head and leaves the body unread.
- `/bad-status` writes `HTTP/1.1 20`, then closes.
- `/continue-close` writes `HTTP/1.1 100 Continue`, then closes.
- `/redir-self/CODE` answers CODE with `Location` set to the URL just posted, then closes.
- `/redir-once/CODE` does the same the first time, and answers 200 the next time.
- `/drop-now` is `/drop-fin` without its 20 ms pause.
- `/auth-digest` answers 401 with `WWW-Authenticate: Digest ...` and `Connection: close`, then closes.
- `/status/CODE` answers CODE with no body and keeps the connection open.

The target also reads `Transfer-Encoding: chunked` bodies and logs the decoded size.

## Run it

Requirements: Python 3.10 or later (recorded on 3.14 and 3.13), the `openssl` command, and curl. Node.js 24 is
optional.

```sh
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
.venv/bin/python lab.py --out local-results.json
```

A full run with the defaults took about 25 minutes here, much of it waiting on timeouts and backoff. Use
`--sections` to run a part of it:

| Section | What it records | `results.json` key |
| --- | --- | --- |
| `A` | the printed error per behaviour and client, controls, origin receipts | `printed_errors`, `controls`, `pip_extras`, `node_fetch_once`, `wire_samples`, `printed_errors_other_builds` |
| `B` | curl: target via proxy, known-good via proxy, target direct | `three_way` |
| `C` | retry table: settings by path, failure and method | `retry_table` |
| `M` | first tunnel delivered then dropped, later tunnels 200 then close | `mixed_sequence` |
| `O` | HTTPX `retries`, aiohttp defaults, curl `--retry` and `--retry-all-errors` | `other_clients` |
| `D` | the narrow retry against each failure, per build | `narrow_retry` |
| `E` | backoff against an outage, with attempt times | `backoff` |
| `T` | a job running while the exit changes | `timeline` |
| `F` | HTTP/2 through the tunnel | `http2` |
| `G` | wording and `_ssl.c` line per build and stage | `wording_by_build` |
| `H` | where the proxy password shows up | `credentials` |
| `I` | the probe script against the fixture, including what it does with no proxy set | `probe` (`probe.primary.no_proxy_refuses`) |
| `S` | the source lines behind the retry behaviour | `source_lines` |
| `X` | redirects, short `Retry` forms, a bad record, cut uploads, wrong fixes, the helper in application code, curl extras, curl_cffi, aiohttp methods, proxy settings from the environment, tokens in output, wording per interpreter, and the keys listed below | `additions` |
| `V` | every urllib3 release from 1.25.1 to 2.8.0. It runs only with `--sweep-python` | `urllib3_sweep` |

Section `X` also records these keys under `additions`:

| Key | What it records |
| --- | --- |
| `redirect_same_url` | a POST redirected to its own URL, then every later handshake dies, on three routes |
| `redirect_other_clients` | `urllib.request` and aiohttp on a redirect to a next hop that dies |
| `redirect_once` | a default Requests session re-sends a POST on 307 and 308 |
| `helper.<build>.own_loop` | your own retry loop around `died_in_handshake`, with controls |
| `helper.<build>.body_opens_https` | a body generator whose own HTTPS call dies in its handshake |
| `helper.<build>.died_in_handshake_httpx` | `died_in_handshake` on what HTTPX raises, on the primary Python |
| `short_forms` | also `Retry(total=3, connect=0, read=0, redirect=0, status=0)`, POST added to `allowed_methods`, and an empty `allowed_methods` |
| `retry_total_none` | `Retry(total=None, connect=3, read=3)`, stopped at 15 s if it hasn't ended |
| `pm_retries` | urllib3 `ProxyManager` used directly, and `retries=False` |
| `https_proxy_route` | a delivered POST through an `https://` proxy URL |
| `delayed_delivery` | the POST reaches the target after the client has printed its error |
| `hello_limit` | what passes a proxy that closes on a large ClientHello, per curl, with `--curves X25519`, per Python, and the probe |
| `three_way_more` | more rows for the comparison block, in keys apart from section `B` |
| `writeout_more` | more curl write-out cases: a reset on `CONNECT`, `--head`, `--retry` into a dead handshake, `location` in `.curlrc`, an auth round, retried statuses |
| `malformed_proxy_url` | what each client prints for a proxy URL it can't parse |

Options:

- `--curl PATH` (repeatable) adds a curl build to every curl cell.
- `--python PATH` (repeatable) adds an interpreter to the wording table, the probe, and, if it has `requests`,
  the narrow-retry check, a small error subset and the per-interpreter cells of section `X`.
- `--retry-python PATH` (repeatable) does the same and also runs the full retry table. To test urllib3 1.26,
  make a second venv from `requirements.txt`, run `pip install urllib3==1.26.20` in it, and pass its Python here.
- `--sweep-python PATH` (repeatable) adds an interpreter with urllib3 and PySocks to section `V`, one per release. The recipe is
  below.
- `--repeat N` sets the runs per cell (default 3; cells that race get three times as many).
- `--behaviours a,b` runs only those rows in section `A`.
- `--raw PATH` also writes every cell uncompacted, with its wire summary.
- `--skip-node`, `--base-port N`.
- `--serve` only starts the listeners and prints the proxy and target for each behaviour, so you can try commands
  by hand. Pass curl `--cacert` with the CA it prints. It doesn't show what the target received, so use `--sections C` for
  counts. The mixed listener resets after 3 s idle, so wait between runs.

To run the page's shell blocks as written against `--serve`:

1. Run `export CURL_CA_BUNDLE=` followed by the path of the CA it prints.
2. Set `PROXY_URL` and `TARGET_URL` from a row it prints.
3. Set `KNOWN_GOOD_URL=https://known-good.test:PORT/ok`, where PORT is `origin`'s.
4. `.test` names resolve only through the proxies, so change the third line of `compare` to `probe --noproxy '*' https://127.0.0.1:PORT/ok`, with the same PORT.

The first block needs no change.

## Files

- `lab.py`: the fixture and the runner. The fixture needs only the standard library.
- `h2_origin.py`: the HTTP/2 target. It needs `h2`, and the rest still runs without it.
- `client_requests.py`, `client_urllib3.py`, `client_httpx.py`, `client_aiohttp.py`, `client_urllib.py`,
  `client_fetch.mjs`: one request each, printing the outcome as JSON. `client_common.py` is shared by the Python ones.
- `client_retry.py`: the retry settings, plus a read-only trace of how urllib3 classified each failure.
- `client_plain.py`: an uninstrumented script, to show what a reader sees with and without logging.
- `client_misc.py`: section `X` cases that need a few lines of their own, including fixes shown because they do not help.
- `client_sweep.py`: urllib3 alone, for section `V`. It needs urllib3 and PySocks, not Requests.
- `client_curl_cffi.py`: one curl_cffi session. The lab skips it if the primary Python can't import `curl_cffi`, and
  `requirements.txt` installs it only on CPython 3.10 or later.
- `client_timeline.py`: a job that keeps sending GETs while the exit changes.
- `client_wording.py`: standard library only, so it runs on old interpreters.
- `client_sources.py`: prints the source lines behind the behaviour from the packages installed here.
- `retry_handshake_eof.py`: the narrow retry helper. It relies on a private urllib3 method.
- `ssl_eof_probe.py`: a standard-library probe to run against your own proxy. It sends one `HEAD` and never prints
  the proxy login.
- `results.json`: the recorded runs, under `runs.linux` and `runs.macos`.

## Sweep urllib3 releases

Section `V` needs one venv per release. With `uv`, using a Python 3.11 it can find:

```sh
python3 -c "import json,urllib.request as u; d=json.load(u.urlopen('https://pypi.org/pypi/urllib3/json')); k=lambda v: tuple(map(int,v.split('.'))); print('\n'.join(sorted((v for v in d['releases'] if v.replace('.','').isdigit() and d['releases'][v] and (1,25,1)<=k(v)<=(2,8,0)),key=k)))" > versions.txt
for v in $(cat versions.txt); do
  uv venv --python 3.11 "sweep/$v"
  uv pip install --python "sweep/$v/bin/python" "urllib3==$v" PySocks==1.7.1
done
.venv/bin/python lab.py --sections V --out sweep-results.json $(for v in $(cat versions.txt); do echo --sweep-python "sweep/$v/bin/python"; done)
```

With plain `pip`, replace the two lines inside the loop:

```sh
  python3.11 -m venv "sweep/$v"
  "sweep/$v/bin/pip" install "urllib3==$v" PySocks==1.7.1
```

## The probe

```sh
export HTTPS_PROXY=http://USERNAME:PASSWORD@proxy.example.net:8080
python3 ssl_eof_probe.py https://example.com/
```

It prints the stage reached, the `CONNECT` status, where it failed and whether the request was sent. Through an
`https://` proxy URL it stops after `CONNECT`, because it does not run TLS inside TLS. It speaks HTTP `CONNECT` only, so don't
point it at a SOCKS proxy.

With no proxy set it stops with exit 2 and an error, so it can't report a direct connection as a pass. Pass `--no-proxy` to test from your
own address.

## The helper

`retry_handshake_eof.py` re-sends only when the TLS handshake died or the connection never opened. It doesn't do these things:

- It doesn't protect a POST you add to `allowed_methods`. POST to `/drop-fin` with `HandshakeEOFRetry(total=3, other=0, allowed_methods={"POST"})` gave 1 receipt through the HTTP proxy and 4 through SOCKS5h and with no proxy (`additions.short_forms.<build>.helper_post_allowed`).
- `died_in_handshake` in your own code tests the last attempt only. With redirects or a mounted `Retry` on, an earlier attempt can have reached the target (`additions.helper.<build>.own_loop`).

The helper changed on 5 November. An error that came up through urllib3's `request()` no longer counts as a handshake death, so a body generator that opens HTTPS can't trigger a re-send
(`additions.helper.<build>.body_opens_https`). Sections `C`, `M`, `D`, `E`, `I` and `X` were re-recorded with the changed helper and probe.

## Reading `results.json`

A cell holds what was printed (`printed`, or `error` as an index into its block's `errors`), `connects` (CONNECT
requests at the proxy), `origin_requests` (complete requests at the TLS target), `runs`, and `variants` when
repeated runs differed. `buckets` lists how urllib3 classified each failure and whether it retried. Addresses are
written as `0xADDR` and local paths as `<tmp>`, `<lab>`, `<venv>` or `<python>`. The password in the results is
the dummy one. Cells under `additions` and `urllib3_sweep` also hold `origin_heads` (request heads at the target), `receipts_by_request`
and `bodies` (the size of each complete body the target read). Some cells under `additions` hold `passes` (what `died_in_handshake` said on each pass of a loop),
`first_read_bytes` (what `hello_limit` read in that run), `capped_at_s` (the wall-clock cap, if it stopped the call), `client_failed_after_ms`, `receipts_when_client_saw_it` and `receipts_1s_later`.

This prints the POST counts of section `C`, most of the page's retry table, from a local run. `retry_connect3` there is `Retry(total=None, connect=3, read=0, other=0, status=0, redirect=0)`, not the page's `Retry(connect=3)`. The page's short forms are in `additions.short_forms`, from section `X`, and its curl and HTTPX rows in `ssl-eof-real-proxy.zip`. Section `C` alone took about 3.5 minutes per interpreter here.

```sh
python3 -c "import json; d=json.load(open('local-results.json')); b=next(iter(d['retry_table'].values())); [print(k, v['POST']['origin_requests']) for k, v in b['http_proxy']['delivered_fin']['settings'].items()]"
```

## A limit on `time_appconnect`

On a new connection with TLS early data off, `time_appconnect` 0 with a non-zero exit always meant the attempt that printed
it was not sent. A reused connection reports 0 as well, so the rule stops there. The cell is
`additions.curl_extras.<curl>.reused_appconnect`: one curl with `https://origin.test:PORT/ok`, `--next`, and a POST to `/mid-cl-fin`, or to
`/hang` with `--max-time 2`, on the same proxy. It records `connect=000 appconnect=0.000000`, `%{num_connects}` 0 and the POST
delivered.

The rule covers that one attempt and nothing earlier. With two URLs, a POST lost on the reused connection and re-sent into a dead
handshake prints exit 35 and `time_appconnect` 0 with the POST delivered (`additions.curl_extras.<curl>.two_urls_dead_handshake`).
After `-L` from an `https://` first hop, exit 35 comes with `time_appconnect` above 0 and the POST at the first URL
(`additions.redirect_then_dead_hop.curl`). With `--tls-earlydata`, the POST reached the server with exit 35 and `time_appconnect` 0
(`additions.curl_extras.<curl>.earlydata`).

## Recorded run

Recorded on 4 November 2026. Sections C, M, D, E were re-recorded and X, V added on 5 November after the helper changed. Later on 5 November sections C, M, D, E, I and X were
recorded again after the helper and the probe changed, and X gained cells. V stays as first recorded. `results.json` holds two runs, and `sections_recorded_utc` in each gives the time per
section. The builds are the same in all the recordings.

`runs.linux`, Ubuntu 26.04 (x86_64):

- curl 8.18.0 (OpenSSL 3.5.5) and Node.js 24.22.1 (one `fetch` cell)
- Requests 2.34.2, urllib3 2.8.0, HTTPX 0.28.1, httpcore 1.0.9, h2 4.4.1, aiohttp 3.14.5, pip 26.3 and PySocks 1.7.1 on
  Python 3.14.4 (OpenSSL 3.5.5)
- the retry table again on urllib3 1.26.20, and on urllib3 `main` at commit
  `c5347bbd39203d68d7b0e3ffb83ced3e3b4b9109` (it reports itself as 2.8.1.dev35)
- the narrow retry and wording on Python 3.9.25 (OpenSSL 3.5.4), 3.10.11 and 3.11.3 (OpenSSL 1.1.1s), 3.12.3
  (OpenSSL 3.0.13) and 3.15.2 (OpenSSL 3.5.9); wording only on 3.13.16 and 3.14.10 (OpenSSL 3.5.9)

`runs.macos`, macOS 15.7 (arm64):

- curl 8.7.1 (LibreSSL 3.3.6) and curl 8.23.0 (OpenSSL 4.1.0)
- the same client versions on Python 3.13.18 (OpenSSL 3.6.5), with the retry table on urllib3 2.8.0 and 1.26.20
- the narrow retry and wording on Python 3.9.6 (LibreSSL 2.8.3, Requests 2.32.5, urllib3 2.6.3), 3.9.25 (OpenSSL 3.5.4,
  same packages), 3.10.11 and 3.11.3 (OpenSSL 1.1.1s) and 3.15.1 (OpenSSL 4.1.0); wording only on 3.12.15
  (OpenSSL 3.5.9) and 3.14.10 (OpenSSL 4.1.0)

Section `X` also ran curl_cffi 0.17.0 (it bundles libcurl 8.23.0 with BoringSSL) and socksio 1.0.0 on the primary Python on each OS, and
PyPy 3.11.16 (PyPy 8.0.0, OpenSSL 3.3.1) with Requests 2.34.2 and urllib3 2.8.0, recorded in `additions` under labels that start `pypy-`.
Section `V` ran on Python 3.11.3 (OpenSSL 1.1.1s) on each OS.

The certificates were created for each run and deleted afterwards.

## What this does not cover

Windows; any commercial proxy, mobile device or real network; a target's or proxy's own reasons for closing; TLS
libraries other than OpenSSL and LibreSSL, apart from the one curl_cffi cell on BoringSSL. The relay turns a target's RST into a FIN towards the client. So did every
relay in the named-software run, which is still a small set of programs. Loopback timings say nothing about real paths. The HTTP/2 target is a small script on the `h2`
package.