# Python 3.15 scraping stack: install and smoke test

This download backs the post at https://ipvolt.com/blog/python-3-15-web-scraping.
It installs 19 common scraping packages on Python 3.15 with pip and with uv, one
fresh virtual environment per case and no package cache, then runs a small smoke
test for each against a local `http.server` on `127.0.0.1`. No proxy account is
needed; the only network traffic is the package downloads from PyPI.

## Files

- `lab315.py`: the runner. Writes one JSON file per run.
- `smoke315.py`: one smoke test per install spec (import plus one trivial call;
  Scrapy runs one spider with `async def start()`).
- `summarize315.py`: merges runs into `results.json` and `results.csv`.
- `wheels315.py`: lists which packages have cp315 wheels on PyPI right now.
- `results.json`, `results.csv`: the recorded runs (14 October 2026).
- `results-2026-10-16.json`: a rerun without a compiler on 16 October 2026 of
  the Scrapy, `--only-binary brotli` and zope.interface cases
  (`--only scrapy only-binary-brotli only-binary-brotli-pinned zope.interface`).
  Same results as 14 October except zope.interface: 8.7 (15 October) has cp315
  wheels, so it installed as a binary wheel with its C extension.
- `pullimage.py`, `inroot.sh`: how the recorded runs got a `python:3.15-slim`
  file system on a host without Docker (see below).

## Run it with Docker

```sh
docker run --rm -it -v "$PWD":/lab python:3.15-slim bash
# inside the container:
python3 -m venv /opt/uv && /opt/uv/bin/pip install uv && ln -s /opt/uv/bin/uv /usr/local/bin/uv
python3 /lab/lab315.py --label no-compiler --out /lab/results-no-compiler.json
apt-get update && apt-get install -y --no-install-recommends gcc libc6-dev
python3 /lab/lab315.py --label compiler --out /lab/results-compiler.json
```

Each run takes a few minutes; the compiler run is slower because pip and uv
build brotli, zstandard and zope.interface from source. Leave out uv with
`--tools pip`, or run single cases with `--only scrapy "httpx[brotli]"`.

## How the recorded runs were made

The host had no Docker. `pullimage.py library/python 3.15-slim DIR` downloaded
the official image's layers from Docker Hub (linux/amd64, index digest
`sha256:ea4f1f290e071bf7d28199cfb3e09f5dbad31b50e9c32fd58f2f2d7be915f3ae`)
and unpacked them. `inroot.sh DIR CMD` runs a command in that file system in a
private mount and PID namespace (`unshare` + `chroot`, needs root). One copy
stayed as shipped (no C compiler); a second copy got
`apt-get install -y --no-install-recommends gcc libc6-dev`.

## Reading the results

`package_origin` (CSV) and `origin` (JSON) come from the wheel tag recorded in
each installed package's `.dist-info/WHEEL` file:

- `binary wheel`: a manylinux wheel from PyPI, compiled by the maintainers.
- `pure-Python wheel`: a `py3-none-any` wheel.
- `built from source here`: pip or uv built it from the source archive on this
  machine (tag `cp315-cp315-linux_x86_64`). Without a compiler that only works
  for packages whose C code is optional, like zope.interface.

One exception: Playwright's file on PyPI is tagged `py3-none-manylinux1_x86_64`
(it bundles a platform-specific driver), but its `WHEEL` file says
`py3-none-any`, so it is listed as a pure-Python wheel. The `file` field in
`results.json` has the real file name for pip installs.

`last_error_or_warning` is the last error or warning line of the install
output; the full tail is in `results.json`.