Check whether your client requests compression
Compare tested Accept-Encoding defaults and verify the transferred bytes before changing a scraper.
See where your scraper's proxy bandwidth goes. Open the report in your browser.
Find which hosts and resources use your scraper’s bandwidth. Scrapescope is a free, open-source local metering proxy for scrapers and browser agents. Install the tool to measure a run, or open an existing report in the browser viewer below.
Use your own compatible HTTP proxy, or run in direct sizing mode without a proxy account. Scrapescope and this viewer do not require an ipvolt proxy.
From the bundled sample report
- Three pages, as counted
- 783,026 bytes
- Modelled with images, media and fonts blocked
- 127,386 bytes
The sample is a real three-page run on books.toscrape.com, recorded in direct sizing mode. Images and fonts were 655,640 of its 783,026 bytes (84%). The after figure is the report's what-if: a model built from allocated bytes, not a second measured run.
Open a report.
Drop a scrapescope report.json into the frame, choose a file, or load the bundled sample. The report is parsed inside an isolated frame on this page and nothing is uploaded. You can compare two runs, price the bytes at a rate you type and download a share card rendered on your device.
Measure a run. Inspect the report.
pipx install 'scrapescope @ git+https://github.com/ipvolt/scrapescope'
pipx inject --include-apps scrapescope playwright
playwright install chromiumpip install 'scrapescope[browser] @ git+https://github.com/ipvolt/scrapescope' && python -m playwright install chromiumPyPI publication is planned. These Git install commands were checked on macOS; Windows is untested.
After installing, run an existing proxy-aware Python scraper through Scrapescope. Replace my_scraper.py with your own script. Direct sizing uses your machine’s connection; it does not test a proxy provider or predict how a protected target will behave through one.
scrapescope run --direct --env-all --out report.json -- python my_scraper.pyscrapescope report report.jsonOpen report.json in the viewer above to inspect it visually. The wrapper supplies proxy environment variables; your client must honor them or use an explicit proxy configuration. Tunnel totals do not require browser instrumentation, but resource attribution needs the Playwright Python helper or Requests/HTTPX hooks documented in the repository.
For a different task, use scrapescope find URL --value V --direct to look for a response containing a known value. Replace URL with your page and V with the value you need, quoting both in your shell. The --direct option uses your machine’s own connection without a proxy account. This uses the installed Chromium browser and observes the initial page load; check the capture limits and replay result before changing an extraction job.
Full usage, helper setup and upstream-proxy configurationFind captured responses that contain your value.
scrapescope find URL --value V loads a page once in headless Chromium and inspects captured text responses for the values you name, including common encoded forms. It ranks matches by an estimated standalone-fetch transfer size and flags cookies or tokens sent by the browser. It can miss oversized, evicted, binary or worker responses and content loaded after that initial capture. An optional replay checks a candidate without a browser; a match alone does not establish that you can reuse it.
The local meter counts bytes on its connection to an upstream proxy. The browser helper adds request information so the report can allocate those totals by resource type. These are different measurements: a tunnel total is measured; the share assigned to a script or image is allocated. Read the labels before comparing with your provider’s dashboard.
Every host
Tunnel counts and bytes per host, with and without the CONNECT exchange, and the buckets for tunnels no request explains.
Resource types
Bytes per resource type, such as document, script, image, font, media and XHR, allocated from each host's tunnel totals.
Browser background traffic
Traffic the browser started by itself, labelled with a catalogued component such as Chromium's optimisation-guide model downloads, but only when the host is in the evidence-backed background catalog.
What-if, with generated fixes
The modelled effect of blocking images, media and fonts or of denying catalogued background hosts, with fixes for your stack that appear only when the matching problem was measured in your run.
Sizing before you buy
Direct sizing mode runs without a proxy account. It measures that direct run; a protected target may behave differently through a proxy, so it does not predict a proxied run or a bill. See pages per GB measured this way on 34 public pages.
Per-1,000-unit figures
Bytes per 1,000 page loads and, when you give your own rate, an estimated cost per run, per 1,000 units and per 1,000 successful units.
Three labels, one unit, no bill.
Tunnel-measured, allocated, estimated
In upstream mode, per-host totals are tunnel-measured: bytes written to and read from the socket to your provider, TLS included, shown with and without the CONNECT exchange. Direct sizing measures the destination connection and estimates an added CONNECT exchange. Per-request and per-type bytes are allocated from host totals in proportion to sizes reported by the instrumented client. Cost is an estimate: convert the chosen byte total to GB or GiB, then multiply by the rate you entered.
GB means 10⁹ bytes
scrapescope reports in GB = 10⁹ bytes by default; a GiB (2³⁰ bytes) is about 7.4% larger, and a --gib switch reports in GiB instead. Check which unit your provider uses before comparing.
It is not a bill
Providers meter differently: with or without the CONNECT exchange, in GB or GiB, with minimums or rounding, including or excluding failed requests. No reconciliation against a provider's billing has been published yet, so no figure in a report is billing-grade.
Hosted browsers are not covered
Cloud browser services send their proxy traffic from the service's own network, so a local meter cannot see it. scrapescope makes no claims about them.
Blocking can break extraction
Blocking images, media and fonts, or denying background hosts, can change how a page behaves or attract anti-bot scrutiny, and denying Chromium's component updates also stops certificate-revocation and Safe Browsing data. Every what-if is a prediction: compare a second run before relying on it.
How the figures are computed.
Scrapescope is an open-source local metering proxy for scraper and browser-agent traffic routed through it. It measures tunnel bytes by host; an instrumented client adds resource attribution. Its find command ranks captured responses containing a named value by estimated transfer size, and its run command supports a byte budget.
Billed basis
find ranks matches by encoded body bytes + response header bytes + request header bytes + one new TLS handshake estimate of 7,200 bytes for HTTPS. This approximates one standalone fetch of that response on a new connection by a client that accepts the same compression as the browser. It leaves out the CONNECT exchange, redirects and a second handshake after a failed resumption. On HTTP/2 and HTTP/3 there are no separate header sizes, so both header fields are 0 and the match is marked multiplexed.
The 7,200-byte estimate
The model adds a fixed 7,200-byte TLS handshake estimate for HTTPS. Actual certificate chains and connection reuse vary, so a real fetch can move more or less. This is a modelling assumption, not a measurement of the candidate response’s handshake.
Buckets
Tunnels of hosts with no matching request are labelled by rules: background:<id> only for hosts in the background catalog; before_attach for tunnels opened before the helper attached; preconnect_idle for tunnels that did not fail and carried at most 3,072 payload bytes up and 6,144 down, typical of a connection opened in advance and never used; and unattributed otherwise. An uncatalogued host is never called background, and the buckets add up to the with-CONNECT total.
Cost
Costs appear only when you give a rate: the total bytes, with and without CONNECT, divided by 10⁹ (or 2³⁰ with --gib) and multiplied by your rate, plus the same per 1,000 units and, when success is known, per 1,000 successful units. The label is estimated billable transfer.
Turn measured bytes into a cost estimate.
Estimated transfer cost = bytes ÷ 1,000,000,000 × your price per GB. For a comparison, subtract the second run’s bytes from the first run’s bytes, using the same CONNECT basis and the same number of successful units. The viewer has no default rate; enter your own provider’s rate.
Hypothetical example: a first run uses 200,000,000 bytes and a second uses 150,000,000 bytes for the same 100 successful page loads. At an assumed $4 per GB, their estimated transfer costs are $0.80 and $0.60. The difference is $0.20 per 100 successful loads, or $2 per 1,000 if the same result repeats. These are invented inputs to demonstrate the formula, not measured savings or a provider quote.
Check extraction results before treating fewer bytes as an improvement. Use your provider’s GB or GiB unit and billing rules, account for minimum charges, and compare the estimate with the time required to implement and verify a change.
Your report stays on your machine.
Your report is parsed in your browser inside an isolated frame and never uploaded; nothing about it is sent to analytics. If you accept optional analytics, this page records only fixed labels: that a report was opened from a file, a drop or the bundled sample and whether it parsed; that two runs were compared; that a share card was downloaded; that an install command was copied; and that the link to the homepage was used. Hostnames, byte counts, prices and file names never leave the frame.
Privacy and analyticsBuilt on ideas others implemented first.
Scrapescope builds on existing work in request inspection, local proxy traffic statistics and analysis of Chromium background traffic. The project README credits those sources and documents the measurement model and its limits.
A few clear answers.
Do I need an ipvolt account or proxy to use scrapescope?
No. Use a compatible HTTP proxy or direct sizing mode without a proxy account. Scrapescope has no default provider or rate. This viewer opens the report.json files the tool writes.
Is the viewer free, and what does it send?
Yes. The page and the tool are free. The report is parsed inside an isolated frame in your browser and is never uploaded. Optional analytics, if you accept it, receives only fixed labels, never hostnames, byte counts, prices or file names.
Why do the figures differ from my provider's dashboard?
scrapescope counts every byte on its socket to your provider, TLS included. A provider may count a different subset: with or without the CONNECT exchange, in GB or GiB, with rounding or minimums, including or excluding failed requests. Treat the report as a measurement on your machine, not a bill.
Which clients does it cover?
Clients configured to route their HTTP or HTTPS requests through Scrapescope get per-host totals. Requests that bypass the meter are not counted. Resource attribution needs the Playwright Python helper or Requests/HTTPX hooks. Hosted browsers are not covered because their traffic does not pass through your local meter.
Can I share a report without exposing my targets?
The share card is rendered on your device, and you choose whether to download it. The tool itself keeps query strings out of reports and can replace hosts with catalog ids or keyed hashes (--redact-hosts). Check a report before you share it; it is your data.
Does blocking always reduce the bytes?
No. The what-if is a model of your measured run. Blocking can change how a page behaves, break extraction or attract anti-bot scrutiny, and a page that loads more on a second visit can move the numbers the other way. The second, compared run is the measurement; the what-if is a prediction.
Need a mobile IP in a specific country? Dedicated mobile proxies from $9 a day.
Use the tools today.
Scrapescope is free to use with a compatible proxy or in direct sizing mode. If you also need a provider, ipvolt sells mobile proxies, and residential proxies open soon.