How to use a proxy with curl: the -x flag, http_proxy and https_proxy variables, SOCKS5 with socks5h, proxy authentication, and reading CONNECT and 407 errors.
ipvolt / field notesDeveloper library
Build a betterconnection.
Practical proxy knowledge, from the first request to the details that matter. Clear examples. Useful distinctions. Fewer assumptions.
Start with your stack- Your client
cURL · Python · Node.js · Playwright
Protocol & authentication
- Proxy gateway
Connection · Credentials · Session
Request & response
- Destination
The service you’re connecting to
A general proxy flow. These guides are independent of any ipvolt service availability.
Integration
Connect your stack.
Explicit configuration, bounded requests and a clear way to verify the connection.
Configure a proxy in Python Requests: the proxies dictionary, Session defaults, credentials, SOCKS5 via requests[socks], environment variables and ProxyError.
Route Node.js native fetch through an Undici ProxyAgent with explicit authentication, an abort deadline and response cleanup.
Set an authenticated HTTP proxy for a Playwright browser context, keep test state isolated, and diagnose navigation separately from subresources.
Configure an HTTPX async proxy client, close streamed responses correctly, and reproduce a PoolTimeout locally before changing your proxy settings.
Handle Amazon SP-API listing events with a tested SQLite inbox: deduplicate deliveries, recover after restart and reject superseded reads.
Run 12 Australian browser-clock checks in Playwright, including Lord Howe’s 30-minute change. Separate fixed UTC inputs, browser time zones and the Node runner.
Use a configuration snapshot and paired visits to test Shopify Canada localization. Separate IP, URL, language, saved choice and shipping-address effects.
curl, Wget 1.x, urllib, node:http, PHP curl, Guzzle and java.net.http.HttpClient send no Accept-Encoding (or identity) by default. Tested fixes and byte checks.
Configure Axios proxies in Node.js, separate explicit settings from environment and agent routing, and reproduce authentication and TLS checks.
Set up an HTTP or SOCKS proxy in aiohttp 3.14: replace the deprecated BasicAuth and proxy_auth, turn on trust_env, and read the 407 and Bad status line errors.
Scrapy 2.19 proxy setup: credentials in meta["proxy"], per-request rotation, HTTP_PROXY, and fixes for TunnelError 407, ignored 407 responses and socks5.
curl_cffi proxy setup on 0.16.3: credentials with impersonate, why a CONNECT 407 raises ConnectionError and not ProxyError, HTTP_PROXY rules, socks5h.
Block images, fonts and media in Playwright with route() in Python and Node.js, for a whole browser context or by host, plus the cache and service worker caveats.
Configure a Git HTTP or HTTPS proxy, diagnose 407 and certificate failures, and remove the correct setting. Includes a tested routing fixture.
Set npm proxy and https-proxy, resolve npmrc and environment conflicts, verify package downloads, and fix trust or routing without disabling TLS checks.
Concept
Understand the tradeoffs.
Choose a protocol, session model or network type with the right questions in mind.
Compare HTTP CONNECT and SOCKS5, understand DNS placement and TLS boundaries, and select a protocol your actual client and provider support.
Choose rotation around the smallest complete workflow, distinguish exit affinity from cookies, and test what happens when a sticky session ends early.
Compare carrier and household exit networks for QA, keep browser emulation separate, and measure the capabilities your workflow actually needs.
Troubleshooting
Find the failure.
Separate authentication, connectivity and destination errors before changing your setup.
Diagnose Proxy Authentication Required in a repeatable order: gateway, authentication method, credential encoding, account scope and client configuration.
Diagnose HTTP_PROXY, HTTPS_PROXY, ALL_PROXY and NO_PROXY routing differences in curl, Python Requests and Node.js with an isolated local check.
Separate proxy DNS, TCP, CONNECT, TLS and response delays with curl timings, then set request deadlines and decide whether a retry is safe.
ERR_TUNNEL_CONNECTION_FAILED and ERR_PROXY_CONNECTION_FAILED explained with a verified proxy failure matrix, plus how to recover the CONNECT status Chromium hides.
Decode Node.js TypeError: fetch failed behind a proxy: print the err.cause chain, fix an ignored HTTPS_PROXY or undici skew, and read a proxy 403, 407 or 502.
Node's built-in fetch silently skips HTTP_PROXY and HTTPS_PROXY, so requests time out with UND_ERR_CONNECT_TIMEOUT while curl works. Here's why, and how to fix it on every Node version.
The same SSL wrong version number error comes from two hops behind a proxy. Tell which one failed in curl, Requests and HTTPX, then apply the right fix.
Fix InvalidSchema: Missing dependencies for SOCKS support in Python Requests, pip and HTTPX. Install requests[socks], or clear the ALL_PROXY variable behind it.
Scrapy 2.16 removed Spider.start_requests(): the spider logs Crawled 0 pages and closes as finished with no warning. Fix: async def start(). Tested on 2.19.
Evaluate local networks, exit locations and the evidence a country-specific proxy test needs.
Evaluate Netherlands proxies with Dutch network context, exit-IP and city checks, ASN evidence, and a practical trial checklist before choosing a provider.
Evaluate USA proxies by country, state and city. Compare network evidence, plan regional checks and measure session behavior before choosing a provider.
Evaluate UK proxies with current carrier context, UK and GB geography, postcode limitations and a practical test plan for country or network requirements.
Evaluate Germany proxies with current carrier and roaming context, city-location limits, browser-language controls and a practical acceptance checklist.
Evaluate France proxies with carrier and brand distinctions, mainland and overseas scope, exit-IP evidence, and a practical localization test matrix.
Brazil proxies, mobile or residential: what the label proves, Vivo, Claro and TIM context, states and time zones, CEP and CPF inputs and a trial plan.
Specify and verify Portugal proxies: mainland and island scope, ANACOM operator names, pt-PT settings, Portuguese checkout fields and a four-row trial.
Choose the evidence an Australian proxy test needs: country or city, Telstra/Optus/TPG network labels, nbn provider layers and session requirements.
Evaluate Canadian proxies with a country, carrier and session acceptance matrix. Separate Rogers, Bell, TELUS and Quebecor brands from the evidence your test needs.
Evaluate Japan proxies with separate IP-location, egress and mobile-access evidence. Work through au/UQ branding and Rakuten roaming before accepting a sample.
Developer tools
Inspect your traffic.
Open a report and understand what your scraper transferred.
Free open-source tool that shows where your scraper's proxy bandwidth goes and finds the smallest response containing your data. Works with any provider.