# Cloudflare AI-agent response probe

A pre-ingestion diagnostic for one authorized HTML target, using an authenticated HTTP proxy and Playwright Chromium. It reports one JSON record and never emits the target URL, response body, proxy credentials or raw exception text. It does not extract content, retry at the application level, solve challenges or make payments. Browsers can still perform authentication handshakes, redirects and internal transport retries.

## Install and run

Use Node.js 22 or later, npm, and curl in a new directory. The lockfile pins Playwright 1.63.0.

```sh
curl -fSLO https://ipvolt.com/downloads/cloudflare-ai-agent/package.json
curl -fSLO https://ipvolt.com/downloads/cloudflare-ai-agent/package-lock.json
curl -fSLO https://ipvolt.com/downloads/cloudflare-ai-agent/probe.mjs
npm ci
npx playwright install chromium
```

Populate these environment variables through your existing secret/configuration mechanism. Keep credentials out of command history and disable Playwright debug logging. The examples below are placeholders, not ipvolt endpoints or credentials.

| Variable | Required value |
| --- | --- |
| `PROXY_SERVER` | Explicit HTTP proxy URL such as `http://proxy.example:3128`, without embedded credentials, a path or query. |
| `PROXY_USERNAME` | Your proxy account username, including any provider-defined session parameter if applicable. |
| `PROXY_PASSWORD` | Your proxy account password. |
| `TARGET_URL` | One authorized `https://` or `http://` page. |
| `EXPECTED_SELECTOR` | A site-specific CSS selector for expected content, for example `main article[data-document-id]`. It must match an element containing non-whitespace text. |

```sh
node probe.mjs
```

Navigation has a 15-second timeout; content may take another 5 seconds. Each invocation creates and closes a fresh context. Cookies persist inside that context only. A fixed proxy endpoint does not establish a stable egress IP; choose the provider's session mode separately. TLS certificate validation remains enabled. Linux environments may also require the operating-system packages listed in the [Playwright browser installation documentation](https://playwright.dev/docs/browsers#install-system-dependencies).

## Interpret the result

Successful synthetic example:

```json
{"status":200,"outcome":"accepted_content","cf_mitigated":null,"cf_ray":null,"retry_after":null,"transport_code":null}
```

The process exits `0` only for `accepted_content`; all other reported outcomes exit `2`. Treat these outcomes as a diagnostic before your existing extraction pipeline. This probe closes its context and returns no content; an accepted probe cannot approve a later fetch. Apply the checks to the exact response and content you will extract or ingest:

| Outcome | Meaning and next action |
| --- | --- |
| `accepted_content` | The observed main document is HTTP 200 with `text/html`, no observed challenge header, and a nonempty selector match. Apply your own content, provenance and authorization checks before ingestion. |
| `challenge` | The main response contains `cf-mitigated: challenge`, even if status is 200. Withhold it from ingestion and review the site's access route. |
| `payment_or_access_review` | HTTP 402. Review the publisher's access terms and any payment requirements; this code alone does not prove Cloudflare Pay Per Crawl. |
| `access_review` | HTTP 403. Review authentication, permissions and site policy; the code alone does not identify a particular blocking rule. |
| `proxy_auth_error` | An HTTP 407 was exposed to Playwright. Review proxy credentials. Chromium can instead hide this response behind a navigation exception. |
| `rate_limited` | HTTP 429. Stop this attempt; use any valid `retry_after` in a separately bounded scheduling policy. |
| `upstream_error` | Main-document HTTP 5xx. Investigate the proxy/origin path; do not attribute it from status alone. |
| `unexpected_status` | A different status, including non-200 success codes. Review it explicitly. |
| `invalid_content` | Non-HTML response, invalid CSS selector, empty/missing expected content, or selector wait failure. Withhold it. |
| `transport_error` | Navigation failed or returned no HTTP response. `transport_code`, when present, is a Chromium network error token. This is not evidence of a Cloudflare policy decision. |
| `navigation_changed` | Another main document appeared during the content check; no stable document was accepted. Investigate the navigation flow. |
| `config_error` | Missing or invalid input configuration. |
| `browser_setup_error` | Chromium/context startup failed. Check installation and runtime configuration. |

`status` comes from the inspected main-document response, or is `null` when navigation did not yield an inspectable response. `cf_mitigated` is `challenge` or `null`. `cf_ray` contains only an accepted Ray-ID-shaped value; other values become `null`. `retry_after` preserves bounded numeric seconds or normalizes an accepted HTTP date; it is not acted on. `transport_code` contains only an `ERR_*` token, never a raw exception. A Ray ID helps the site operator locate a request; it does not name the rule that blocked it.

The selector check is only structural plausibility. A login page, error page, incomplete article or fabricated text can satisfy a weak selector. The probe does not inspect every API, iframe, image or other subresource: a required JSON request can fail while the main document passes. Add validators for the resources and fields your scraper, RAG pipeline or browser agent actually consumes.

## Reproduce the synthetic checks

After the same installation, download and execute the fixture:

```sh
curl -fSLO https://ipvolt.com/downloads/cloudflare-ai-agent/test-fixture.mjs
node test-fixture.mjs
```

The fixture runs the exact `node probe.mjs` command against a local origin through a local authenticating HTTP forward proxy. Its reserved `fixture.invalid` hostname is resolved only inside that proxy, which refuses other destinations. It prints JSON with versions, timestamps, the probe SHA-256, each result and the denominator. The [recorded synthetic results](https://ipvolt.com/downloads/cloudflare-ai-agent/example-output.json) are also available. No live Cloudflare or ipvolt target is contacted. This HTTP fixture does not demonstrate HTTPS CONNECT behavior, TLS transport, provider egress stability, live blocking rates or successful policy bypass.

Implementation references: [Playwright context proxy options](https://playwright.dev/docs/api/class-browser#browser-new-context-option-proxy), [navigation semantics](https://playwright.dev/docs/api/class-page#page-goto), [bounded predicate waits](https://playwright.dev/docs/api/class-page#page-wait-for-function) and [Cloudflare's challenge response header](https://developers.cloudflare.com/cloudflare-challenges/challenge-types/challenge-pages/detect-response/).