Can your website use pay-per-crawl today?
Check admission before budgeting revenue. As checked November 10, 2026, Cloudflare still describes Pay Per Crawl as a closed beta. Its official overview directs interested publishers to a signup route, or an account executive for existing Enterprise customers. A Cloudflare account alone does not establish admission.
Cloudflare’s current beta status is the starting point. This guide does not imply Data Mining Masters or your domain has been accepted.
What an admitted publisher needs
- An active Cloudflare account with the domain added.
- A domain using Cloudflare nameservers or DNS managed by Cloudflare.
- Administrator or Super Administrator permissions for account setup.
- An administrator with access to all domains to make the relevant domain’s controls visible.
The documented sequence is Manage Account → Settings → Pay Per Crawl → Domain Access → Visible. Visibility exposes configuration controls; it does not itself change security rules or start charging. See account setup requirements.
Editorial starting point: inventory your original articles and current crawler activity first. A site with many bot requests but little distinctive content has no demonstrated paying demand. Keep an alternative route open through our TollBit profile and licensing models guide; their terms and workflows require separate evaluation.
Which crawler requests can generate revenue?
A request becomes a charge event through the payment-enabled retrieval process, not simply because a bot visited. Cloudflare describes payment intent in request headers, successful HTTP 200 delivery, and HTTP 402 responses that communicate payment requirements. Security blocks can override a charge setting. Read the mechanism.
| Traffic event | How to treat it in a revenue model |
|---|---|
| Total observed AI requests | Starting traffic count only. Contains requests that never become paid access. |
| Allowed or excluded pages | No paid retrieval in this model. Cloudflare lists robots.txt and sitemap.xml among always-free paths. |
| Payment-required response or error | Not a completed paid delivery. Do not count both a challenge and its later successful retry. |
| Successful charged retrieval | Use the recorded billable event and its applicable price. |
| Repeat paid retrieval | Can be charged again. A page URL is not the same unit as a retrieval event. |
Cloudflare’s FAQ distinguishes free paths, repeat access and non-billed error responses. For accounting, reconcile actual charge records rather than estimating revenue from user-agent counts.
Pay-per-crawl revenue calculator
Separate the size of the traffic pool from the fraction that pays. The defaults below are a fictional monthly scenario, not observed rates, a quote or an earnings forecast. “Eligible share” means requests on charged content from crawlers configured to be charged, after free routes and blocked traffic are excluded. “Paid success share” is the portion of that pool that completes a billable retrieval.
Deductions are your planning assumption, not Cloudflare’s published fee. Zero means none modeled, not a claim of fee-free service. This tool sends no inputs and changes no crawler settings.
Formula: billable retrievals = total requests × eligible share × paid success share. Gross revenue = billable retrievals × price. Planning remainder = gross × (1 − deductions) − costs. The remainder is not a promised payout or accounting profit; taxes and unmodeled costs are excluded.
One million requests does not necessarily mean $5,000
Imagine a specialist maintenance publisher with one million monthly AI requests. If 20% qualify for its charged content and 10% of those complete paid delivery, only 20,000 retrievals are billable. At the illustrative $0.005 price, gross is $100. Multiplying all one million requests by that price would overstate this scenario by $4,900.
At 0% paid success, gross is $0 and the $25 operating cost remains. At 50% paid success with the other inputs unchanged, gross is $500. Neither case predicts buyer behavior. Higher prices can change willingness to pay, so do not increase price while assuming paid volume stays fixed.
If your logs count a 402 challenge and a retry as separate requests, use a paid-success share calibrated to that same denominator. Once you have charge events, use their count directly for reconciliation. For mixed prices, sum actual event prices or use a documented weighted average; the calculator assumes one price.
How do pricing and payouts work?
For an admitted and configured domain, Cloudflare documents enabling Pay Per Crawl from AI Crawl Control’s Payments tab and setting a default price. The stated minimum is $0.001 USD per successful crawl. Optional dynamic pricing is available for different content. See pricing configuration. Choosing a price creates no purchase obligation for a crawler.
Cloudflare’s payout documentation requires a dedicated Stripe Connect account created through its dashboard; an existing Stripe account is not compatible with this feature. An account administrator completes business and bank onboarding. Recorded delivery charges are aggregated and reconciled, with monthly publisher payments for accounts in good standing. The docs say accrued balances are not currently visible in the dashboard and can be requested from the Cloudflare team.
Before relying on proceeds, confirm your accepted commercial terms: deductions, payout eligibility, tax handling, adjustments and support procedures. The pages reviewed here do not establish a universal publisher fee percentage. That is why the calculator lets you supply an assumption instead of presenting an invented fee as fact.
A practical first test for a specialist publisher
Begin with one clearly owned section and a measurable decision. The following is an original planning example, not a tested implementation or a recommendation to change live settings without review.
- Inventory 100 original technical articles. Record authorship, contributor agreements, embedded third-party material and which pages can be offered under the applicable terms.
- Measure a baseline. Keep total AI requests, unique URLs, crawler identities, successful deliveries and useful referral visits separate over a consistent period.
- Confirm access and terms. Resolve beta admission, domain permissions and payout setup before expecting revenue.
- Define a small paid scope. Record free pages and existing security blocks. Check that ordinary readers and intended discovery paths still work.
- Review outcomes on a fixed date. Compare actual charge events and collected revenue with operating cost and any change in useful referrals. Keep or revise the approach based on those outcomes.
Paid access and a negotiated dataset license solve different needs. An archive buyer may want a delivered snapshot, defined usage scope and ongoing updates. For that route, read where to sell a dataset and prepare a dataset brief. Individual AI task work, such as the Handshake AI Fellowship, is a separate earning model.
Start with the requests you can account for.
Use a recent month of crawler data to replace the calculator defaults. If you cannot identify billable retrievals yet, treat the result as a scenario and compare other licensing routes before counting revenue.
Model your crawler trafficSources and method
Cloudflare’s official overview, account setup, pricing, payout and FAQ pages were checked November 10, 2026 and are linked beside the relevant claims. This guide focuses on its documented implementation; it does not assert that every crawler or publisher participates. Examples and calculator assumptions are original editorial illustrations. No program application or outreach was made.
External links here are direct documentation links, not affiliate signup links. Data Mining Masters may receive compensation from separately disclosed referrals elsewhere on the site. See our disclosure and methodology.