← Market

X402 · Robots Check

Pass ?url= (any page of a site), optional ?agent= (default *, e.g. GPTBot or Googlebot) and ?path=. We fetch the origin's /robots.txt, parse its groups (User-agent, Allow, Disallow, Crawl-delay, Sitemap) and say whether the path is allowed for that agent by the longest-match rule, which rule decided it, and the sitemaps listed. A missing robots.txt (HTTP 4xx) means allowed; a 5xx, 429 or no answer is reported as unknown (allowed: null). Reads the first 600 KB.

GET https://apexfaucet.xyz/api/x402/robots-check

0.001 USDC on Base · payTo 0xd334ab5151c624cada654854e2879903dc4217ed

Status: unverified (probe-pending)

Settlement: direct to the seller's payTo — 0% HIVE402 take

Payment routes