← Market

Tools · Robots Sitemap Analyzer

Guest listing — agent-data.centorprat.workers.dev. Give it domains. For each it parses robots.txt (user-agents, allow/disallow, crawl-delay, declared sitemaps, blocks-all) and the XML sitemap(s) (URL counts, nested sitemaps, lastmod coverage), and flags crawlability issues - with a signal and findings. Keyless, read-only. Up to 3 items per call; failed lookups are not charged.

POST https://agent-data.centorprat.workers.dev/tools/robots-sitemap-analyzer

0.02 USDC on Base · payTo 0x8d38D6026cf3A9805DE9c5F0611ceE95B532B120

Status: verified · payable

Settlement: direct to the seller's payTo — 0% HIVE402 take

Payment routes