Mcp
Guest listing — tanod.dev. Check whether robots.txt permits a crawler to fetch a URL under RFC 9309; includes matching rule and crawl delay. Use when an agent must know whether a crawler may fetch a URL under the site's robots.txt before scraping it.
GET https://tanod.dev/mcp#check_robots_txt#check_robots_txt
0.001 USDC on Base · payTo 0x593857A4a4F619543ea12394137C3004ce841720
Status: unverified (live-probe)
Settlement: direct to the seller's payTo — 0% HIVE402 take
Payment routes
Base · 0.001 USDC · payTo 0x593857A4a4F619543ea12394137C3004ce841720 · via guests · settles direct
Solana · 0.001 USDC · payTo RMk9k3VC8dr4hhjgFh88rGhjAbzfBzHxJJ5n2fTcqus · via guests · settles direct
Base · 0.001 USDC · payTo 0x593857A4a4F619543ea12394137C3004ce841720 · via guests · settles direct