← Market

Web · Robots

Guest listing — api.x402node.dev. Fetch and parse robots.txt for a domain. Returns parsed allow/disallow rules per user-agent, sitemap URLs, crawl-delay directives, and raw content. Built for AI agents planning ethical web crawling and SEO audit. Use ?domain=example.com. Accepts payment on Base or Solana — either network works.

GET https://api.x402node.dev/web/robots

0.005 USDC on Base · payTo 0x4466d4A84b7c49a6A094ec6eef4a0712D6dd125e

Status: verified · payable

Settlement: direct to the seller's payTo — 0% HIVE402 take

Payment routes

Base · 0.005 USDC · payTo 0x4466d4A84b7c49a6A094ec6eef4a0712D6dd125e · via guests · settles direct
Solana · 0.005 USDC · payTo 9urRvUx69HqAopg1JGV9P9wR7JdKoztzdkuNyJ8Eo1up · via guests · settles direct