← Market

Robots

Read and parse a website's robots.txt: user-agent groups with allow/disallow rules, declared sitemaps, and which AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot...) are blocked from the whole site. Fast (under 2 s).

POST https://www.aypiai.online/api/robots

0.002 USDC on Base · payTo 0x4b1965f9d407ce9f032e9e2465afe7ff8bfb7f3d

Status: verified · payable

Operator badge: embed this endpoint's live status badge — it reads "Verified Payable" only while the probe record backs it.

Settlement: direct to the seller's payTo — 0% HIVE402 take

Payment routes

Base · 0.002 USDC · payTo 0x4b1965f9d407ce9f032e9e2465afe7ff8bfb7f3d · via cdp · settles direct
{
  "url": "https://stripe.com"
}
{
  "aiBots": {
    "ClaudeBot": "allowed",
    "GPTBot": "blocked"
  },
  "exists": true,
  "groups": [],
  "sitemaps": [
    "https://example.com/sitemap.xml"
  ],
  "url": "https://example.com/"
}

Script Master Labs LLC · SDVOSB · Kinston, NC · Terms · Refunds · Privacy · Contact

SML network: ScriptMasterLabs — the main site · SML Dual Market — the human storefront · VA Ratings — VA disability ratings, free for veterans

Agent intel trust badge

Trust scorepending / 100 · breakdown
Probe uptime (30d)— over 0 probes · daily
Avg 402 latency— ms
Paid calls (30d, on-chain)0 calls · 0 buyers

How to call

POST https://www.aypiai.online/api/robots · input in body · spec from live-challenge

Input schema
{
  "type": "object",
  "properties": {
    "url": {
      "type": "string",
      "description": "Public http(s) URL"
    }
  },
  "required": [
    "url"
  ]
}
Output schema
{
  "type": "object",
  "properties": {
    "url": {},
    "exists": {},
    "sitemaps": {},
    "aiBots": {},
    "groups": {}
  }
}
curl -i -X POST 'https://www.aypiai.online/api/robots' \
  -H 'content-type: application/json' -d '{"url":"https://stripe.com"}'

Agents: POST https://hive402.scriptmasterlabs.com/mcp → route_task / proxy_call · SDK /sdk/hive402.mjs

Try it (your wallet signs; HIVE402 never holds funds)