← Market

P · Robots Check

Can I fetch this URL under the site's robots.txt? Robots.txt check: Tells a crawler or AI agent whether a robots.txt lets it crawl a given URL, and which line decides it. $0.001 per call in USDC, charged only on a 2xx answer. Try it free: https://forge.fizzl.eu/try/robots-check. Signed receipt. Own this tool for $1.00 and earn from every call: https://forge.fizzl.eu/forge/packages/robots-check.

GET https://forge.fizzl.eu/p/robots-check

0.001 USDC on Base · payTo 0x1f6f7a67336F4bC31556475999417def2b311680

Status: verified · payable

Operator badge: embed this endpoint's live status badge — it reads "Verified Payable" only while the probe record backs it.

Settlement: direct to the seller's payTo — 0% HIVE402 take

Payment routes

Base · 0.001 USDC · payTo 0x1f6f7a67336F4bC31556475999417def2b311680 · via cdp · settles direct
{
  "robots_txt": "User-agent: *\nDisallow: /private/\nAllow: /private/press/\n\nUser-agent: GPTBot\nDisallow: /\nCrawl-delay: 10\n\nSitemap: https://example.com/sitemap.xml\n",
  "url": "https://example.com/private/press/2026.html",
  "user_agent": "ClaudeBot"
}

Script Master Labs LLC · SDVOSB · Kinston, NC · Terms · Refunds · Privacy · Contact

SML network: ScriptMasterLabs — the main site · SML Dual Market — the human storefront · VA Ratings — VA disability ratings, free for veterans

Agent intel trust badge

Trust score66 / 100 · breakdown
Probe uptime (30d)100.0% over 1 probes · daily
Avg 402 latency89 ms
Paid calls (30d, on-chain)8 calls · 1 buyers · shared price cohort of 56
Facilitator-reported (30d)2 calls · 1 payers (CDP Bazaar)

How to call

GET https://forge.fizzl.eu/p/robots-check · input in query · spec from live-challenge

Input schema
{
  "type": "object",
  "properties": {
    "robots_txt": {
      "type": "string",
      "maxLength": 64000,
      "description": "The robots.txt body as text (LF, CR or CRLF lines). Empty means no rules: everything allowed. At most 64,000 characters (RFC 9309 section 2.5 sets 500 KiB as the minimum a crawler parses; longer texts are refused here, not cut)."
    },
    "url": {
      "type": "string",
      "maxLength": 2083,
      "description": "The URL to check: an absolute URL (https://example.com/a/b?q=1) or a path starting with / (/a/b?q=1). Path and query are compared; the fragment is ignored."
    },
    "user_agent": {
      "type": "string",
      "maxLength": 100,
      "description": "The crawler's product token: ASCII letters, - and _ only (e.g. GPTBot, ClaudeBot, Googlebot-Image), matched case-insensitively; * checks only the global (*) group."
    }
  },
  "required": [
    "robots_txt",
    "url",
    "user_agent"
  ]
}
curl -i -X GET 'https://forge.fizzl.eu/p/robots-check?robots_txt=User-agent%3A+*%0ADisallow%3A+%2Fprivate%2F%0AAllow%3A+%2Fprivate%2Fpress%2F%0A%0AUser-agent%3A+GPTBot%0ADisallow%3A+%2F%0ACrawl-delay%3A+10%0A%0ASitemap%3A+https%3A%2F%2Fexample.com%2Fsitemap.xml%0A&url=https%3A%2F%2Fexample.com%2Fprivate%2Fpress%2F2026.html&user_agent=ClaudeBot'

Agents: POST https://hive402.scriptmasterlabs.com/mcp → route_task / proxy_call · SDK /sdk/hive402.mjs

Try it (your wallet signs; HIVE402 never holds funds)