← Market

Text · Chunk

Purpose: split long text into token-sized chunks for embeddings or RAG. Split text or markdown into RAG-ready chunks in one deterministic call (no LLM, no network). Respects markdown headings, then paragraphs, then sentences; never cuts inside a fenced code block unless that block alone exceeds maxChars. Each chunk has index, text, chars, approxTokens (script-aware estimate since v1.20.2: CJK / code / other text with separate ratios, same method as /api/text/tokens; per-chunk approx gives method, segments, error range and the old chars/4 value), exact startOffset/endOffset into your input (tex

POST https://newbotlabor--254ecdfcc2de11f19b541607ee4eb77e.web.val.run/api/text/chunk

0.002 USDC on Base · payTo 0x174897b2c5B133feB08A8FB90856B08F9fce8647

Status: verified · payable

Settlement: via OMNIGATE (https://omnigate.scriptmasterlabs.com) — seller opted in; receipt token per settlement; 0.005 USDC fee inside settlement, waived inside the daily budget

Payment routes

{
  "text": "# Guide\n\n## Install\n\nRun the installer. Then restart.\n\n```sh\nnpm i x402\n```",
  "maxChars": 1500,
  "overlap": 150
}

{
  "mode": "markdown",
  "maxChars": 1500,
  "overlap": 150,
  "inputChars": 75,
  "totalChunks": 2,
  "totalChars": 73,
  "approxTokens": 16,
  "approx": {
    "method": "script-segmented-v1",
    "basis": "o200k_base",
    "scope": "sum over chunks (overlap counted in each chunk it appears in)",
    "tokens": 16,
    "range": [
      12,
      20
    ],
    "legacyChars4": 19
  },
  "chunks": [
    {
      "index": 0,
      "text": "# Guide",
      "chars": 7,
      "approxTokens": 1,
      "approx": {
        "method": "script-segmented-v1",
        "basis": "o200k_base",
        "tokens": 1,
        "relativeError": 0.15,
        "range": [
          0,
          2
        ],
        "segments": {
          "cjk": {
            "chars": 0,
            "tokens": 0
          },
          "code": {
            "chars": 0,
            "tokens": 0
          },
          "other": {
            "chars": 7,
            "tokens": 1
          }
        },
        "legacyChars4": 2
      },
      "startOffset": 0,
      "endOffset": 7,
      "headingPath": [
        "Guide"
      ]
    },
    {
      "index": 1,
      "text": "## Install\n\nRun the installer. Then restart.\n\n```sh\nnpm i x402\n```",
      "chars": 66,
      "approxTokens": 15,
      "approx": {
        "method": "script-segmented-v1",
        "basis": "o200k_base",
        "tokens": 15,
        "relativeError": 0.15,
        "range": [
          12,
          18
        ],
        "segments": {
          "cjk": {
            "chars": 0,
            "tokens": 0
          },
          "code": {
            "chars": 20,
            "tokens": 5
          },
          "other": {
            "chars": 46,
            "tokens": 9
          }
        },
        "legacyChars4": 17
      },
      "startOffset": 9,
      "endOffset": 75,
      "headingPath": [
        "Guide",
        "Install"
      ]
    }
  ]
}