Tokenize a repo

When you finish, a repo that has never used Sorb tokens has an open pull request adding a starter token set and a wiring plan — with no source files changed. You need a project with GitHub connected; this is for the "we don't have tokens yet" case, not for updating an existing set (that's Token version to GitHub PR).

What it scans

POST /api/projects/:id/tokenize clones the connected repo and scans its CSS/SCSS and JSX/TSX for hardcoded colors, spacing, and radii — literal values like #0f65ef, 16px, or 8px used as a border-radius — counting how many times each value appears and where. It groups near-duplicate colors and sizes into a primitive scale, then adds semantic aliases for the ones used consistently for one purpose (a primary action color, body text, a danger color, and so on).

What the PR adds

The PR is additive only — it changes no source file. It adds:

  • tokens/primitive.json, tokens/semantic.json — DTCG token files. Primitives are the raw scanned values (color.blue.500: #0F65EF); semantics are {ref} aliases onto them (color.action.primary: {color.blue.500}).
  • sd.config.js — a Style Dictionary config that builds those files into CSS custom properties.
  • sorb/onboarding/WIRING.md — the plan for wiring your app up to the new tokens by hand: run npx style-dictionary build, import the generated variables.css once at your app root, then a per-file table of suggested var(--token, fallback) replacements for your top hardcoded values, each with its token name, CSS variable, value, usage count, and file:line locations.
  • sorb/onboarding/report.json — the same data as WIRING.md, structured, for a script or a follow-up tool to consume.

Sorb can never verify an edited repo still compiles, so it never edits your source — the suggested var() replacements in WIRING.md are exactly that: suggestions, to apply and test yourself. Optionally, npx sorb-seed adapt --mode codemod --write generates the same replacements as an applyable codemod for your JS/JSX call sites.

Deterministic vs. AI naming

Grouping values into a scale (color.blue.500) and building semantic aliases (color.action.primary) can run two ways:

  • Deterministic — always available, no API key required. Runs when no AI provider is configured, or when an org's AI entitlement is exhausted.
  • AI-assisted — when a provider is configured and the org is entitled, Sorb asks it to name groups more idiomatically. Every attempted call (even a rejected one) is metered against the org's AI usage for the period, decided before any call is made — an org over its cap gets a 402 and no call happens.

sorb/onboarding/report.json's namedBy field says which path produced a given run ("fallback" for deterministic, or the provider name).

Gates

  • approve_merge — a session caller needs this org role; a Bearer key's write scope is its own authorization.
  • GitHub must be connected — same requirement as Token version to GitHub PR. Without it, the route returns pr_not_configured.
  • One run at a time, per project — starting a second run while one is active returns 409 run_in_progress with the running run's id. A run that makes no progress for 15 minutes is reaped automatically so a dead process can't wedge the project.
  • Idempotent on an unchanged repo — re-running against the same commit converges on the same PR (existing: true) instead of opening v2, v3, ...; a new commit on the base branch bumps to a new version.

Request & polling

POST /api/projects/:id/tokenize
Authorization: <session, or Bearer <project-key> with write scope>

No body — the repo, branch, and credentials come from the project's connected GitHub source, never from the request. The scan and PR happen in-request (there's no background queue), so a real repo can take a while; poll while it runs:

GET /api/projects/:id/tokenize
{
  "run": {
    "id": "…",
    "status": "running",
    "stage": "scan",
    "attempt": 1,
    "prUrl": null,
    "prNumber": null,
    "stats": null
  }
}

On success, status becomes "succeeded" and stats fills in with the counts from report.json (colorTokens, spaceTokens, radiusTokens, semanticAliases, namedBy, and more) alongside the PR URL.

Errors

CodeStatusMeaning
forbidden403The session user lacks approve_merge, or the key is read-only.
pr_not_configured400No GitHub connection on this project yet.
run_in_progress409A tokenize run is already active for this project.
repo_too_large400The repo exceeds the clone size limit.
ai-not-on-plan / ai-cap-exceeded402The org's plan has no AI naming, or its per-period cap is used up. This gate only exists when an AI provider is configured at all; with none configured, every run names deterministically and this error can't happen.
rate_limited429More than 2 tokenize starts for this project in the current window.

Next

Works with Figma. Not affiliated with, or endorsed by, Figma. Figma is a trademark of Figma, Inc.