Tokenize a repo
When you finish, a repo that has never used Sorb tokens has an open pull request adding a starter token set and a wiring plan — with no source files changed. You need a project with GitHub connected; this is for the "we don't have tokens yet" case, not for updating an existing set (that's Token version to GitHub PR).
What it scans
POST /api/projects/:id/tokenize clones the connected repo and scans its
CSS/SCSS and JSX/TSX for hardcoded colors, spacing, and radii — literal
values like #0f65ef, 16px, or 8px used as a border-radius — counting how
many times each value appears and where. It groups near-duplicate colors and
sizes into a primitive scale, then adds semantic aliases for the ones used
consistently for one purpose (a primary action color, body text, a danger
color, and so on).
What the PR adds
The PR is additive only — it changes no source file. It adds:
tokens/primitive.json,tokens/semantic.json— DTCG token files. Primitives are the raw scanned values (color.blue.500: #0F65EF); semantics are{ref}aliases onto them (color.action.primary: {color.blue.500}).sd.config.js— a Style Dictionary config that builds those files into CSS custom properties.sorb/onboarding/WIRING.md— the plan for wiring your app up to the new tokens by hand: runnpx style-dictionary build, import the generatedvariables.cssonce at your app root, then a per-file table of suggestedvar(--token, fallback)replacements for your top hardcoded values, each with its token name, CSS variable, value, usage count, and file:line locations.sorb/onboarding/report.json— the same data asWIRING.md, structured, for a script or a follow-up tool to consume.
Sorb can never verify an edited repo still compiles, so it never edits your
source — the suggested var() replacements in WIRING.md are exactly that:
suggestions, to apply and test yourself. Optionally, npx sorb-seed adapt --mode codemod --write generates the same replacements as an applyable
codemod for your JS/JSX call sites.
Deterministic vs. AI naming
Grouping values into a scale (color.blue.500) and building semantic aliases
(color.action.primary) can run two ways:
- Deterministic — always available, no API key required. Runs when no AI provider is configured, or when an org's AI entitlement is exhausted.
- AI-assisted — when a provider is configured and the org is entitled,
Sorb asks it to name groups more idiomatically. Every attempted call (even
a rejected one) is metered against the org's AI usage for the period,
decided before any call is made — an org over its cap gets a
402and no call happens.
sorb/onboarding/report.json's namedBy field says which path produced a
given run ("fallback" for deterministic, or the provider name).
Gates
approve_merge— a session caller needs this org role; a Bearer key's write scope is its own authorization.- GitHub must be connected — same requirement as
Token version to GitHub PR. Without it, the route
returns
pr_not_configured. - One run at a time, per project — starting a second run while one is
active returns
409 run_in_progresswith the running run's id. A run that makes no progress for 15 minutes is reaped automatically so a dead process can't wedge the project. - Idempotent on an unchanged repo — re-running against the same commit
converges on the same PR (
existing: true) instead of openingv2,v3, ...; a new commit on the base branch bumps to a new version.
Request & polling
POST /api/projects/:id/tokenize
Authorization: <session, or Bearer <project-key> with write scope>
No body — the repo, branch, and credentials come from the project's connected GitHub source, never from the request. The scan and PR happen in-request (there's no background queue), so a real repo can take a while; poll while it runs:
GET /api/projects/:id/tokenize
{
"run": {
"id": "…",
"status": "running",
"stage": "scan",
"attempt": 1,
"prUrl": null,
"prNumber": null,
"stats": null
}
}
On success, status becomes "succeeded" and stats fills in with the
counts from report.json (colorTokens, spaceTokens, radiusTokens,
semanticAliases, namedBy, and more) alongside the PR URL.
Errors
| Code | Status | Meaning |
|---|---|---|
forbidden | 403 | The session user lacks approve_merge, or the key is read-only. |
pr_not_configured | 400 | No GitHub connection on this project yet. |
run_in_progress | 409 | A tokenize run is already active for this project. |
repo_too_large | 400 | The repo exceeds the clone size limit. |
ai-not-on-plan / ai-cap-exceeded | 402 | The org's plan has no AI naming, or its per-period cap is used up. This gate only exists when an AI provider is configured at all; with none configured, every run names deterministically and this error can't happen. |
rate_limited | 429 | More than 2 tokenize starts for this project in the current window. |
Next
- Token version to GitHub PR — for a repo that already has tokens and you're publishing a new version.
- Tokens & taxonomy — the DTCG format and the primitive/semantic split these files use.
- Cloud REST API — the full route reference.
Works with Figma. Not affiliated with, or endorsed by, Figma. Figma is a trademark of Figma, Inc.