Skip to Content
PlansStep 3a — Controlled Vocab Bootstrap (Design Spec)

Step 3a — Controlled Vocab Bootstrap (Design Spec)

Source: plans/step-3.1-vocab-bootstrap-design.md · synced 2026-06-12

# Step 3a — Controlled Vocab Bootstrap (Design Spec) Date: 2026-05-25 Phase: A.2. 9-step coverage: Step 3a (tag vocab derived from data, not pre-defined). Status: Draft — awaiting greenlight. Depends on: ✅ CV parser shipped, parser-hardened JSONs in calibre-app/outputs/parsed-cvs/. 41 JDs sitting in reference/JD/. Closes: D7 in STATE-LOG.md.

Context

The original normaliser plan (title-normaliser-design.md) assumed a pre-defined tag vocab — Ben + Calibre sit down and design {cargo-ops: [energy:oil, metals:base, ...]} by hand. That approach has two problems with the current data set:

  1. Coverage gaps. Hand-drafted vocab misses categorical terms that appear repeatedly in Calibre’s actual JDs/CVs but weren’t on the designer’s mental list.
  2. Premature normalisation. Designing vocab before observing the data fixes the wrong shape — too coarse in some sub-verticals, too fine in others.

Step 3a fixes both by letting the data shape the vocab. Run a lean LLM-driven pass across all 41 JDs + ~107 parsed CVs. Cluster the categorical terms Claude keeps using. Name them. Lock as controlled vocab.

This is the bootstrap that everything downstream depends on:

  • Step 3b — JD normaliser uses the locked vocab
  • Step 5 CV normaliser uses the locked vocab
  • Step 6 Layer 1 hard-fit does set intersection on the vocab
  • Step 7 Layer 2 deep assessment can reference vocab tags as evidence anchors

Without 3a, every downstream step is blocked.

Architectural Decisions

DecisionChoiceRationale
Bootstrap vs designBootstrap from dataCoverage + correctness on real distribution
LLM vs rulesLLM-driven term extraction; deterministic clusteringClaude is strong at categorical extraction; merging “LNG” / “Liquefied Natural Gas” is mechanical
Granularity3-level: sub_function → sub_vertical → tagSub-verticals are pre-locked (SOT §3b); tags emerge per (sub_function, sub_vertical) cell
Tag formathyphen-case slug, ≤ 24 chars, no nested colonsSimple to write rubrics against; URL-safe; greppable
Cardinality target5–15 tags per (sub_function, sub_vertical) cell< 5 = too coarse, can’t differentiate; > 15 = too fine, overfits
RatificationCalibre + the Calibre team spot-check final list before lockIndustry-expert validation — same pattern as adjacency resolver
Re-runVocab is regenerated automatically every 30 days or when data crosses size thresholdsSelf-extending. Approved entries persist; new terms surface for review

End-to-End Architecture

                                    ┌─────────────────────────────┐
                                    │  Pre-locked structural data │
                                    │  - 5 sub_functions          │
                                    │  - sub_verticals per SF     │
                                    │    (SOT §3b)                │
                                    └──────────────┬──────────────┘
                                                   │
   ┌─────────────────────┐   ┌──────────────────────▼──────────────────────┐
   │  41 JD PDFs         │──▶│  Stage 1: Term Extraction (per document)    │
   │  reference/JD/      │   │  Claude tool-use call per JD/CV:            │
   │                     │   │  → categorical terms, specialisations,       │
   │  ~107 parsed CVs    │──▶│    tools, certifications, geographies,       │
   │  outputs/parsed-cvs │   │    seniority indicators                      │
   └─────────────────────┘   └──────────────────────┬──────────────────────┘
                                                    │ raw_terms[]
                                                    â–¼
                              ┌─────────────────────────────────────────┐
                              │  Stage 2: Bucket by (SF, SV)            │
                              │  Use parser's sub_function hint +       │
                              │  Calibre's folder placement (filename)   │
                              │  → terms[] per (sub_function, sub_ver)  │
                              └──────────────────────┬──────────────────┘
                                                     â–¼
                              ┌─────────────────────────────────────────┐
                              │  Stage 3: Cluster + Name (per cell)     │
                              │  Claude tool-use call per cell:         │
                              │  → merge synonyms (LNG ≡ liquefied      │
                              │    natural gas), drop noise, propose    │
                              │    hyphen-case slug names,              │
                              │    target cardinality 5–15              │
                              └──────────────────────┬──────────────────┘
                                                     â–¼
                              ┌─────────────────────────────────────────┐
                              │  Stage 4: Vocab Candidate JSON          │
                              │  config/tag-vocab.candidate.json        │
                              │  Per cell: tag[], synonyms[],           │
                              │  source_docs[] for traceability         │
                              └──────────────────────┬──────────────────┘
                                                     â–¼
                              ┌─────────────────────────────────────────┐
                              │  Stage 5: Ratification                  │
                              │  Calibre + the Calibre team review CLI:               │
                              │  approve / rename / merge / reject      │
                              │  → config/tag-vocab.json (LOCKED)       │
                              └─────────────────────────────────────────┘

Stage 1 — Term Extraction (per document)

One Claude tool-use call per JD and per CV. The CV parser already extracts most of these fields — for CVs, reuse the parsed JSON rather than re-querying Claude.

From a CV (reuse existing parser output)

Pull these fields out of each outputs/parsed-cvs/<name>.json:

  • signals.commodity_specialisations.value[]
  • signals.vertical_signals.value[]
  • signals.tools_and_systems.value[]
  • signals.certifications.value[]
  • current_role.raw_title.value + employment_history[].raw_title.value
  • identity.location.value (geo signal)

No new Claude call. Pure JSON extraction.

From a JD (new Claude call)

JDs are not yet parsed. One Claude tool-use call per JD, schema:

interface JDTerms {
  source_file: string
  sub_function: SubFunction      // hint from folder placement
  sub_vertical: SubVertical      // hint from folder/filename
  raw_role_title: string
  categorical_terms: string[]    // domain-specific specialisations: "LNG", "Crude Oil", "Veg Oil"
  tools_and_systems: string[]
  certifications: string[]
  geo_signals: string[]          // "Singapore", "Asia-Pacific"
  seniority_indicators: string[] // "Senior", "Lead", "Head of"
  technical_knowledge: string[]  // "HSSE", "charter party", "AML", "KYC"
}

Prompt: extract every categorical term the JD uses to describe what the role needs. Be exhaustive — over-extract; clustering will dedupe.

41 calls. Cheap.

Stage 2 — Bucket by (sub_function, sub_vertical)

Each JD has a known (sub_function, sub_vertical) from its folder path:

  • reference/JD/Operations- cargo, commodities brokerage/Agri/ → (cargo-ops, agri)
  • reference/JD/Risk/ → (risk-analytics, ?) — Risk JDs are flat; sub_vertical comes from filename (“JD for Credit Risk” → credit)
  • reference/JD/Finance- RM.../ → (finance-fo, ?) — same; sub_vertical from filename

For CVs, the folder gives sub_function but not sub_vertical. CVs span multiple sub-verticals (e.g. an LNG operator’s CV mentions both LNG and LPG). Strategy:

  • Tag CVs to multiple sub-verticals if their signals span more than one
  • Cell membership is non-exclusive: a CV can contribute terms to cargo-ops.energy AND cargo-ops.agri if it touches both

Output: a dict keyed by (sub_function, sub_vertical), value = list of every term observed across all source docs in that cell, with provenance (which doc each term came from).

Stage 3 — Cluster + Name (per cell)

One Claude tool-use call per (sub_function, sub_vertical) cell. Total cells = ~5 × ~4 = ~20 cells. So ~20 calls.

Input: all observed terms for that cell + count of how often each appears.

interface CellTermBundle {
  sub_function: SubFunction
  sub_vertical: SubVertical
  terms: { term: string, count: number, source_docs: string[] }[]
}

Claude does:

  1. Merge synonyms. “LNG” ≡ “Liquefied Natural Gas” ≡ “LNG carriers” → one tag.
  2. Drop noise. Single-occurrence stop-words, generic terms (“operations”, “team-player”).
  3. Propose tag slugs. hyphen-case, ≤ 24 chars (e.g. lng, crude-oil, precious-metals, mas-compliance).
  4. Aim for 5–15 tags per cell. Hint in prompt.

Output:

interface CellVocabCandidate {
  sub_function: SubFunction
  sub_vertical: SubVertical
  tags: {
    slug: string             // "lng"
    display: string          // "LNG"
    synonyms: string[]       // ["liquefied natural gas", "LNG carrier", "LNG ops"]
    source_docs: string[]    // doc paths where this cluster appeared
    confidence: number       // 0–1, Claude's confidence the cluster is coherent
  }[]
  total_unique_terms_observed: number
  rationale: string          // 1–2 sentence summary of choices made
}

Stage 4 — Vocab Candidate JSON

Write all cell outputs into calibre-app/config/tag-vocab.candidate.json:

{
  "version": "candidate-2026-05-25",
  "generated_at": "2026-05-25T...",
  "cells": [
    {
      "sub_function": "cargo-ops",
      "sub_vertical": "energy",
      "tags": [
        {
          "slug": "lng",
          "display": "LNG",
          "synonyms": ["liquefied natural gas", "lng carrier", "lng ops"],
          "source_docs": ["JD/.../Cargo Operator (LNG).pdf", "parsed-cvs/.../adrian-tan.json"],
          "confidence": 0.95
        }
      ]
    }
  ]
}

This is the candidate — not locked. Calibre + the Calibre team look at this next.

Stage 5 — Ratification

A small CLI script: scripts/review-vocab-candidate.ts. For each cell, present the candidate tags and offer the Calibre team four actions per tag:

ActionEffect
ApproveTag locked as-is
RenameTag moves to a new slug (with reason)
MergeTag merged into another tag (with reason)
RejectTag dropped (with reason)

CLI also lets recruiter:

  • Add a missing tag manually (slug + display + rationale)
  • Add synonyms to any tag

When all cells are reviewed, write:

calibre-app/config/tag-vocab.json  (LOCKED — what normalisers read)
calibre-app/config/tag-vocab.audit.json  (full history: who approved/rejected/renamed each tag, when, why)

The audit file is the grant-defensible artifact — recruiter-validated, traceable.

File Map

FileResponsibility
src/lib/vocab-bootstrap/types.tsJDTerms, CellTermBundle, CellVocabCandidate, LockedVocab
src/lib/vocab-bootstrap/jd-term-extractor.tsClaude tool-use call: JD text → JDTerms
src/lib/vocab-bootstrap/cv-term-extractor.tsPure JSON extraction from parsed-cvs JSON → terms
src/lib/vocab-bootstrap/bucketer.tsGroup terms by (sub_function, sub_vertical) cell
src/lib/vocab-bootstrap/clusterer.tsClaude tool-use call per cell: terms → CellVocabCandidate
src/lib/vocab-bootstrap/index.tsPublic API: bootstrapVocab(): Promise<CandidateVocab>
scripts/run-vocab-bootstrap.tsRuns stages 1–4, writes tag-vocab.candidate.json
scripts/review-vocab-candidate.tsCLI for the Calibre team to ratify → tag-vocab.json + audit
config/tag-vocab.candidate.jsonOutput of bootstrap pass
config/tag-vocab.jsonLOCKED — read by all normalisers
config/tag-vocab.audit.jsonRatification history per tag
tests/lib/vocab-bootstrap/clusterer.test.tsSynonym-merge regression set
tests/lib/vocab-bootstrap/bucketer.test.ts(sub_function, sub_vertical) assignment correctness

Validation Strategy

CheckHow
CoverageEvery JD must contribute ≥ 1 categorical term to its (SF, SV) cell
CardinalityFinal locked vocab has 5–15 tags per cell (warn on outliers)
Synonym coverageSample 10 raw terms from CVs not in any source list → 80 %+ should map to a locked tag
Ratification auditEvery locked tag has an approved_by + approved_at + reason if non-default action taken

What This Plan Does NOT Cover

  • Sub-vertical inference for CVs — CVs are tagged manually to sub-verticals via filename + folder. A CV-level sub-vertical classifier (analogous to a sub_function classifier) is the JOB of the CV normaliser (Step 5b), not this plan.
  • Tag-to-rubric mapping — that’s Step 6 (Layer 1 rubric configs).
  • Live-inference for new tags at runtime — Step 5b industry-expert filter can propose new tags; that’s a separate flow. This bootstrap is the one-shot lock.
  • Vocabulary versioning beyond version-1 — when more data arrives later, re-run bootstrap and produce candidate-2026-XX-XX.json. Recruiter compares vs locked and approves diffs only. Plan that flow when it happens, not now.

Open Questions

QuestionOwnerResolution path
What if a sub-vertical only has 1 JD (e.g. cargo-ops.metals.precious-metals — 1 JD)?EngineeringCardinality floor is per-cell. Sub-verticals with ≤ 2 source docs get marked coverage_low and recruiter manually seeds tags.
Should CV-side terms get a different weight than JD-side terms?EngineeringDefault: weight equal. Calibre can override during ratification.
Tag namespacing — flat slug or scoped (energy:lng)?LockedFlat slug. Sub-vertical is already structural, slug doesn’t need to repeat it. Layer 1 rubrics read the tuple (sub_function, sub_vertical, tag).
Multi-language synonyms (e.g. Mandarin role titles in Finance CVs)?CalibreCapture as synonyms during ratification. Lock English slug.

Next Step

Approve → invoke writing-plans to produce the implementation plan (sequential tasks per stage). Estimate: 1.5–2 days build time. ~62 Claude calls total (~41 JDs + ~20 cell-clusters + ~1 sanity-check pass). Cost ~5–10 USD.

Last updated on
Outreach Pipeline — Design Spec (Step 9)Step 3.1 — Tag-Vocab Bootstrap Implementation Plan