Step 6.1 — Rubric Bootstrap Extension Implementation Plan

Source: plans/step-6-phase-1-rubric-bootstrap-impl.md · synced 2026-06-12

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Step coverage: Step 3.1 (Stage 1 prompt extension) · Step 5.3 (hard-gate + boost/penalty rule scaffolds) · Step 6.1 (22 per-sub-vertical rubric configs). Design: amendment-3.1+5.3+6.1-bootstrap-rubric-extension.md Extends: step-3.1-vocab-bootstrap-impl.md — same harness, same branch, same test convention. Repo: calibre-app/ (branch feat/vocab-bootstrap already cut; this plan adds commits on top). Outputs (Stage 4 candidates): calibre-app/config/tag-vocab.candidate.json (already shipped) + reference/expert-rules/hard-gates.candidate.json + reference/expert-rules/boost-penalty.candidate.json + reference/rubric-configs/{sub-function}.{sub-vertical}.candidate.json × 22. Outputs (Stage 5 ratified): strip .candidate → canonical paths. Audit log per artefact.

Goal: Extend the existing Step 3.1 harness so a single live mine emits three artefact families instead of one. Ratify all three in one Dan + Jon session. This unblocks Step 3.2, Step 5.2, Step 5.3, and Step 6.1 in parallel — see amendment §7.

Architecture (recap — full version in amendment §3): Stage 1 prompt extended to also pull stated_must_haves, stated_nice_to_haves, stated_disqualifiers, stated_seniority_floor, stated_years_floor, and recurring_hard_rules per JD. Stage 2 unchanged. Stages 3b and 3c are new sibling Claude tool-use calls per cell. Stage 4 is extended to emit two new artefact families. Stage 5 is the new three-worksheet ratification CLI.

Tech Stack: Unchanged from Step 3.1 — TypeScript 5 (strict) · Next.js 16 / Node 20 · @anthropic-ai/sdk 0.52 · Vitest 3.1 · tsx 4.19. No new dependencies.

Cost & scale: Mine fires from same orchestrator; extended prompt adds ~+0.5 USD per JD vs Step 3.1. ~41 JD calls + ~20 Stage 3a calls + ~20 Stage 3b calls + ~22 Stage 3c calls ≈ ~103 Claude calls total. Estimate USD 20–40. Wallclock ~30–45 min mine + ~3–4 h build + ~2 h Dan + Jon ratification.

Code location anchor: the vocab-bootstrap modules live at calibre-app/src/engine/step-3-normalise/ (not src/lib/vocab-bootstrap/ — that path was renamed in the Step 3.1 build). Tests live at calibre-app/tests/engine/step-3-normalise/. All paths in this plan use the current locations.

File Structure (delta vs Step 3.1)

calibre-app/
├── src/engine/step-3-normalise/
│   ├── types.ts                          # EXTEND: add JD stated-* fields · HardRuleCandidate · SignalCandidate · ExpertRule · RubricConfig · Signal · TitleNorm
│   ├── jd-term-extractor.ts              # EXTEND Stage 1: add 6 new fields to tool schema + prompt
│   ├── rule-clusterer.ts                 # NEW Stage 3b: cluster recurring binary patterns → ExpertRule[]
│   ├── signal-clusterer.ts               # NEW Stage 3c: cluster signal candidates per cell → Signal[]
│   ├── rubric-emitter.ts                 # NEW Stage 4c: signals + frequency → RubricConfig (one cell)
│   ├── ratified-applier.ts               # NEW: apply ratification actions to expert-rules + rubric-configs (mirrors locked-vocab.ts)
│   └── index.ts                          # EXTEND: re-export new modules
├── scripts/
│   ├── run-vocab-bootstrap.ts            # EXTEND: after Stage 3a/4a, run Stage 3b/4b and Stage 3c/4c; write 3 new artefact families
│   ├── review-vocab-candidate.ts         # UNCHANGED — still drives worksheet 1 (vocab) only
│   └── review-rubric-bootstrap.ts        # NEW: drives worksheets 2 (expert rules) + 3 (22 rubrics) sequentially, then promotes + archives
├── config/
│   └── tag-vocab.candidate.json          # UNCHANGED location (Step 3.1 output)
└── tests/engine/step-3-normalise/
    ├── jd-term-extractor.test.ts         # EXTEND: assert new 6 fields in tool response
    ├── rule-clusterer.test.ts            # NEW
    ├── signal-clusterer.test.ts          # NEW
    ├── rubric-emitter.test.ts            # NEW
    ├── ratified-applier.test.ts          # NEW
    └── orchestration.test.ts             # EXTEND: assert 3-artefact wiring

reference/
├── expert-rules/                         # NEW dir
│   ├── hard-gates.candidate.json         # Stage 4b output
│   ├── hard-gates.json                   # Stage 5 ratified output
│   ├── hard-gates.audit.json             # Stage 5 audit log
│   ├── boost-penalty.candidate.json      # Stage 4b output
│   ├── boost-penalty.json                # Stage 5 ratified output
│   └── boost-penalty.audit.json          # Stage 5 audit log
└── rubric-configs/
    ├── README.md                         # UNCHANGED (already updated to point at amendment plan)
    ├── {sub-function}.{sub-vertical}.candidate.json   # Stage 4c × 22
    ├── {sub-function}.{sub-vertical}.json             # Stage 5 ratified × 22
    └── rubric-configs.audit.json                       # Stage 5 audit log (one shared log for all 22)

The 4 grant-vertical rubric stubs (commodities.json, maritime.json, offshore.json, tech-gaming.json) were already archived 2026-06-11 to _archive/2026-06-11-rubric-stubs-4-vertical/. No backwards-compatibility shim is built — see amendment §2 row “Output file paths”.

Conventions (read once, follow throughout)

• Same conventions as Step 3.1 impl plan §“Conventions” — strict TS, no any, Vitest, @/ alias for src/, Claude tool-use shape mirrors cv-parser pattern, frequent commits, all work on feat/vocab-bootstrap branch.

• Sub-function + sub-vertical IDs — use the canonical 22-cell enumeration from SOURCE-OF-TRUTH.md §3b:

  sub_function	sub_verticals	cells
  cargo-ops	agri, energy, metals	3
  ship-ops	tanker, drybulk, container, offshore-support, broker	5
  risk-analytics	credit, market, operational, product-controller, trading-analytics	5
  finance-fo	rm, wealth-mgmt, institutional-sales, portfolio-mgmt, trading	5
  ops-back-office	aml, compliance, assistant-rm, finance-ops	4
  	Total	22

• Task numbering continues from Step 3.1 — Step 3.1 impl plan ended at Task 12. This plan starts at Task 13 and runs through Task 24.

• reference/expert-rules/ and reference/rubric-configs/ live at the workspace root (outside calibre-app/). The orchestrator + ratification CLI use path.resolve(__dirname, '../../reference/...') — mirrors how run-vocab-bootstrap.ts already resolves reference/JD/.

Task 13 — Extend types module for the 3-artefact mine

Files:

• Modify: calibre-app/src/engine/step-3-normalise/types.ts

• Test: none (pure type module)

• Step 1: Add the new types

Append to types.ts:

// ------------------------------------------------------------------
// Step 6.1 / Step 5.3 — extended Stage 1 JD fields
// ------------------------------------------------------------------
 
/**
 * Hard-rule candidate observed in a JD — used by Stage 3b to cluster into ExpertRule[].
 */
export interface HardRuleCandidate {
  pattern_kind:
    | 'geo_must_be_in'
    | 'geo_must_not_be_in'
    | 'employer_must_be_in'
    | 'employer_must_not_be_in'
    | 'cert_required'
    | 'language_required'
    | 'tenure_floor_years'
    | 'custom'
  values: string[]                       // e.g. ['Singapore'] · ['Vitol', 'Trafigura'] · ['HSSE']
  raw_quote: string                      // verbatim JD text that triggered the inference
}
 
/**
 * Stage 1 JD-side fields added on top of Step 3.1's DocTerms.
 */
export interface JDStatedFields {
  stated_must_haves: string[]            // explicit "Required:" / "Must have:" bullets
  stated_nice_to_haves: string[]         // explicit "Preferred:" / "Bonus:" bullets
  stated_disqualifiers: string[]         // explicit "Not eligible:" / "Not suitable:" bullets
  stated_seniority_floor: string | null  // e.g. "Senior", "5+ years"
  stated_years_floor: number | null
  recurring_hard_rules: HardRuleCandidate[]
}
 
/**
 * Extended DocTerms — same as Step 3.1 DocTerms plus JDStatedFields (JD only; null for CVs).
 */
export interface DocTermsExtended extends DocTerms {
  jd_stated?: JDStatedFields             // present only when source_kind === 'jd'
}
 
// ------------------------------------------------------------------
// Step 5.3 — Expert rules (mirrors plans/step-5.3-industry-filter-design.md §Rule Schema)
// ------------------------------------------------------------------
 
export type RuleCategory =
  | 'geography' | 'competitor' | 'pedigree' | 'career_shape'
  | 'certification' | 'language' | 'compensation' | 'custom'
 
export type RuleScope =
  | { applies_to: 'all' }
  | { applies_to: 'sub_function'; sub_function: SubFunction }
  | { applies_to: 'sub_vertical'; sub_function: SubFunction; sub_vertical: SubVertical }
  | { applies_to: 'tag'; tag: string }
  | { applies_to: 'jd_match'; jd_pattern: string }
 
export type TriggerPredicate =
  | { type: 'cv_location_not_in'; values: string[] }
  | { type: 'cv_employer_in'; values: string[] }
  | { type: 'cv_employer_matches'; pattern: string }
  | { type: 'cv_consecutive_short_tenures'; threshold_months: number; count: number }
  | { type: 'cv_missing_certification'; cert: string }
  | { type: 'cv_missing_language'; language: string }
  | { type: 'cv_tag_present'; tag: string }
  | { type: 'cv_employer_attended_school'; school_pattern: string }
  | { type: 'combo'; op: 'and' | 'or'; predicates: TriggerPredicate[] }
 
export type RuleEffect =
  | { kind: 'drop'; reason_code: string }
  | { kind: 'block_at_l2'; reason_code: string }
  | { kind: 'boost'; delta: number; reason_code: string }
  | { kind: 'penalty'; delta: number; reason_code: string }
  | { kind: 'require_evidence'; evidence_field: string }
 
export interface ExpertRule {
  id: string                             // ULID — generated at emit time
  category: RuleCategory
  scope: RuleScope
  trigger: TriggerPredicate
  effect: RuleEffect
  rationale: string
  ratified_by?: string                   // populated at Stage 5
  ratified_at?: string
  source: 'scaffold' | 'live_inference' | 'manual'
  active: boolean                        // default true
  source_docs: string[]                  // JDs the rule was inferred from (audit trail)
  observed_in_pct: number                // 0–1, share of JDs in scope that surfaced this pattern
}
 
export interface ExpertRulesFile {
  version: string                        // 'bootstrap-YYYY-MM-DD' or 'ratified-YYYY-MM-DD'
  tenant_id: string                      // 'calibre'
  generated_from: { run_id: string; cv_count: number; jd_count: number }
  rules: ExpertRule[]
}
 
// ------------------------------------------------------------------
// Step 6.1 — per-sub-vertical rubric configs
// ------------------------------------------------------------------
 
export interface Signal {
  type: 'tag' | 'keyword' | 'certification' | 'employer_class' | 'geography' | 'language'
  values: string[]
  weight: number                         // 1–5 (relative within signal kind)
  source_docs: string[]
  observed_in_pct: number                // 0–1
}
 
export interface TitleNorm {
  from_pattern: string                   // regex
  to_canonical: string
  seniority_signal?: 'Junior' | 'Mid' | 'Senior' | 'Lead' | 'Head'
}
 
export interface RubricScoringWeights {
  sub_function_match: number             // default 0.20
  tag_overlap: number                    // default 0.30
  seniority_delta: number                // default 0.20
  years_experience: number               // default 0.15
  red_flag_penalty: number               // default 0.15
}
 
export interface RubricConfig {
  sub_function: SubFunction
  sub_vertical: SubVertical
  display_name: string
  version: string                        // 'bootstrap-YYYY-MM-DD' or 'ratified-YYYY-MM-DD'
  generated_from: { run_id: string; cv_count_in_cell: number; jd_count_in_cell: number }
  must_have_signals: Signal[]
  nice_to_have_signals: Signal[]
  disqualifying_signals: Signal[]
  title_normalisations: TitleNorm[]
  scoring_weights: RubricScoringWeights  // must sum to 1.0
  minimum_pass_score: number             // default 4.0
  status: 'bootstrap-draft' | 'ratified' | 'deprecated'
  ratified_by?: string
  ratified_at?: string
  notes?: string                         // free text — thin-data warnings, merges, deferrals
}
 
// ------------------------------------------------------------------
// Stage 5 — ratification entries for the 2 new artefact families
// ------------------------------------------------------------------
 
export interface RuleAuditEntry {
  timestamp: string
  rule_id: string                        // ULID from candidate
  action: 'approve' | 'tune_delta' | 'tune_trigger' | 'reject' | 'add'
  resolved_rule?: ExpertRule             // present for approve / tune / add
  reason: string
  reviewer: string
}
 
export interface RubricAuditEntry {
  timestamp: string
  sub_function: SubFunction
  sub_vertical: SubVertical
  target: 'signal' | 'weight' | 'cell'
  action:
    | 'approve_signal' | 'move_to_must' | 'move_to_nice' | 'move_to_disqualifying'
    | 'drop_signal' | 'add_signal'
    | 'adjust_weight' | 'approve_thin' | 'merge_cells' | 'defer_cell'
  payload: unknown                        // shape varies by action — typed at point of use
  reason: string
  reviewer: string
}
 
export interface RuleAuditLog { version: string; entries: RuleAuditEntry[] }
export interface RubricAuditLog { version: string; entries: RubricAuditEntry[] }

• Step 2: Verify it compiles

Run: npx tsc --noEmit Expected: PASS.

• Step 3: Commit

git add src/engine/step-3-normalise/types.ts
git commit -m "feat(rubric-bootstrap): extend types for 3-artefact mine (Step 6.1 Task 13)"

Task 14 — Extend Stage 1: JD term extractor pulls 6 new fields

Extend jdToDocTerms to populate JDStatedFields alongside the existing categorical/tools/cert/geo/seniority/technical lists. One Claude call per JD — the new fields ride the same tool-use response. Acceptance: extended tool schema validates against a canned response; absent fields default to [] / null.

Files:

• Modify: calibre-app/src/engine/step-3-normalise/jd-term-extractor.ts

• Modify: calibre-app/tests/engine/step-3-normalise/jd-term-extractor.test.ts

• Step 1: Extend the failing test first

Open tests/engine/step-3-normalise/jd-term-extractor.test.ts. Add a new test case below the existing 'returns DocTerms with source_kind=jd ...' test:

it('extracts the 6 extended JD-stated fields when present', async () => {
  const fakeClient = {
    messages: {
      create: vi.fn().mockResolvedValue({
        content: [{
          type: 'tool_use', name: 'extract_jd_terms',
          input: {
            // Existing Step 3.1 fields
            categorical_terms: ['LNG'], tools_and_systems: [], certifications: ['HSSE'],
            geo_signals: ['Singapore'], seniority_indicators: ['Senior'], technical_knowledge: [],
            // New Step 6.1 fields
            stated_must_haves: ['5+ years LNG ops', 'HSSE cert'],
            stated_nice_to_haves: ['Mandarin'],
            stated_disqualifiers: ['No tanker-only background'],
            stated_seniority_floor: 'Senior',
            stated_years_floor: 5,
            recurring_hard_rules: [
              { pattern_kind: 'geo_must_be_in', values: ['Singapore'], raw_quote: 'Singapore-based only' },
              { pattern_kind: 'cert_required', values: ['HSSE'], raw_quote: 'HSSE certification mandatory' },
            ],
          },
        }],
      }),
    },
  } as unknown as import('@anthropic-ai/sdk').default
 
  const result = await jdToDocTerms(
    fakeClient, 'sample JD text', 'reference/JD/.../JD for LNG Operator.pdf',
    'cargo-ops', 'energy'
  )
 
  expect(result.jd_stated).toBeDefined()
  expect(result.jd_stated!.stated_must_haves).toEqual(['5+ years LNG ops', 'HSSE cert'])
  expect(result.jd_stated!.stated_years_floor).toBe(5)
  expect(result.jd_stated!.recurring_hard_rules).toHaveLength(2)
  expect(result.jd_stated!.recurring_hard_rules[0].pattern_kind).toBe('geo_must_be_in')
})
 
it('defaults extended fields to empty / null when Claude returns nothing for them', async () => {
  const fakeClient = {
    messages: {
      create: vi.fn().mockResolvedValue({
        content: [{
          type: 'tool_use', name: 'extract_jd_terms',
          input: {
            categorical_terms: [], tools_and_systems: [], certifications: [],
            geo_signals: [], seniority_indicators: [], technical_knowledge: [],
            stated_must_haves: [], stated_nice_to_haves: [], stated_disqualifiers: [],
            stated_seniority_floor: null, stated_years_floor: null, recurring_hard_rules: [],
          },
        }],
      }),
    },
  } as unknown as import('@anthropic-ai/sdk').default
 
  const result = await jdToDocTerms(fakeClient, 'x', 'p', 'cargo-ops', 'energy')
  expect(result.jd_stated!.stated_must_haves).toEqual([])
  expect(result.jd_stated!.stated_seniority_floor).toBeNull()
  expect(result.jd_stated!.recurring_hard_rules).toEqual([])
})

Run: npx vitest run tests/engine/step-3-normalise/jd-term-extractor.test.ts Expected: FAIL (jd_stated is undefined; tool schema rejects new fields).

• Step 2: Extend the tool schema + prompt + return shape

Edit src/engine/step-3-normalise/jd-term-extractor.ts:

1. Inside JD_TERM_TOOL.input_schema.properties, add 6 new entries:

   • stated_must_haves: { type: 'array', items: { type: 'string' }, description: 'Explicit "Required:" / "Must have:" bullets. Verbatim.' }
   • stated_nice_to_haves: { type: 'array', items: { type: 'string' }, description: 'Explicit "Preferred:" / "Bonus:" / "Nice to have:" bullets.' }
   • stated_disqualifiers: { type: 'array', items: { type: 'string' }, description: 'Explicit "Not eligible:" / "Not suitable for:" bullets.' }
   • stated_seniority_floor: { type: ['string', 'null'], description: 'Required seniority level: "Junior" / "Mid" / "Senior" / "Lead" / "Head" / "Director" or null.' }
   • stated_years_floor: { type: ['number', 'null'], description: 'Minimum years of experience required, or null.' }
   • recurring_hard_rules: { type: 'array', items: { type: 'object', required: ['pattern_kind', 'values', 'raw_quote'], properties: { pattern_kind: { type: 'string', enum: ['geo_must_be_in', 'geo_must_not_be_in', 'employer_must_be_in', 'employer_must_not_be_in', 'cert_required', 'language_required', 'tenure_floor_years', 'custom'] }, values: { type: 'array', items: { type: 'string' } }, raw_quote: { type: 'string' } } } }

2. Add all 6 keys to input_schema.required.

3. Update SYSTEM_PROMPT — append: Also extract stated must-haves, nice-to-haves, disqualifiers, seniority floor, years floor, and recurring hard rules. Hard rules are binary patterns the JD enforces — geographic restrictions, employer blacklists/whitelists, certification requirements, language requirements, tenure floors. Quote the JD verbatim in raw_quote.

4. Update the input cast inside jdToDocTerms to include the 6 new fields with their TypeScript types.

5. Update the return statement to include jd_stated: { stated_must_haves: input.stated_must_haves, stated_nice_to_haves: input.stated_nice_to_haves, stated_disqualifiers: input.stated_disqualifiers, stated_seniority_floor: input.stated_seniority_floor, stated_years_floor: input.stated_years_floor, recurring_hard_rules: input.recurring_hard_rules }.

6. Change the return type from DocTerms to DocTermsExtended and update the import.

• Step 3: Run, watch pass

Run: npx vitest run tests/engine/step-3-normalise/jd-term-extractor.test.ts Expected: PASS, all original tests + 2 new tests green.

• Step 4: Commit

git add src/engine/step-3-normalise/jd-term-extractor.ts tests/engine/step-3-normalise/jd-term-extractor.test.ts
git commit -m "feat(rubric-bootstrap): extend Stage 1 JD prompt with 6 stated fields (Step 6.1 Task 14)"

Task 15 — Stage 3b — Rule clusterer (recurring binary patterns → ExpertRule[])

Cluster recurring_hard_rules[] + stated_disqualifiers[] from every JD in the cell into ExpertRule[]. One Claude tool-use call per cell. Split by effect.kind is mechanical post-Claude — clusterer just returns the full rule set.

Files:

• Create: calibre-app/src/engine/step-3-normalise/rule-clusterer.ts

• Create: calibre-app/tests/engine/step-3-normalise/rule-clusterer.test.ts

• Step 1: Write the failing test

Create tests/engine/step-3-normalise/rule-clusterer.test.ts:

import { describe, it, expect, vi } from 'vitest'
import { clusterRules } from '@/engine/step-3-normalise/rule-clusterer'
import type { DocTermsExtended } from '@/engine/step-3-normalise/types'
 
describe('clusterRules', () => {
  it('returns ExpertRule[] for one cell, split-able by effect.kind', async () => {
    const jdDocs: DocTermsExtended[] = [
      {
        source_file: 'jd-1.pdf', source_kind: 'jd', sub_function: 'cargo-ops',
        sub_verticals: ['energy'], categorical_terms: [], tools_and_systems: [],
        certifications: [], geo_signals: [], seniority_indicators: [], technical_knowledge: [],
        jd_stated: {
          stated_must_haves: ['HSSE cert'], stated_nice_to_haves: [],
          stated_disqualifiers: ['No tanker-only candidates'],
          stated_seniority_floor: null, stated_years_floor: null,
          recurring_hard_rules: [
            { pattern_kind: 'geo_must_be_in', values: ['Singapore'], raw_quote: 'SG only' },
            { pattern_kind: 'cert_required', values: ['HSSE'], raw_quote: 'HSSE mandatory' },
          ],
        },
      },
    ]
 
    const fakeClient = {
      messages: {
        create: vi.fn().mockResolvedValue({
          content: [{
            type: 'tool_use', name: 'cluster_expert_rules',
            input: {
              rules: [
                {
                  category: 'geography', scope: { applies_to: 'sub_vertical', sub_function: 'cargo-ops', sub_vertical: 'energy' },
                  trigger: { type: 'cv_location_not_in', values: ['Singapore'] },
                  effect: { kind: 'penalty', delta: -1.5, reason_code: 'non-sg' },
                  rationale: 'energy ops require SG residency',
                  source_docs: ['jd-1.pdf'], observed_in_pct: 1.0,
                },
                {
                  category: 'certification', scope: { applies_to: 'sub_vertical', sub_function: 'cargo-ops', sub_vertical: 'energy' },
                  trigger: { type: 'cv_missing_certification', cert: 'HSSE' },
                  effect: { kind: 'block_at_l2', reason_code: 'no-hsse' },
                  rationale: 'LNG roles need HSSE',
                  source_docs: ['jd-1.pdf'], observed_in_pct: 1.0,
                },
              ],
            },
          }],
        }),
      },
    } as unknown as import('@anthropic-ai/sdk').default
 
    const rules = await clusterRules(fakeClient, 'cargo-ops', 'energy', jdDocs)
 
    expect(rules).toHaveLength(2)
    expect(rules.every(r => typeof r.id === 'string' && r.id.length > 0)).toBe(true)  // ULIDs assigned
    expect(rules.every(r => r.source === 'live_inference')).toBe(true)
    expect(rules.every(r => r.active === true)).toBe(true)
    expect(rules.filter(r => r.effect.kind === 'block_at_l2')).toHaveLength(1)
  })
 
  it('returns [] when the cell has no recurring_hard_rules and no stated_disqualifiers', async () => {
    const jdDocs: DocTermsExtended[] = [{
      source_file: 'jd-empty.pdf', source_kind: 'jd', sub_function: 'cargo-ops',
      sub_verticals: ['energy'], categorical_terms: [], tools_and_systems: [],
      certifications: [], geo_signals: [], seniority_indicators: [], technical_knowledge: [],
      jd_stated: {
        stated_must_haves: [], stated_nice_to_haves: [], stated_disqualifiers: [],
        stated_seniority_floor: null, stated_years_floor: null, recurring_hard_rules: [],
      },
    }]
    const fakeClient = { messages: { create: vi.fn() } } as unknown as import('@anthropic-ai/sdk').default
    const rules = await clusterRules(fakeClient, 'cargo-ops', 'energy', jdDocs)
    expect(rules).toEqual([])
    expect(fakeClient.messages.create).not.toHaveBeenCalled()  // skip Claude call when no input
  })
})

• Step 2: Run, watch fail

Run: npx vitest run tests/engine/step-3-normalise/rule-clusterer.test.ts Expected: FAIL — module missing.

• Step 3: Implement

Create src/engine/step-3-normalise/rule-clusterer.ts — a Claude tool-use module that mirrors clusterer.ts. Tool name: cluster_expert_rules. Input schema: array of rule objects matching the ExpertRule shape minus id / ratified_* / active / source. Post-Claude: assign ULID id, set source: 'live_inference', active: true. Short-circuit and return [] if no JD in the cell has recurring_hard_rules.length > 0 AND no stated_disqualifiers.length > 0.

Reference the existing clusterer.ts for the tool-use + schema pattern. Use crypto.randomUUID() for the ULID (acceptable substitute — no new dep).

System prompt seed: You are encoding recruiter heuristics as deterministic rules. Given the binary patterns this JD set enforces, cluster them into a minimal ExpertRule set. Aim for 5–10 rules per cell. Each rule must trace to ≥1 JD via source_docs. Include the 10 example rules from step-5.3-industry-filter-design.md §Example Rules verbatim in the user message as exemplars.

• Step 4: Run, watch pass

Run: npx vitest run tests/engine/step-3-normalise/rule-clusterer.test.ts Expected: PASS, 2/2.

• Step 5: Commit

git add src/engine/step-3-normalise/rule-clusterer.ts tests/engine/step-3-normalise/rule-clusterer.test.ts
git commit -m "feat(rubric-bootstrap): Stage 3b rule clusterer (Step 6.1 Task 15)"

Task 16 — Stage 3c — Signal clusterer (signal candidates per cell)

Cluster stated_must_haves[] + stated_nice_to_haves[] + stated_disqualifiers[] per cell into proposed Signal[]. One Claude tool-use call per cell. Output is three lists: must / nice / disqualifying. Frequency calculation happens deterministically in Task 17 — this task only produces the signal proposals.

Files:

• Create: calibre-app/src/engine/step-3-normalise/signal-clusterer.ts

• Create: calibre-app/tests/engine/step-3-normalise/signal-clusterer.test.ts

• Step 1: Write the failing test

Create tests/engine/step-3-normalise/signal-clusterer.test.ts. Cover:

1. Returns { must_have_signals, nice_to_have_signals, disqualifying_signals } for one cell from a fake Claude response containing 2 signals per tier.
2. Asserts Signal.type ∈ the locked enum (tag · keyword · certification · employer_class · geography · language).
3. Skips Claude call and returns three empty arrays when the cell’s JDs surface zero stated_* entries.

Pattern: mirror the test shape from clusterer.test.ts and the new rule-clusterer.test.ts.

• Step 2: Implement

Create src/engine/step-3-normalise/signal-clusterer.ts. Tool name: cluster_cell_signals. Tool schema: three array properties (must_have_signals, nice_to_have_signals, disqualifying_signals), each item carries { type, values, source_docs }. Weight and observed_in_pct are computed downstream (Task 17), NOT in this Claude call — leave them out of the schema. Short-circuit + return three []s when input is empty.

System prompt seed: You are a recruitment-rubric extractor. Given JD-stated must-haves, nice-to-haves, and disqualifiers for one (sub_function, sub_vertical) cell, propose Signal[] for each tier. Each Signal groups synonymous JD statements into a typed cluster (tag / keyword / certification / employer_class / geography / language). Drop noise. Aim for 4–10 must, 3–8 nice, 1–5 disqualifying per cell.

• Step 3: Run, watch pass

Run: npx vitest run tests/engine/step-3-normalise/signal-clusterer.test.ts Expected: PASS, all tests green.

• Step 4: Commit

git add src/engine/step-3-normalise/signal-clusterer.ts tests/engine/step-3-normalise/signal-clusterer.test.ts
git commit -m "feat(rubric-bootstrap): Stage 3c signal clusterer (Step 6.1 Task 16)"

Task 17 — Rubric emitter (Stage 4c — Signal[] + frequency → RubricConfig)

Pure function. Takes the Stage 3c output for one cell + the full JD list for that cell + the run metadata, and emits a RubricConfig. This is where observed_in_pct and weight get calculated deterministically. Per amendment §4c bootstrap-weight rule:

• Signals with observed_in_pct ≥ 0.7 → must_have_signals with weight = 5.
• Signals with 0.3 ≤ observed_in_pct < 0.7 → nice_to_have_signals with weight = round(2 + 3*(observed_in_pct - 0.3) / 0.4) (i.e. 2–3 proportional).
• Below 0.3 → discarded.

Disqualifying signals always land in disqualifying_signals regardless of frequency (Dan rescues at ratification).

Default scoring_weights and minimum_pass_score come from amendment §4c schema: { sub_function_match: 0.20, tag_overlap: 0.30, seniority_delta: 0.20, years_experience: 0.15, red_flag_penalty: 0.15 } and minimum_pass_score: 4.0. Defaults match pipeline-1.1+2.1+3.2+6.1+7.1+7.2+8.1-matching-engine-design.md §Layer 1.

title_normalisations are populated empty ([]) at bootstrap — Dan adds during ratification.

Files:

• Create: calibre-app/src/engine/step-3-normalise/rubric-emitter.ts

• Create: calibre-app/tests/engine/step-3-normalise/rubric-emitter.test.ts

• Step 1: Write the failing test

Create tests/engine/step-3-normalise/rubric-emitter.test.ts. Cover:

1. Frequency promotion — signal observed in 8/10 JDs → must_have_signals with weight === 5.
2. Frequency middle band — signal in 4/10 JDs → nice_to_have_signals with weight === 2 (round(2 + 3*0.1/0.4) = 3 — adjust expectation to match the formula precisely).
3. Frequency floor — signal in 2/10 JDs → dropped.
4. Disqualifying signals copied through regardless of frequency.
5. Default scoring_weights sum to exactly 1.0.
6. display_name derived from the (sub_function, sub_vertical) lookup table.
7. generated_from.jd_count_in_cell and cv_count_in_cell populated from input args.
8. status === 'bootstrap-draft' and version matches bootstrap-YYYY-MM-DD.

• Step 2: Implement

Create src/engine/step-3-normalise/rubric-emitter.ts. Pure function emitRubric(args: { run_id, sub_function, sub_vertical, jds, cvs_in_cell, signals: { must, nice, disqualifying } }): RubricConfig. Include a DISPLAY_NAME_MAP constant for all 22 cells (e.g. 'cargo-ops.energy' → 'Cargo Operations — Energy (LNG/LPG/Crude/Petrochem)'). Source the strings from amendment §4c examples + SOURCE-OF-TRUTH.md §3a/§3b.

• Step 3: Run, watch pass

Run: npx vitest run tests/engine/step-3-normalise/rubric-emitter.test.ts Expected: PASS, all 8 tests.

• Step 4: Commit

git add src/engine/step-3-normalise/rubric-emitter.ts tests/engine/step-3-normalise/rubric-emitter.test.ts
git commit -m "feat(rubric-bootstrap): Stage 4c rubric emitter with frequency-weighted promotion (Step 6.1 Task 17)"

Task 18 — Extend orchestrator: wire Stages 3b/3c/4b/4c into the mine

Modify scripts/run-vocab-bootstrap.ts. After Stage 3a/4a finishes (vocab candidate), run Stage 3b + Stage 3c per cell, then Stage 4b (split into hard-gates vs boost-penalty by effect.kind) and Stage 4c (one rubric per cell). Write all three new artefact families.

Files:

• Modify: calibre-app/scripts/run-vocab-bootstrap.ts

• Modify: calibre-app/src/engine/step-3-normalise/index.ts

• Step 1: Re-export new modules

Edit src/engine/step-3-normalise/index.ts — add:

export { clusterRules } from './rule-clusterer'
export { clusterSignals } from './signal-clusterer'
export { emitRubric } from './rubric-emitter'

• Step 2: Extend the orchestrator

In scripts/run-vocab-bootstrap.ts:

1. Add path constants:

   const EXPERT_RULES_DIR = path.resolve(__dirname, '../../reference/expert-rules')
   const RUBRICS_DIR = path.resolve(__dirname, '../../reference/rubric-configs')

   and fs.mkdirSync(EXPERT_RULES_DIR, { recursive: true }) / same for RUBRICS_DIR at the top of main().

2. Generate a run_id via crypto.randomUUID() at the top of main(). Thread it through Stages 3b/3c/4b/4c.

3. After the existing Stage 3 loop (which currently produces candidateCells: CellVocabCandidate[]), add a parallel loop per cell:

   const allRules: ExpertRule[] = []
   const rubrics: RubricConfig[] = []
   for (const bundle of cells) {
     const cellJDs = allDocs.filter(d => d.source_kind === 'jd' && d.sub_function === bundle.sub_function && d.sub_verticals.includes(bundle.sub_vertical))
     const cellCVs = allDocs.filter(d => d.source_kind === 'cv' && d.sub_function === bundle.sub_function && d.sub_verticals.includes(bundle.sub_vertical))
    
     // Stage 3b
     try {
       const rules = await vb.clusterRules(client, bundle.sub_function, bundle.sub_vertical, cellJDs)
       allRules.push(...rules)
     } catch (err) { console.error(`  3b ✗ ${bundle.sub_function}/${bundle.sub_vertical}: ${(err as Error).message}`) }
    
     // Stage 3c
     try {
       const sigs = await vb.clusterSignals(client, bundle.sub_function, bundle.sub_vertical, cellJDs)
       const rubric = vb.emitRubric({
         run_id, sub_function: bundle.sub_function, sub_vertical: bundle.sub_vertical,
         jds: cellJDs, cvs_in_cell: cellCVs.length, signals: sigs,
       })
       rubrics.push(rubric)
     } catch (err) { console.error(`  3c ✗ ${bundle.sub_function}/${bundle.sub_vertical}: ${(err as Error).message}`) }
   }

4. Add Stage 4b: split allRules by effect.kind and write two files:

   const hardGates: ExpertRulesFile = {
     version: `bootstrap-${dateStr}`, tenant_id: 'calibre',
     generated_from: { run_id, cv_count: cvDocs.length, jd_count: jdDocs.length },
     rules: allRules.filter(r => r.effect.kind === 'drop' || r.effect.kind === 'block_at_l2'),
   }
   const boostPenalty: ExpertRulesFile = {
     version: `bootstrap-${dateStr}`, tenant_id: 'calibre',
     generated_from: { run_id, cv_count: cvDocs.length, jd_count: jdDocs.length },
     rules: allRules.filter(r => ['boost', 'penalty', 'require_evidence'].includes(r.effect.kind)),
   }
   fs.writeFileSync(path.join(EXPERT_RULES_DIR, 'hard-gates.candidate.json'), JSON.stringify(hardGates, null, 2))
   fs.writeFileSync(path.join(EXPERT_RULES_DIR, 'boost-penalty.candidate.json'), JSON.stringify(boostPenalty, null, 2))

5. Add Stage 4c: write one file per rubric:

   for (const rubric of rubrics) {
     const file = `${rubric.sub_function}.${rubric.sub_vertical}.candidate.json`
     fs.writeFileSync(path.join(RUBRICS_DIR, file), JSON.stringify(rubric, null, 2))
   }

6. Update the final summary log: console.log( ${rubrics.length} rubrics, ${allRules.length} rules (${hardGates.rules.length} hard + ${boostPenalty.rules.length} boost/penalty)).

• Step 3: Type-check (cannot run live yet — that’s Task 23)

Run: npx tsc --noEmit Expected: PASS.

• Step 4: Commit

git add scripts/run-vocab-bootstrap.ts src/engine/step-3-normalise/index.ts
git commit -m "feat(rubric-bootstrap): orchestrator wires Stages 3b/3c/4b/4c (Step 6.1 Task 18)"

Task 19 — Ratified applier (apply ratification actions to expert rules + rubrics)

Pure-function module mirroring locked-vocab.ts. Two entry points: applyRuleActions(candidate, actions, reviewer) and applyRubricActions(candidates, actions, reviewer). Returns ratified files + audit logs.

Files:

• Create: calibre-app/src/engine/step-3-normalise/ratified-applier.ts

• Create: calibre-app/tests/engine/step-3-normalise/ratified-applier.test.ts

• Step 1: Write the failing test

Create tests/engine/step-3-normalise/ratified-applier.test.ts. Cover for rules:

1. approve — rule copied through with ratified_by + ratified_at set.
2. tune_delta — boost/penalty rule’s effect.delta replaced; rationale appended.
3. tune_trigger — trigger predicate replaced.
4. reject — rule absent from ratified file; entry persists in audit.
5. add — new rule appended with fresh ULID.

Cover for rubrics:

1. approve_signal — signal carried through to ratified file.
2. move_to_must — signal moves from nice/disqualifying tier to must.
3. move_to_nice — signal moves from must/disqualifying to nice.
4. move_to_disqualifying — signal moves to disqualifying.
5. drop_signal — signal removed from ratified cell.
6. adjust_weight — scoring_weights updated; assert post-update sum still ≈1.0 (within 1e-9).
7. approve_thin — cell carried through with notes: 'thin data — accepted as growth area'.
8. merge_cells — two cells consolidated; merged cell’s notes records the merge.
9. defer_cell — cell absent from ratified output; audit records deferral.
10. add_signal — new signal inserted into the named tier.

• Step 2: Implement

Create src/engine/step-3-normalise/ratified-applier.ts with applyRuleActions + applyRubricActions. Mirrors locked-vocab.ts structure (work map + switch on action kind + final emit + audit log).

• Step 3: Run, watch pass

Run: npx vitest run tests/engine/step-3-normalise/ratified-applier.test.ts Expected: PASS, 15/15.

• Step 4: Commit

git add src/engine/step-3-normalise/ratified-applier.ts tests/engine/step-3-normalise/ratified-applier.test.ts
git commit -m "feat(rubric-bootstrap): ratification action appliers for rules + rubrics (Step 6.1 Task 19)"

Task 20 — Ratification CLI (scripts/review-rubric-bootstrap.ts) — xlsx-driven batch mode

Batch CLI. Reads the marked-up xlsx workbook emitted by Task 23 (from outputs/rubric-ratification/ratification-YYYY-MM-DD.xlsx), parses every tab’s Decision + Tuned_value + Notes columns into action objects, then applies them via the appliers from Task 19. No interactive console row-walking. Worksheet 1 (vocab tab) is parsed here too — review-vocab-candidate.ts from Step 3.1 stays for the legacy interactive path but is bypassed when running the unified xlsx flow. After action application, writes .ratified.json siblings then promotes them by stripping .candidate from filenames + writing audit logs. Archive verification of the 4 grant-vertical stub files runs at the end (no-op confirmation they remain in _archive/ from 2026-06-11).

The applier surface (applyRuleActions + applyRubricActions) from Task 19 is unchanged — Task 20 only swaps the input source from interactive prompts to xlsx parsing. A third entry point applyVocabActions (from Step 3.1’s locked-vocab.ts) is invoked for the vocab tab.

See: amendment §5 Stage 5 — single session, three worksheets. Implementation surface differs (xlsx workbook end-to-end) but decision taxonomy per tab matches the amendment table verbatim.

Files:

• Create: calibre-app/scripts/review-rubric-bootstrap.ts

(No interactive flow — Task 22 covers fixture-driven validation; live workbook validation runs at Task 24.)

• Step 1: Implement the CLI

Create scripts/review-rubric-bootstrap.ts. Skeleton:

import fs from 'fs'
import path from 'path'
import * as XLSX from 'xlsx'
// eslint-disable-next-line @typescript-eslint/no-require-imports
const vb = require('../src/engine/step-3-normalise') as typeof import('../src/engine/step-3-normalise')
 
const EXPERT_RULES_DIR = path.resolve(__dirname, '../../reference/expert-rules')
const RUBRICS_DIR = path.resolve(__dirname, '../../reference/rubric-configs')
const TAG_VOCAB_PATH = path.resolve(__dirname, '../config/tag-vocab.candidate.json')
 
interface CliArgs { input: string; reviewer: string }
 
function parseArgs(): CliArgs {
  const args = process.argv.slice(2)
  const get = (flag: string) => { const i = args.indexOf(flag); return i >= 0 ? args[i + 1] : undefined }
  const input = get('--input')
  if (!input) throw new Error('Missing --input <path-to-ratification.xlsx>')
  return { input, reviewer: get('--reviewer') ?? 'calibre-team' }
}
 
async function main() {
  const { input, reviewer } = parseArgs()
  const wb = XLSX.readFile(input)
 
  // ─── Tab 1: tag-vocab ───
  const vocabRows = XLSX.utils.sheet_to_json<Record<string, string>>(wb.Sheets['tag-vocab'])
  const vocabCand = JSON.parse(fs.readFileSync(TAG_VOCAB_PATH, 'utf-8'))
  const vocabActions = vb.parseVocabSheet(vocabRows, reviewer)
 
  // ─── Tab 2: hard-gates + Tab 3: boost-penalty ───
  const hardGatesCand = JSON.parse(fs.readFileSync(path.join(EXPERT_RULES_DIR, 'hard-gates.candidate.json'), 'utf-8'))
  const boostCand = JSON.parse(fs.readFileSync(path.join(EXPERT_RULES_DIR, 'boost-penalty.candidate.json'), 'utf-8'))
  const hardGateRows = XLSX.utils.sheet_to_json<Record<string, string>>(wb.Sheets['hard-gates'])
  const boostRows = XLSX.utils.sheet_to_json<Record<string, string>>(wb.Sheets['boost-penalty'])
  const ruleActions = vb.parseRuleSheets(hardGateRows, boostRows, reviewer)
 
  // ─── Tabs 4–25: rubrics × 22 ───
  const rubricFiles = fs.readdirSync(RUBRICS_DIR).filter(f => f.endsWith('.candidate.json'))
  const rubricCands = rubricFiles.map(f => JSON.parse(fs.readFileSync(path.join(RUBRICS_DIR, f), 'utf-8')))
  const rubricActions = vb.parseRubricSheets(wb, rubricCands, reviewer)
 
  // Apply
  const { vocabRatified, auditVocab } = vb.applyVocabActions(vocabCand, vocabActions, reviewer)
  const allRules = [...hardGatesCand.rules, ...boostCand.rules]
  const { hardGatesRatified, boostRatified, auditRules } = vb.applyRuleActions(allRules, ruleActions, reviewer)
  const { rubricsRatified, auditRubrics } = vb.applyRubricActions(rubricCands, rubricActions, reviewer)
 
  // Stage 5a — write .ratified.json siblings (for audit trail)
  fs.writeFileSync(path.join(EXPERT_RULES_DIR, 'hard-gates.ratified.json'), JSON.stringify(hardGatesRatified, null, 2))
  fs.writeFileSync(path.join(EXPERT_RULES_DIR, 'boost-penalty.ratified.json'), JSON.stringify(boostRatified, null, 2))
  for (const r of rubricsRatified) {
    fs.writeFileSync(path.join(RUBRICS_DIR, `${r.sub_function}.${r.sub_vertical}.ratified.json`), JSON.stringify(r, null, 2))
  }
 
  // Stage 5b — promote: strip .candidate / .ratified → canonical paths
  fs.writeFileSync(path.resolve(__dirname, '../config/tag-vocab.json'), JSON.stringify(vocabRatified, null, 2))
  fs.renameSync(path.join(EXPERT_RULES_DIR, 'hard-gates.ratified.json'), path.join(EXPERT_RULES_DIR, 'hard-gates.json'))
  fs.renameSync(path.join(EXPERT_RULES_DIR, 'boost-penalty.ratified.json'), path.join(EXPERT_RULES_DIR, 'boost-penalty.json'))
  for (const r of rubricsRatified) {
    const src = path.join(RUBRICS_DIR, `${r.sub_function}.${r.sub_vertical}.ratified.json`)
    const dst = path.join(RUBRICS_DIR, `${r.sub_function}.${r.sub_vertical}.json`)
    fs.renameSync(src, dst)
  }
 
  // Audit logs
  fs.writeFileSync(path.resolve(__dirname, '../config/tag-vocab.audit.json'), JSON.stringify(auditVocab, null, 2))
  fs.writeFileSync(path.join(EXPERT_RULES_DIR, 'hard-gates.audit.json'), JSON.stringify({ ...auditRules, scope: 'hard-gates' }, null, 2))
  fs.writeFileSync(path.join(EXPERT_RULES_DIR, 'boost-penalty.audit.json'), JSON.stringify({ ...auditRules, scope: 'boost-penalty' }, null, 2))
  fs.writeFileSync(path.join(RUBRICS_DIR, 'rubric-configs.audit.json'), JSON.stringify(auditRubrics, null, 2))
 
  // Stage 5c — archive verification (4 grant-vertical stubs)
  const ARCHIVED_STUB_NAMES = ['commodities.json', 'maritime.json', 'offshore.json', 'tech-gaming.json']
  for (const stub of ARCHIVED_STUB_NAMES) {
    if (fs.existsSync(path.join(RUBRICS_DIR, stub))) {
      console.warn(`⚠ legacy stub still at ${stub} — move to _archive/2026-06-11-rubric-stubs-4-vertical/ manually`)
    }
  }
 
  // Stage 5d — summary table (per tab: approved / tuned / rejected / skipped)
  printSummary({ vocabActions, ruleActions, rubricActions })
  console.log(`\n✓ Ratified ${hardGatesRatified.rules.length} hard gates + ${boostRatified.rules.length} boost/penalty + ${rubricsRatified.length} rubrics`)
}
 
main().catch(err => { console.error(err); process.exit(1) })

Decision-column parsing rules (mirror the dropdown values from Task 23):

• Tab tag-vocab — approve / rename (use Tuned_value as new name) / merge (use Tuned_value as merge target) / reject. Blank cell → skipped (no action recorded).
• Tabs hard-gates + boost-penalty — approve / tune-delta (read numeric Tuned_value for effect.delta) / reject / replace-trigger (read Tuned_value as serialized predicate JSON).
• Tabs {sub-function}.{sub-vertical} × 22 — per-signal row: approve / move-to-must / move-to-nice / move-to-disq / drop. Weight header block rows (rows 1–8): Tuned_value updates the named scoring_weights field; rationale captured in Notes.

Move the parser helpers (parseVocabSheet, parseRuleSheets, parseRubricSheets, printSummary) into src/engine/step-3-normalise/ratified-applier.ts next to the appliers so they share the action-type definitions. Re-export from index.ts.

• Step 2: Add npm script

Edit calibre-app/package.json "scripts":

"rubric-review": "tsx scripts/review-rubric-bootstrap.ts"

Invocation: npm run rubric-review -- --input ../outputs/rubric-ratification/ratification-2026-06-12.xlsx --reviewer dan+jon.

• Step 3: Acceptance test — fixture xlsx in → canonical paths out

Create tests/engine/step-3-normalise/review-rubric-bootstrap.test.ts. Pattern: build an in-memory xlsx fixture using XLSX.utils.book_new() with 25 tabs and a small marked-up set of rows per tab, write it to os.tmpdir(), point the CLI at it, then assert:

1. config/tag-vocab.json materialises with only approved/renamed/merged tags (rejected tags absent).
2. reference/expert-rules/hard-gates.json + boost-penalty.json materialise — effect.delta reflects any tune-delta overrides; replace-trigger rows show the new predicate.
3. Exactly 22 ratified rubric files written to reference/rubric-configs/ ({sub-function}.{sub-vertical}.json) — none retain .candidate / .ratified siblings.
4. Each weight-block override on a rubric tab lands in the corresponding scoring_weights field; post-override sum still ≈1.0 within 1e-9.
5. Per-tab summary counts (rows-approved / rows-tuned / rows-rejected / rows-skipped) match the fixture’s known marks.
6. Audit logs exist at config/tag-vocab.audit.json + reference/expert-rules/{hard-gates,boost-penalty}.audit.json + reference/rubric-configs/rubric-configs.audit.json with non-empty entries arrays.

Drive the CLI via await import('../../../scripts/review-rubric-bootstrap') with process.argv swapped; restore after.

Run: npx vitest run tests/engine/step-3-normalise/review-rubric-bootstrap.test.ts Expected: PASS, 6/6.

• Step 4: Type-check

Run: npx tsc --noEmit Expected: PASS.

• Step 5: Commit

git add scripts/review-rubric-bootstrap.ts src/engine/step-3-normalise/ratified-applier.ts src/engine/step-3-normalise/index.ts tests/engine/step-3-normalise/review-rubric-bootstrap.test.ts package.json
git commit -m "feat(rubric-bootstrap): xlsx-driven batch ratification CLI (Step 6.1 Task 20)"

Task 21 — End-to-end orchestration test (mocked, no Claude)

Extend the existing tests/engine/step-3-normalise/orchestration.test.ts to assert the 3-artefact wiring. Mocked Claude responses for Stage 3a, 3b, 3c. Verify the orchestrator pure-function path emits the right shapes.

Files:

• Modify: calibre-app/tests/engine/step-3-normalise/orchestration.test.ts

• Step 1: Add a new describe block

Append a new describe('Stages 3b + 3c + 4b + 4c wiring') block. Mock the Anthropic client to return canned responses for cluster_expert_rules and cluster_cell_signals calls. Assert:

1. clusterRules output flows into a hard-gates list + boost-penalty list split correctly by effect.kind.
2. clusterSignals output flows into emitRubric and produces a RubricConfig with correct display_name.
3. Final orchestrator state has allRules.length > 0 and rubrics.length === number of cells fed in.

• Step 2: Run the full suite — assert nothing regressed

Run: npm test Expected: ALL 47 original tests + ~20 new tests (Tasks 14–19) pass.

• Step 3: Commit

git add tests/engine/step-3-normalise/orchestration.test.ts
git commit -m "test(rubric-bootstrap): Stages 3b+3c+4b+4c wiring (Step 6.1 Task 21)"

Task 22 — Live mine + validation

Execute the extended pipeline. Inspect the three artefact families before handing off to Dan + Jon.

• Step 1: Pre-flight — confirm vocab artefact exists and JD inventory is 41

Run: ls calibre-app/config/tag-vocab.candidate.json && find reference/JD -type f \( -name "*.pdf" -o -name "*.docx" -o -name "*.doc" \) | wc -l Expected: file exists; count 41.

• Step 2: Run the extended bootstrap

Run: cd calibre-app && npm run vocab-bootstrap 2>&1 | tee /tmp/rubric-bootstrap-run.log Expected:

• Stage 1: same as Step 3.1 + per-JD prompt now extracts 6 new fields without errors.
• Stage 2: same 15–25 cells.
• Stage 3a: tag vocab candidate written (overwrites prior file).
• Stage 3b: cluster_expert_rules called per cell that has any recurring_hard_rules or stated_disqualifiers. Log shows N rules per cell.
• Stage 3c: cluster_cell_signals called per cell. Log shows M signals (must=X, nice=Y, disqual=Z).
• Stage 4: writes reference/expert-rules/hard-gates.candidate.json, reference/expert-rules/boost-penalty.candidate.json, and 22 files at reference/rubric-configs/{sub-function}.{sub-vertical}.candidate.json.

Estimated wall-clock: 30–45 min. Cost: USD 20–40.

• Step 3: Validate cardinality + invariants

Create calibre-app/scripts/validate-rubric-candidates.ts (mirrors validate-candidate.ts from Step 3.1 Task 11). Check:

• Both expert-rules files contain only rules with the right effect.kind (hard-gates ∈ {drop, block_at_l2}; boost-penalty ∈ {boost, penalty, require_evidence}).
• Total rule count across both files: 30–60 (amendment §4b cardinality target).
• Exactly 22 rubric-config files exist, one per cell in SOURCE-OF-TRUTH.md §3b.
• Every rubric has scoring_weights summing to 1.0 within 1e-9.
• Every rubric has must_have_signals.length + nice_to_have_signals.length ≥ 1 OR is flagged coverage_low: true via notes.
• Every signal in every rubric has 0 ≤ observed_in_pct ≤ 1, 1 ≤ weight ≤ 5, and source_docs.length ≥ 1.

Run: tsx scripts/validate-rubric-candidates.ts Expected: 0 errors. Warnings acceptable for thin cells (finance-fo, ops-back-office).

• Step 4: Commit the live candidates + validator

git add scripts/validate-rubric-candidates.ts reference/expert-rules/ reference/rubric-configs/
git commit -m "chore(rubric-bootstrap): live candidates + validator (Step 6.1 Task 22)"

Task 23 — Emit ratification xlsx workbook + pre-circulate to Dan + Jon

Per amendment §8 R6 mitigation — pre-circulate 24h ahead so the session is for disputed-row discussion, not first read. The implementation surface for amendment §5 Stage 5 is one xlsx workbook with 25 tabs, not Markdown tables. Decision taxonomy per tab matches the amendment §5 Stage 5 table verbatim — only the rendering surface differs.

Files:

• Create: calibre-app/scripts/emit-ratification-workbook.ts

• Output: outputs/rubric-ratification/ratification-YYYY-MM-DD.xlsx (one workbook, 25 tabs)

• Step 0: Install xlsx dependency

xlsx (SheetJS) is not yet in calibre-app/package.json. Add it:

cd calibre-app && npm install xlsx

Verify package.json "dependencies" now lists "xlsx": "^0.18.x" (or whatever the current pinned major is) and commit package.json + package-lock.json in Step 5 below.

• Step 1: Implement the workbook emitter

Create scripts/emit-ratification-workbook.ts. Skeleton:

import fs from 'fs'
import path from 'path'
import * as XLSX from 'xlsx'
 
const SUB_VERTICAL_CELLS: Array<{ sub_function: string; sub_vertical: string }> = [
  // 22 cells from SOURCE-OF-TRUTH.md §3b — enumerate verbatim
  { sub_function: 'cargo-ops', sub_vertical: 'agri' },
  { sub_function: 'cargo-ops', sub_vertical: 'energy' },
  { sub_function: 'cargo-ops', sub_vertical: 'metals' },
  // ... ship-ops × 5, risk-analytics × 5, finance-fo × 5, ops-back-office × 4
]
 
const VOCAB_DECISIONS = ['approve', 'rename', 'merge', 'reject']
const RULE_DECISIONS  = ['approve', 'tune-delta', 'reject', 'replace-trigger']
const RUBRIC_DECISIONS = ['approve', 'move-to-must', 'move-to-nice', 'move-to-disq', 'drop']
 
const COLUMN_HEADERS = ['proposed_value', 'evidence (source_docs + observed_in_pct)', 'Decision', 'Tuned_value', 'Rationale', 'Notes']
 
function addDecisionValidation(ws: XLSX.WorkSheet, decisionCol: string, firstRow: number, lastRow: number, allowed: string[]): void {
  // SheetJS data-validation hook: set !dataValidation on the sheet
  // Reference: https://docs.sheetjs.com/docs/csf/features/dvranges
  ws['!dataValidations'] = ws['!dataValidations'] ?? []
  ws['!dataValidations'].push({
    sqref: `${decisionCol}${firstRow}:${decisionCol}${lastRow}`,
    type: 'list',
    formula1: `"${allowed.join(',')}"`,
  })
}
 
function main() {
  const date = new Date().toISOString().slice(0, 10)
  const outDir = path.resolve(__dirname, '../../outputs/rubric-ratification')
  fs.mkdirSync(outDir, { recursive: true })
 
  const wb = XLSX.utils.book_new()
 
  // Tab 1: tag-vocab
  const vocab = JSON.parse(fs.readFileSync(path.resolve(__dirname, '../config/tag-vocab.candidate.json'), 'utf-8'))
  const vocabRows = vocab.tags.map((t: { tag: string; source_docs: string[]; observed_in_pct: number }) => ({
    proposed_value: t.tag,
    'evidence (source_docs + observed_in_pct)': `${t.source_docs.join('; ')} (${(t.observed_in_pct * 100).toFixed(0)}%)`,
    Decision: '', Tuned_value: '', Rationale: '', Notes: '',
  }))
  const vocabWs = XLSX.utils.json_to_sheet(vocabRows, { header: COLUMN_HEADERS })
  addDecisionValidation(vocabWs, 'C', 2, vocabRows.length + 1, VOCAB_DECISIONS)
  XLSX.utils.book_append_sheet(wb, vocabWs, 'tag-vocab')
 
  // Tab 2: hard-gates
  const hardGates = JSON.parse(fs.readFileSync(path.resolve(__dirname, '../../reference/expert-rules/hard-gates.candidate.json'), 'utf-8'))
  const hardRows = hardGates.rules.map(serializeRuleRow)
  const hardWs = XLSX.utils.json_to_sheet(hardRows, { header: COLUMN_HEADERS })
  addDecisionValidation(hardWs, 'C', 2, hardRows.length + 1, RULE_DECISIONS)
  XLSX.utils.book_append_sheet(wb, hardWs, 'hard-gates')
 
  // Tab 3: boost-penalty
  const boost = JSON.parse(fs.readFileSync(path.resolve(__dirname, '../../reference/expert-rules/boost-penalty.candidate.json'), 'utf-8'))
  const boostRows = boost.rules.map(serializeRuleRow)
  const boostWs = XLSX.utils.json_to_sheet(boostRows, { header: COLUMN_HEADERS })
  addDecisionValidation(boostWs, 'C', 2, boostRows.length + 1, RULE_DECISIONS)
  XLSX.utils.book_append_sheet(wb, boostWs, 'boost-penalty')
 
  // Tabs 4–25: rubrics × 22 — sub-functions kept contiguous so Dan can pattern-match across siblings
  // (mitigates amendment §8 R3 — cognitive load)
  for (const cell of SUB_VERTICAL_CELLS) {
    const rubricPath = path.resolve(__dirname, `../../reference/rubric-configs/${cell.sub_function}.${cell.sub_vertical}.candidate.json`)
    if (!fs.existsSync(rubricPath)) continue
    const rubric = JSON.parse(fs.readFileSync(rubricPath, 'utf-8'))
    const ws = buildRubricSheet(rubric)
    XLSX.utils.book_append_sheet(wb, ws, `${cell.sub_function}.${cell.sub_vertical}`)
  }
 
  const outFile = path.join(outDir, `ratification-${date}.xlsx`)
  XLSX.writeFile(wb, outFile)
  console.log(`✓ Wrote ${outFile} — ${wb.SheetNames.length} tabs`)
}
 
function serializeRuleRow(r: { id: string; rationale: string; effect: { kind: string; delta?: number }; source_docs: string[]; observed_in_pct: number }) {
  return {
    proposed_value: `${r.id} — ${r.rationale} [${r.effect.kind}${r.effect.delta !== undefined ? ' Δ=' + r.effect.delta : ''}]`,
    'evidence (source_docs + observed_in_pct)': `${r.source_docs.join('; ')} (${(r.observed_in_pct * 100).toFixed(0)}%)`,
    Decision: '', Tuned_value: '', Rationale: '', Notes: '',
  }
}
 
function buildRubricSheet(rubric: {
  sub_function: string; sub_vertical: string; display_name: string
  scoring_weights: Record<string, number>; minimum_pass_score: number
  must_have_signals: Array<{ values: string[]; weight: number; source_docs: string[]; observed_in_pct: number }>
  nice_to_have_signals: Array<{ values: string[]; weight: number; source_docs: string[]; observed_in_pct: number }>
  disqualifying_signals: Array<{ values: string[]; weight: number; source_docs: string[]; observed_in_pct: number }>
}): XLSX.WorkSheet {
  // Header block (rows 1–8): display_name + 5 scoring weights — Dan tunes inline via Tuned_value column
  // Row 9: blank spacer · Row 10: column headers · Row 11+: signals
  const aoa: (string | number)[][] = []
  aoa.push([`Rubric: ${rubric.display_name}`])
  aoa.push(['weight_field', 'current', 'Decision', 'Tuned_value', 'Rationale', 'Notes'])
  for (const k of ['sub_function_match', 'tag_overlap', 'seniority_delta', 'years_experience', 'red_flag_penalty']) {
    aoa.push([k, rubric.scoring_weights[k], '', '', '', ''])
  }
  aoa.push(['minimum_pass_score', rubric.minimum_pass_score, '', '', '', ''])
  aoa.push([])  // spacer row 9
  aoa.push(COLUMN_HEADERS)  // row 10
 
  const sigRows: (string | number)[][] = []
  for (const tier of ['must', 'nice', 'disqualifying'] as const) {
    const list = tier === 'must' ? rubric.must_have_signals : tier === 'nice' ? rubric.nice_to_have_signals : rubric.disqualifying_signals
    for (const s of list) {
      sigRows.push([
        `[${tier}] ${s.values.join(' / ')} (weight=${s.weight})`,
        `${s.source_docs.join('; ')} (${(s.observed_in_pct * 100).toFixed(0)}%)`,
        '', '', '', '',
      ])
    }
  }
  aoa.push(...sigRows)
 
  const ws = XLSX.utils.aoa_to_sheet(aoa)
  // Decision dropdown on signal rows only (starts at row 11 in 1-indexed terms)
  const firstSignalRow = 11
  const lastSignalRow = 10 + sigRows.length
  if (sigRows.length > 0) addDecisionValidation(ws, 'C', firstSignalRow, lastSignalRow, RUBRIC_DECISIONS)
  return ws
}
 
main()

Tab layout summary:

• Tab 1 tag-vocab — one row per proposed tag; Decision dropdown = approve / rename / merge / reject.
• Tab 2 hard-gates — one row per rule with effect.kind ∈ {drop, block_at_l2}; Decision dropdown = approve / tune-delta / reject / replace-trigger.
• Tab 3 boost-penalty — one row per rule with effect.kind ∈ {boost, penalty, require_evidence}; same Decision dropdown as hard-gates.
• Tabs 4–25 {sub-function}.{sub-vertical} × 22 — header block (rows 1–8: title + 5 scoring weights + minimum_pass_score, tuneable in-place via Tuned_value column); blank spacer; column headers; signal rows tagged [must] / [nice] / [disqualifying]. Decision dropdown on signal rows = approve / move-to-must / move-to-nice / move-to-disq / drop.

Every tab carries the same 6 columns: proposed_value | evidence (source_docs + observed_in_pct) | Decision | Tuned_value | Rationale | Notes.

• Step 2: Acceptance test — workbook structure

Create tests/engine/step-3-normalise/emit-ratification-workbook.test.ts. Build a minimal fixture (tag-vocab.candidate.json + hard-gates.candidate.json + boost-penalty.candidate.json + 22 rubric candidate files with 1–2 signals each) in os.tmpdir(), run the emitter via dynamic import with cwd redirected, then re-read the produced xlsx via XLSX.readFile and assert:

1. Workbook has exactly 25 tabs — names: tag-vocab, hard-gates, boost-penalty, plus 22 {sub-function}.{sub-vertical} tabs matching SOT §3b.
2. Every tab’s row 1 (header row, except rubric tabs where it’s row 10) contains the 6 expected columns verbatim.
3. Decision dropdown allowed values on each tab match the locked taxonomy (tag-vocab → 4 values; hard-gates + boost-penalty → 4 values; rubric tabs → 5 values). Read via ws['!dataValidations'].
4. Rubric tabs carry the weight block in rows 1–8 (5 named scoring_weights rows + minimum_pass_score) with editable Tuned_value cells.
5. Sub-functions stay contiguous in tab order (per amendment §8 R3 mitigation).

Run: npx vitest run tests/engine/step-3-normalise/emit-ratification-workbook.test.ts Expected: PASS, 5/5.

• Step 3: Emit the live workbook

Run: cd calibre-app && tsx scripts/emit-ratification-workbook.ts Expected: outputs/rubric-ratification/ratification-YYYY-MM-DD.xlsx written; console reports 25 tabs.

• Step 4: Email Dan + Jon

Subject: Rubric-bootstrap ratification — please review by [date]. Attach the single xlsx workbook. Body explains the tab order walks in sequence: vocab → expert rules (hard-gates then boost-penalty) → 22 rubrics grouped by sub-function. Total estimated session time: 2 hours. Decision dropdowns are pre-validated on every Decision column; Tuned_value is free text.

• Step 5: Commit the emitter, test, dep bump, and workbook

git add calibre-app/scripts/emit-ratification-workbook.ts calibre-app/tests/engine/step-3-normalise/emit-ratification-workbook.test.ts calibre-app/package.json calibre-app/package-lock.json outputs/rubric-ratification/
git commit -m "chore(rubric-bootstrap): emit ratification xlsx workbook + pre-circulate (Step 6.1 Task 23)"

Task 24 — Ratification session + promote to canonical paths

Manual step. The xlsx workbook is the working surface; CLI is post-session batch mode only.

• Step 1: Schedule the session

Block 2 hours with Dan + Jon, ~24h after pre-circulation (Task 23). Walk Tab 1 vocab (~30 min) → Tab 2 hard-gates + Tab 3 boost-penalty (~30 min) → Tabs 4–25 rubrics × 22 (~60 min, grouped by sub-function).

• Step 2: Walk the xlsx live with Dan + Jon

Open outputs/rubric-ratification/ratification-YYYY-MM-DD.xlsx on screenshare. Dan + Jon’s pre-filled Decision columns are the starting point — session resolves disputed rows only, applies Tuned_value adjustments, and Dan adds 3–5 new rows from memory directly in the xlsx. Save the marked-up file as ratification-YYYY-MM-DD.final.xlsx.

• Step 3: Run npm run rubric-review in batch mode

npm run rubric-review -- --input outputs/rubric-ratification/ratification-2026-06-XX.final.xlsx --reviewer "dan+jon@calibre.sg"

CLI reads the marked-up xlsx, applies every row’s action via the Task 19 appliers, writes .ratified.json siblings, promotes to canonical paths, archives 4 old grant-vertical rubric stubs. Prints summary table (rows-approved / tuned / rejected / skipped per tab).

• Step 4: Inspect the ratified outputs

Run:

ls reference/expert-rules/*.json
ls reference/rubric-configs/*.json
cat reference/expert-rules/hard-gates.audit.json | jq '.entries | group_by(.action) | map({action: .[0].action, count: length})'
cat reference/rubric-configs/rubric-configs.audit.json | jq '.entries | length'

Expected: 2 ratified rules files (no .candidate / .ratified siblings remain); 22 ratified rubric files; audit logs with non-empty entries.

• Step 5: Confirm legacy stubs stay archived

Run: ls _archive/2026-06-11-rubric-stubs-4-vertical/ && ls reference/rubric-configs/{commodities,maritime,offshore,tech-gaming}.json 2>/dev/null Expected: archive contains the 4 old stubs; canonical path does NOT contain them.

• Step 6: Commit the ratified outputs

git add reference/expert-rules/ reference/rubric-configs/
git commit -m "feat(rubric-bootstrap): lock expert rules + 22 rubrics via Dan + Jon ratification (Step 6.1 Task 24 — Step 6.1 Phase 1 complete)"

• Step 7: Update SOURCE-OF-TRUTH + PLANS-LOG

In the outer Calibre/ repo:

• Edit SOURCE-OF-TRUTH.md §4 Step 5.3 status → ✅ Locked YYYY-MM-DD.
• Edit SOURCE-OF-TRUTH.md §4 Step 6.1 status → ✅ Rubric content locked YYYY-MM-DD — scorer build next.
• Edit plans/PLANS-LOG.md rows for Step 5.3 and Step 6.1 → ✅ Locked.
• Update the engine map (outputs/engine-map-YYYY-MM-DD.html) accordingly.

git add SOURCE-OF-TRUTH.md plans/PLANS-LOG.md outputs/
git commit -m "docs(sot): Step 5.3 + Step 6.1 rubric content locked"
git push origin main

• Step 8: Notify downstream

Step 3.2 (JD normaliser), Step 5.2 (CV normaliser), Step 5.3 (runtime loader), Step 6.1 (Layer 1 scorer) all unblock. Open the next impl plans in parallel per amendment §10.

Self-Review

1. Spec coverage — every stage of the amendment has at least one task:

• Stage 1 extension (amendment §5 Stage 1): Tasks 13 (types) + 14 (extractor).
• Stage 3b (amendment §5 Stage 3b): Task 15.
• Stage 3c (amendment §5 Stage 3c): Task 16.
• Stage 4b/4c (amendment §5 Stage 4): Tasks 17 (emitter) + 18 (orchestrator).
• Stage 5 (amendment §5 Stage 5): Tasks 19 (appliers) + 20 (CLI) + 23 (pre-circulate) + 24 (drive session).
• Validation (amendment §“What This Plan Does NOT Cover”): Tasks 21 (mocked E2E) + 22 (live + validator).

2. Placeholder scan — no TBDs. Every task names the files it touches and the acceptance test that proves it.

3. Type consistency — DocTermsExtended, JDStatedFields, HardRuleCandidate, ExpertRule, Signal, RubricConfig, RuleAuditEntry, RubricAuditEntry all defined in Task 13 and used consistently from Task 14 onward.

4. Skill alignment — TDD throughout (test before implementation in every code task). Frequent commits (one per task). DRY: Step 3.1 harness reused (Stage 2 unchanged; Stage 3a/4a untouched). YAGNI: no backwards-compatibility shims for the archived 4-vertical stubs; no rubric versioning beyond bootstrap → ratified; no multi-tenant rubric overrides (deferred to Step 10).

Execution Handoff

Plan complete and saved to plans/step-6-phase-1-rubric-bootstrap-impl.md. Two execution options:

1. Subagent-Driven (recommended) — Dispatch a fresh subagent per task. Review between tasks.

2. Inline Execution — Execute tasks in this session using the executing-plans skill.

Which approach?