Skip to Content
PlansStep 5b — Industry-Expert Filter (Design Spec)

Step 5b — Industry-Expert Filter (Design Spec)

Source: plans/step-5.3-industry-filter-design.md · synced 2026-06-12

# Step 5b — Industry-Expert Filter (Design Spec) Date: 2026-05-25 Phase: A.4. 9-step coverage: Step 5b (recruiter intuition layer, complementary to 5a LLM inference). Status: Draft — awaiting greenlight + the Calibre team scaffold-review session. Depends on: Locked tag vocab from Step 3a bootstrap. Normalised JD + CV records. Layer 1 hard-fit composite score. Closes: D8 in STATE-LOG.md.

⚠️ AMENDED 2026-06-12 — Step 6 restructure (see SOURCE-OF-TRUTH.md §8 2026-06-12 entry). Where this doc conflicts, the following wins:

  1. Step ID 5.3 retired. This filter is split into Step 6.0 (drop-type hard gates — fire BEFORE Layer 1 scoring) and Step 6.2 (boost/penalty + block_at_l2 / require_evidence flags — fire AFTER the Layer 1 composite). This filename keeps the 5.3 prefix to preserve links.
  2. Boost/penalty deltas are net-capped at ±0.5 — all firing deltas sum, then clip to ±0.5 on the composite. Any per-rule delta values in this doc (±0.3–1.5) are superseded. Rationale: rules nudge borderline candidates; a rule strong enough to overturn the rubric belongs as an L1 dimension.
  3. Code targets: matching/entry-gates.ts (6.0) + matching/rules-overlay.ts (6.2) — replaces matching/industry-filter/.

Context

Layer 1 (Step 6) is deterministic — sub-function match, tag overlap, seniority delta, years, red-flag penalty. Layer 2 (Step 7) is LLM-driven evidence-grounded deep assessment. Between them, neither captures recruiter intuition — the rules the Calibre team would apply in seconds that no rubric or LLM would surface on its own:

  • “Anyone from $competitor we don’t touch.”
  • “Singapore-based only — no relocation candidates.”
  • “Top-tier shop alumni get a boost regardless of years.”
  • “LNG cargo role — must have HSSE certification, no exceptions.”
  • “China-coverage role — must speak Mandarin even if JD doesn’t say.”
  • “More than two 18-month tenures back-to-back = pass, no matter how good the CV reads.”
  • “If they’ve never worked Singapore hours, they won’t survive 24/7 ops.”

These are recruiter taste rules. They are not “skills” or “experience” — they’re business policies + pattern recognition the Calibre team have built over a career. The grant calls this the “Conditional Mapping Logic engine (recruiter-validated rules)” in AD2.

Step 5b is the system that encodes those rules, applies them mechanically, and lets Calibre + the Calibre team ratify/extend them over time. The ratified rule file IS the proprietary domain layer.

Architectural Decisions

DecisionChoiceRationale
Position in pipelineAfter Layer 1 hard-fit composite is computed; before Layer 2 deep assessmentHard-fit filters obvious misses cheaply; expert filter trims survivor pool further before expensive Claude calls; Layer 2 only runs on (L1 + 5b)-passing candidates
Rule storageJSON config file, hot-reloadableSame pattern as adjacency resolver. Editable without code change. Versionable.
Rule kindsHard (drop / block) + Soft (boost / penalty)Hard = “do not surface”. Soft = “shift composite score by ±x”.
DiscoveryTwo-tier: scaffold + live LLM proposalSame pattern as adjacency. Scaffold seeds the file; Claude proposes new rules from observed match outputs; the Calibre team ratify or reject.
Audit valueEvery rule has author, ratification date, rationaleGrant-defensible. Recruiter-validated.
Scope of effectPer-tenant (Calibre has its own ruleset; another agency would have theirs)Multi-tenancy hook for Step 10

Where Step 5b Sits in the Pipeline

   Normalised JD ─┐                       ┌─ Normalised CV
                  │                       │
                  â–¼                       â–¼
            ┌────────────────────────────────────┐
            │  Layer 1 — Hard Fit (Step 6)       │
            │  Composite < 4 → drop              │
            └─────────────┬──────────────────────┘
                          │ survivors
                          â–¼
            ┌────────────────────────────────────┐
            │  Step 5b — Industry-Expert Filter  │   ← THIS PLAN
            │  Apply ratified rules:             │
            │    hard rules: drop / block        │
            │    soft rules: boost / penalty     │
            │  Composite score updated           │
            │  Composite < 4 (post-rules) → drop │
            └─────────────┬──────────────────────┘
                          │ survivors
                          â–¼
            ┌────────────────────────────────────┐
            │  Layer 2 — Deep Assessment (Step 7)│
            │  Evidence-grounded LLM             │
            └────────────────────────────────────┘

Rule Schema

interface ExpertRule {
  id: string                          // ULID
  category: RuleCategory              // see below
  scope: RuleScope                    // when this rule fires
  effect: RuleEffect                  // what happens
  rationale: string                   // 1–2 sentence why (audit-facing)
  ratified_by: string                 // "team@calibre.sg"
  ratified_at: string                 // ISO 8601
  source: "scaffold" | "live_inference" | "manual"
  active: boolean                     // can disable without deleting
}
 
type RuleCategory =
  | "geography"       // location requirements
  | "competitor"      // company exclusion
  | "pedigree"        // alumni boost / penalty
  | "career_shape"    // tenure patterns
  | "certification"   // hard cert requirement
  | "language"        // language requirement
  | "compensation"    // comp-band signal (advanced — phase 2)
  | "custom"          // anything else Calibre adds
 
type RuleScope =
  | { applies_to: "all" }
  | { applies_to: "sub_function"; sub_function: string }
  | { applies_to: "sub_vertical"; sub_function: string; sub_vertical: string }
  | { applies_to: "tag"; tag: string }                       // fires when JD requires this tag
  | { applies_to: "jd_match"; jd_pattern: string }            // fires when JD raw text matches regex
 
type RuleEffect =
  | { kind: "drop"; reason_code: string }                     // hard — candidate excluded
  | { kind: "block_at_l2"; reason_code: string }              // soft hard — flag but keep visible
  | { kind: "boost"; delta: number; reason_code: string }     // +N to composite
  | { kind: "penalty"; delta: number; reason_code: string }   // –N to composite
  | { kind: "require_evidence"; evidence_field: string }      // raise L2 threshold for this dimension
 
interface ExpertRuleTrigger {
  // Each rule has a trigger predicate that is evaluated against a (NormalisedJD, NormalisedCV, Layer1Result) tuple
  predicate: TriggerPredicate
}
 
type TriggerPredicate =
  | { type: "cv_location_not_in"; values: string[] }
  | { type: "cv_employer_in"; values: string[] }
  | { type: "cv_employer_matches"; pattern: string }
  | { type: "cv_consecutive_short_tenures"; threshold_months: number; count: number }
  | { type: "cv_missing_certification"; cert: string }
  | { type: "cv_missing_language"; language: string }
  | { type: "cv_tag_present"; tag: string }
  | { type: "cv_employer_attended_school"; school_pattern: string }
  | { type: "combo"; op: "and" | "or"; predicates: TriggerPredicate[] }

The rule is fully declarative — JSON only, no code per rule.

Example Rules (scaffold seeds)

These are the ~10 rules built into the initial scaffold. the Calibre team audit + tune + extend.

{
  "version": "scaffold-2026-05-25",
  "tenant_id": "calibre",
  "rules": [
    {
      "id": "01HXR-GEO-SG",
      "category": "geography",
      "scope": { "applies_to": "all" },
      "trigger": { "type": "cv_location_not_in", "values": ["Singapore", "SG"] },
      "effect": { "kind": "penalty", "delta": -1.5, "reason_code": "non-sg-location" },
      "rationale": "Calibre's mandate base is Singapore-resident. Non-SG candidates carry relocation friction.",
      "source": "scaffold",
      "active": true
    },
    {
      "id": "01HXR-SHORT-TENURE",
      "category": "career_shape",
      "scope": { "applies_to": "all" },
      "trigger": { "type": "cv_consecutive_short_tenures", "threshold_months": 18, "count": 2 },
      "effect": { "kind": "penalty", "delta": -1.0, "reason_code": "back-to-back-short-tenure" },
      "rationale": "Two back-to-back 18-month tenures is a stability red flag in Singapore commodities."
    },
    {
      "id": "01HXR-LNG-HSSE",
      "category": "certification",
      "scope": { "applies_to": "sub_vertical", "sub_function": "cargo-ops", "sub_vertical": "energy" },
      "trigger": {
        "type": "combo",
        "op": "and",
        "predicates": [
          { "type": "cv_tag_present", "tag": "lng" },
          { "type": "cv_missing_certification", "cert": "HSSE" }
        ]
      },
      "effect": { "kind": "block_at_l2", "reason_code": "lng-no-hsse" },
      "rationale": "LNG cargo roles in Singapore require HSSE certification by regulation. Block at L2 unless Calibre overrides."
    },
    {
      "id": "01HXR-CHINA-MANDARIN",
      "category": "language",
      "scope": { "applies_to": "jd_match", "jd_pattern": "(?i)china|mandarin|prc|sinochem|cnoo" },
      "trigger": { "type": "cv_missing_language", "language": "Mandarin" },
      "effect": { "kind": "drop", "reason_code": "china-cov-no-mandarin" },
      "rationale": "China-coverage roles fail without Mandarin. Drop early."
    },
    {
      "id": "01HXR-TOP-TIER-BOOST",
      "category": "pedigree",
      "scope": { "applies_to": "all" },
      "trigger": {
        "type": "cv_employer_in",
        "values": ["Vitol", "Trafigura", "Glencore", "Mercuria", "Gunvor", "BP Trading", "Shell Trading"]
      },
      "effect": { "kind": "boost", "delta": 0.5, "reason_code": "top-tier-alumni" },
      "rationale": "Top-tier shop alumni signal demonstrably stronger ops chops in commodities sub-function."
    }
  ]
}

The full scaffold will have 10–15 rules covering each RuleCategory. Calibre + the Calibre team’s job is to (a) say which scaffold rules to keep, (b) tune deltas, (c) add their own.

Live LLM Proposal Loop (mirrors adjacency resolver)

After Layer 1 + 5b runs across all 41 JDs × ~107 CVs:

  1. The system identifies match decisions where Layer 1 + 5b agree the candidate passes but the Calibre team’s gut would say “no” (or vice versa). Detection signals:
    • L1 composite + 5b adjustment lands at composite ≥ 4 but Calibre marks it “Strong no” during review
    • Conversely L1 + 5b drops a candidate but Calibre marks “Strong yes”
  2. Claude is given the (JD, CV, L1 result, applied rules) tuple and asked: “what rule, if added, would have flipped this decision the right way?”
  3. Claude returns:
interface ProposedRule {
  proposed_rule: ExpertRule
  evidence: {
    jd_quote: string
    cv_quote: string
  }
  recommend_ratification: boolean
  rationale: string
}
  1. Proposed rules land in a review queue. the Calibre team approve → write to ratified config. Reject → write to a deny-list.

Self-improving. Same pattern as adjacency. Calibre never has to write a rule from scratch — Claude drafts; Calibre ratifies.

Application Order

For each (JD, CV) pair that survives Layer 1:

  1. Load all active === true rules for the tenant
  2. Filter rules whose scope matches the JD/CV
  3. For each matching rule, evaluate trigger.predicate
  4. Collect fired rules into a list
  5. Hard-rule pass first: if any fired rule has effect.kind === "drop" → drop the candidate, return { outcome: "dropped", fired_rules: [...] }
  6. Block-at-L2 pass: if any fired rule has effect.kind === "block_at_l2" → set block_at_l2: true, but candidate continues
  7. Score adjustment pass: sum all boost and penalty deltas → adjust Layer 1 composite
  8. Post-adjustment threshold: if adjusted composite < 4 → drop with reason: "composite_below_4_after_rules"
  9. Return final result + full audit trail of which rules fired
interface Layer5BResult {
  jd_id: string
  cv_id: string
  l1_composite_in: number
  l1_composite_out: number
  fired_rules: { rule_id: string; effect: RuleEffect; evidence_summary: string }[]
  outcome: "passed" | "dropped"
  drop_reason?: string
  block_at_l2: boolean
}

Every match decision now has a full audit trail of which recruiter rules fired and why. This is the grant audit artifact.

File Map

FileResponsibility
src/lib/matching/expert-filter/types.tsExpertRule, TriggerPredicate, Layer5BResult, ProposedRule
src/lib/matching/expert-filter/config.tsLoad + validate config/expert-filter-rules.json
src/lib/matching/expert-filter/evaluator.tsPure functions: evaluate a TriggerPredicate against (JD, CV, L1Result)
src/lib/matching/expert-filter/applier.tsThe 9-step application order above
src/lib/matching/expert-filter/live-inference.tsClaude call: propose new rule from a disagreement case
src/lib/matching/expert-filter/index.tsPublic API: applyExpertFilter(jd, cv, l1Result, ruleset): Layer5BResult
config/expert-filter-rules.scaffold.jsonScaffold (the 10–15 starter rules) — read-only seed
config/expert-filter-rules.jsonRATIFIED (read-write, recruiter-managed)
config/expert-filter-rules.audit.jsonFull history of ratifications, rejections, edits
scripts/review-rule-proposals.tsCLI for the Calibre team to approve/reject ProposedRule items
tests/lib/matching/expert-filter/evaluator.test.tsEach predicate has unit tests with fixtures
tests/lib/matching/expert-filter/applier.test.tsOrder-of-application correctness

Validation Strategy

CheckHow
Scaffold correctnessCalibre walks through each of the 10–15 scaffold rules and confirms intent matches his mental model
End-to-endRun on the full 41 JDs × ~107 CVs matrix after Layer 1. Calibre + the Calibre team spot-review 30 drops and 30 passes. Disagreement rate should be < 20 %
Live inferenceCold-start with the scaffold only. Run for 2 weeks. Measure: how many proposed rules Calibre approves vs rejects. Approval rate ≥ 50 % means Claude is useful as a proposer
Cost containmentA run on the full matrix at production scale should not exceed 0.10 USD per (JD, CV) pair (the L2 deep assessment is the cost driver, not 5b — 5b is mostly deterministic)
AuditEvery drop / boost / penalty in any match output traces to a rule_id + rationale + ratified_by

What This Plan Does NOT Cover

  • Recruiter UI for rule approval / live inference review — Step 10 productisation surface
  • Rule version diffs between time T and T+30 days — phase 2 of recruiter UX
  • Tenant-level rule isolation in DB — Step 10 Supabase RLS work
  • Cross-tenant rule templates (e.g. “Singapore Geography Pack”) — productisation feature; not in MVP
  • Compensation-band rules — flagged as RuleCategory: compensation but no scaffold rule; Calibre adds when he has the comp data

Open Questions

QuestionOwnerResolution path
When a hard rule fires, is the candidate hidden from Calibre’s UI entirely, or shown with a “dropped by rule X” tag?CalibreDefault: shown with reason. Calibre can override per-candidate.
Should boost deltas be additive or multiplicative?EngineeringAdditive. Simpler audit. Composite math stays linear.
Can a single proposed rule trigger 2+ drops in one pass? Audit shows duplicate fires?EngineeringYes, both fire; both are logged. Composite math sums all penalties cleanly.
How does live inference get the disagreement data if there’s no recruiter UI yet (Phase A)?EngineeringCLI tool — Calibre writes verdicts to a JSON file weekly; live inference reads that file.

Annex 3 Traceability

Annex 3 clauseThis plan
AD2 “Conditional Mapping Logic engine (recruiter-validated rules)“This entire plan. The ratified rules file is the audit artifact.
AD2 “AI Scoring Engine (weighted, vertical-specific criteria)“The scope predicate makes rules sub-function- and sub-vertical-aware.
Annex 4 “marketable to other Headhunting Agencies”Per-tenant ruleset isolation supports white-labelling for other agencies in Step 10.

Next Step

Calibre + the Calibre team scaffold-review session. Walk through the 10–15 scaffold rules, tune deltas, add 3–5 of their own from memory. Then invoke writing-plans to produce the implementation plan.

Last updated on
Parser Hardening — Implementation PlanStep 6.1 — Rubric Bootstrap Extension Implementation Plan