Matching Engine — Design Spec (pipeline)

Source: plans/pipeline-1.1+2.1+3.2+6.1+7.1+7.2+8.1-matching-engine-design.md · synced 2026-06-12

# Matching Engine — Design Spec (pipeline)

Date: 2026-05-23 (renamed 2026-06-11 from pipeline-jd-cv-matching-design.md) Covers phases: 1.1 (extractor reuse) · 2.1 (JD parser) · 3.2 (JD normaliser) · 6.1 (Layer 1 hard fit) · 7.1 (adjacency resolver) · 7.2 (Layer 2 evidence-grounded LLM, absorbs retired 5.3) · 8.1 (Layer 3 verdict synthesis) Status: Approved — implementation plan exists (plans/pipeline-1.1+2.1+3.2+6.1+7.1+7.2+8.1-matching-engine-impl.md). Build pending. Schema source of truth (overrides this doc’s schemas): plans/amendment-3.2+5.2+6.1-taxonomy-3level.md. Depends on: plans/step-5.1.1-cv-parser-design.md, plans/step-5.1.2-cv-parser-hardening-design.md. (CV normaliser design moved to amendment-3.2+5.2+6.1-taxonomy-3level 2026-06-11 — the older step-5-phase-2-cv-normaliser-design.md was archived to _archive/2026-06-11-superseded-plans/.)

⚠️ AMENDED 2026-06-12 — scoring locks (see SOURCE-OF-TRUTH.md §8 2026-06-12 entry). Where this doc conflicts, the following wins:

1. sub_function_match is NOT a weighted L1 dimension. It is the Step 6.0 entry gate (binary pass/drop, adjacency may salvage), firing before any rubric scoring. Layer 1 = 4 dims: tag_overlap 0.40 / seniority_delta 0.20 / years_experience 0.20 / red_flag_penalty 0.20 (defaults; Dan tunes per sub-vertical).
2. Step 6 execution order: 6.0 entry gates (sub-function gate + ratified drop rules) → 6.1 rubric composite → 6.2 rules overlay (boost/penalty net-capped ±0.5 + block_at_l2/require_evidence flags + threshold re-check ≥ 4.0). The ex-5.3 industry filter is split across 6.0/6.2 — it no longer fires as one block after Layer 1.
3. New code targets: matching/entry-gates.ts + matching/rules-overlay.ts.

Note (2026-05-25): This plan introduced the finance-fo and ops-back-office sub-functions. The full canonical taxonomy now lives in SOURCE-OF-TRUTH.md §3. Reference data inventory: 41 JDs × ~107 CVs (not “65 CVs” as referenced later in this doc).

Note (2026-05-25 later) — STRUCTURAL OVERRIDE:

1. 3-level taxonomy. NormalisedJD and Layer1Result schemas here assume a flat tags[] array. The canonical model is sub_function → sub_vertical → tag (see SOURCE-OF-TRUTH.md §3). Rubric configs key on (sub_function, sub_vertical) and fire at sub-vertical granularity.
2. Bootstrapped vocab. Vocab is no longer pre-defined. Step 3a (lean pilot CV↔JD match across all 41 JDs × ~107 CVs) runs first; emergent terms get clustered, named, and locked. Then the JD normaliser is built using the locked vocab. D7 in STATE-LOG.
3. Step 5 splits. Matching has two sub-stages: 5a LLM inference (the matching engine designed in this doc) + 5b Industry-expert filter — the Calibre team’s recruiter intuition encoded as deterministic heuristics in a config file, scaffold-first then ratified. 5b has no design yet — tracked as D8 in STATE-LOG. The adjacency-resolver pattern in this doc (ratified config + recruiter review queue) is the model to reuse.

Layer-1/2/3 logic, evidence-grounding rule, verdict bands, validation strategy: all still valid.

Context

The CV parser and title normaliser produce structured, normalised candidate records. Calibre has now delivered a JD reference set (reference/JD/) covering five verticals:

• Finance — RM, wealth management, FA, institutional sales (5 JDs)
• Operations — banks, finance brokerage, funds, asset management (4 JDs)
• Operations — cargo, commodities brokerage (19 JDs across Agri, Energy, Metals)
• Operations — ship, ship brokerage (8 JDs across Drybulk, Tanker, Container, Offshore)
• Risk (5 JDs)

This spec defines how a CV is mapped to a JD — the operational core of Annex 3 Deliverable 2 (“AI Candidate Filtering & Matching”). The pipeline is three layers, each with a single responsibility, each independently testable, each producing artefacts that are defensible to the EnterpriseSG audit.

The output of this system is: for every (JD, CV) pair, a structured match record containing a Layer 1 hard-fit score, a Layer 2 deep-assessment rubric with evidence quotes, and a final verdict (Strong yes / Soft yes / Soft no / Strong no).

Architectural Decisions

End-to-End Architecture

JD file (PDF)
     ↓
  JD parser           ← Claude API, structured JSON output (mirrors CV parser shape)
     ↓
  JD normaliser       ← Claude API, applies controlled tag vocab + seniority tier
     ↓
  Normalised JD ────────────────────┐
                                    │
                                    ↓
                          ┌──────────────────┐
  Normalised CV ────────→ │  Layer 1: Hard   │  deterministic scoring
                          │  Fit             │  composite < 4 → drop
                          └────────┬─────────┘
                                   │ survivors
                                   ↓
                          ┌──────────────────┐
                          │  Adjacency       │  config-driven, live-inference fallback
                          │  Resolver        │  attaches "adjacent_match" evidence
                          └────────┬─────────┘
                                   ↓
                          ┌──────────────────┐
                          │  Layer 2: Deep   │  Claude API, evidence-grounded rubric
                          │  Assessment      │  per-dimension score + verbatim quotes
                          └────────┬─────────┘
                                   ↓
                          ┌──────────────────┐
                          │  Layer 3:        │  composite of L1 + L2 → Strong/Soft Yes/No
                          │  Verdict         │  recruiter-overrideable
                          └────────┬─────────┘
                                   ↓
                              MatchRecord

JD Parser — Output Schema

The JD parser mirrors the CV parser pattern: raw extraction only, confidence scores on every field. One Claude API call per JD.

{
  "role": {
    "raw_title":       { "value": "string", "confidence": 0.0–1.0 },
    "job_summary":     { "value": "string", "confidence": 0.0–1.0 },
    "key_objectives":  { "value": ["string"], "confidence": 0.0–1.0 }
  },
 
  "responsibilities": [
    { "category": "string", "description": "string", "confidence": 0.0–1.0 }
  ],
 
  "requirements": {
    "min_years_experience":    { "value": "integer|null", "confidence": 0.0–1.0 },
    "experience_description":  { "value": "string", "confidence": 0.0–1.0 },
    "must_have_skills":        { "value": ["string"], "confidence": 0.0–1.0 },
    "nice_to_have_skills":     { "value": ["string"], "confidence": 0.0–1.0 },
    "technical_knowledge":     { "value": ["string"], "confidence": 0.0–1.0 },
    "education": [
      {
        "qualification": { "value": "string", "confidence": 0.0–1.0 },
        "field":         { "value": "string", "confidence": 0.0–1.0 },
        "required":      { "value": "boolean", "confidence": 0.0–1.0 }
      }
    ],
    "certifications":          { "value": ["string"], "confidence": 0.0–1.0 },
    "tools_and_systems":       { "value": ["string"], "confidence": 0.0–1.0 },
    "languages":               { "value": ["string"], "confidence": 0.0–1.0 },
    "hsse_compliance":         { "value": "boolean", "confidence": 0.0–1.0 },
    "shift_or_24_7":           { "value": "boolean", "confidence": 0.0–1.0 }
  },
 
  "parse_metadata": {
    "source_file":        "string",
    "vertical_hint":      "string",   // derived from folder path
    "parse_status":       "success|partial|failed",
    "overall_confidence": 0.0–1.0,
    "parsed_at":          "ISO 8601 timestamp"
  }
}

JD Normaliser — Output Schema

The JD normaliser maps the parsed JD onto the same controlled tag vocabulary used by the CV normaliser. This is what makes apples-to-apples comparison possible.

interface NormalisedJD {
  sub_function: "cargo-ops" | "ship-ops" | "risk-analytics" | "finance-fo" | "ops-back-office"
  sub_function_display: string
  tags: string[]                          // controlled vocab — same as CV side
  normalised_title: string
  seniority_tier: "Junior" | "Mid" | "Senior" | "Lead" | "Head"
  min_years: number | null
  must_have_tags: string[]                // hard requirements
  nice_to_have_tags: string[]             // bonus
  confidence: number                      // 0–1
}

Taxonomy extension required: the existing CV normaliser taxonomy covers cargo-ops, ship-ops, risk-analytics. The new JD set introduces two more sub-functions:

The Risk vertical (Credit Risk, Market Risk, Operational Risk, Product Controller, Trading Analytics Risk) folds into the existing risk-analytics sub-function — its tags already cover the dimensions.

The CV normaliser must be re-run on the candidate set after the taxonomy is extended.

Layer 1 — Hard Fit (Deterministic)

Operates entirely on normalised records. No LLM calls. Every score is reproducible.

Dimensions

Gating

• Sub-function == 0 → drop. Hard gate. (Exception: skills-adjacency override path — see Adjacency Resolver below.)
• Composite < 4 → drop. Matches grant scope language verbatim.
• Composite ≥ 4 → forward to Layer 2.

Output

interface Layer1Result {
  jd_id: string
  cv_id: string
  scores: {
    sub_function: number
    tag_overlap:  number
    seniority:    number
    years:        number
    red_flag_penalty: number
  }
  composite: number
  passes: boolean
  drop_reason?: "sub_function_mismatch" | "composite_below_4" | "hard_gate"
  adjacent_match?: AdjacentMatchEvidence  // see Adjacency Resolver
}

Adjacency Resolver — Hybrid Live + Ratified

The adjacency resolver runs between Layer 1 and Layer 2. Its job is to surface non-obvious fits that Layer 1’s strict tag matching would miss — e.g. a Tanker operator applying to an LNG cargo role, or an Edible Oils operator applying to a Veg Oil role.

Two-tier resolution

                    ┌──────────────────────────┐
  JD.must_have_tags │  1. Ratified Adjacency   │  config file, recruiter-approved pairs
  vs CV.tags        │     Config Lookup        │  e.g. {vessel:tanker ↔ energy:lng: 0.8}
                    └────────────┬─────────────┘
                                 │ if no entry
                                 ↓
                    ┌──────────────────────────┐
                    │  2. Live LLM Inference   │  Claude proposes adjacency + reasoning
                    │     (Inspiration mode)   │  + confidence + suggested ratification
                    └────────────┬─────────────┘
                                 ↓
                       AdjacentMatchEvidence
                                 ↓
                      Recruiter review queue
                                 ↓
                       Ratified → write to config
                       Rejected → write to deny-list

Ratified adjacency config

Stored as JSON, recruiter-curated, expandable without code changes:

{
  "adjacencies": [
    {
      "from": "vessel:tanker",
      "to":   "energy:lng",
      "weight": 0.8,
      "rationale": "LNG carriers are specialised tankers; chartering and ops knowledge transfers directly.",
      "ratified_by": "team@calibre.sg",
      "ratified_at": "2026-05-XX"
    },
    {
      "from": "agri:edible-oils",
      "to":   "agri:veg-oils",
      "weight": 0.95,
      "rationale": "Same product category, different labelling convention.",
      "ratified_by": "team@calibre.sg",
      "ratified_at": "2026-05-XX"
    }
  ],
  "deny_list": [
    {
      "from": "vessel:container",
      "to":   "energy:lng",
      "rationale": "Box ship ops does not transfer to LNG cargo handling.",
      "rejected_by": "team@calibre.sg",
      "rejected_at": "2026-05-XX"
    }
  ]
}

Live inference prompt contract

When a CV’s tags don’t intersect the JD’s must_have_tags, the resolver calls Claude with:

• The JD’s must_have_tags and normalised_title
• The CV’s tags, normalised_title, and employment_history[].description_summary
• The current ratified adjacency config (as context, to constrain the model toward similar patterns)

Claude returns:

interface AdjacentMatchEvidence {
  proposed_adjacency: { from: string, to: string }
  weight: number                  // 0–1 — model's confidence in transferability
  reasoning: string               // 1–2 sentence explanation
  cv_evidence: string             // verbatim CV quote supporting the adjacency
  jd_evidence: string             // verbatim JD quote that the adjacency satisfies
  recommend_ratification: boolean // model's suggestion for recruiter review
  status: "live_inference"
}

Boost calculation

If an adjacency is resolved (via config or live inference), the Layer 1 tag_overlap score is recomputed with the adjacency weight applied. A Tanker CV against an LNG JD with a ratified 0.8 adjacency contributes 0.8 * 5 = 4 to tag overlap instead of 0.

Self-improving loop

Every live-inferred adjacency lands in the recruiter review queue. Recruiter clicks Approve → entry moves into the ratified config. Reject → entry moves into the deny-list. Over time the config covers the entire space and live inference fires only for novel pairs.

Audit value: the ratified config is the auditable artefact. Live inference is the discovery mechanism. Both are logged. The system can show EnterpriseSG: “Here is our recruiter-validated taxonomy of skills adjacency. Here is how Claude proposed new entries. Here is the human in the loop.”

Layer 2 — Deep Assessment (LLM, Evidence-Grounded)

Runs only on CVs that pass Layer 1 (with or without adjacency boost). Single Claude API call per (JD, CV) pair. Input: normalised JD + parsed CV (full employment history) + adjacency evidence if any.

Rubric dimensions

Mirrors the 6-phase workflow image (Filter Layer 2: Executive summary, Experience alignment, Red flags + signals).

Output schema

interface Layer2Result {
  jd_id: string
  cv_id: string
  executive_summary: string
  dimensions: {
    experience_alignment:     { score: 1–5, evidence: string[], notes: string }
    skills_adjacency:         { score: 1–5, evidence: string[], notes: string } | null
    technical_knowledge:      { score: 1–5, evidence: string[], notes: string }
    tools_and_systems:        { score: 1–5, evidence: string[], notes: string }
    education_certifications: { score: 1–5, evidence: string[], notes: string }
    red_flags:                { items: { signal: string, evidence: string }[] }
    strengths:                { items: { signal: string, evidence: string }[] }
  }
  composite_l2_score: number   // weighted average, weights configurable per sub-function
  confidence: number
}

Evidence requirement

Every dimension score must be backed by at least one verbatim quote from the CV. If the model cannot find a quote, the dimension score is forced to 1 and a low_evidence flag is set. This is the primary hallucination defence. Pattern proven by the parser hardening work (plans/step-5.1.2-cv-parser-hardening-design.md) — apply the same principle here.

Per-sub-function weights

Different sub-functions weight dimensions differently. E.g. for cargo-ops, technical knowledge and tools weigh more than education. For finance-fo, certifications (CFA, etc) and education weigh more. Weights live in a config file per sub-function.

Layer 3 — Verdict Synthesis

Deterministic composition of Layer 1 + Layer 2 into the final 4-band verdict.

Recruiter can override any verdict via UI (Step 10). Overrides are logged with reason and feed back into rubric weight tuning.

File Map

Validation Strategy

JD parser

Manual review of all 41 parsed JDs against source PDFs. Spot-check confidence scores. Same audit pattern used for the CV parser (current-data.md notes from 2026-05-20).

Layer 1

Reproducibility check: same inputs always produce the same scores. Sample 10 (JD, CV) pairs Calibre rates manually — Layer 1 composite must rank them within ±0.5 of Calibre’s ordering.

Adjacency resolver

Cold-start the system with zero ratified adjacencies and observe what live inference proposes across the full 65 CV × 41 JD matrix. Calibre reviews the proposal list — every approved entry seeds the config. This validation pass IS the system bootstrap.

Layer 2

Calibre rates 20 (JD, CV) pairs as ground truth (Strong yes / Soft yes / Soft no / Strong no). Layer 2 verdict must agree on 16 out of 20 (80%). Disagreements get investigated — usually surface either a weight tuning issue or a missing rubric dimension.

End-to-end

The 41 JDs × 65 CVs = 2,665 pairs. After Layer 1 filtering, expect ~5–10% to survive to Layer 2 — ~150–250 pairs. This is the audit demo set: recruiter shortlist generated end-to-end by the system, with full evidence trail per pair.

What This Spec Does NOT Cover

• Supabase storage of MatchRecords — infrastructure setup
• Recruiter review UI (verdict override, adjacency approval queue) — Step 10
• JD intake UX (upload, paste, URL scrape) — Step 10
• Real-time matching as new CVs are sourced — Step 10
• Outreach generation per match — Step 9
• Cost monitoring and rate limiting for Claude calls — operational concern, separate doc

Open Questions

Annex 3 Traceability

Every component above maps to a line in grant/scope.md Deliverable 2. Add corresponding entries to grant/audit-trail.md once implementation lands.

Next Step

Invoke writing-plans to produce the implementation plan for this spec. Sequence:

1. JD parser (small — mirror CV parser)
2. Taxonomy extension + JD normaliser
3. Layer 1 hard fit (pure functions, fully testable)
4. Adjacency resolver — cold-start bootstrap pass with Calibre
5. Layer 2 deep assessment + per-sub-function weight config
6. Layer 3 verdict synthesis
7. End-to-end batch script + validation against Calibre’s 20 ground-truth pairs