presidio extension bench · measured run ·

Five enterprise classifications, measured before and after

Microsoft Presidio, extended with five context-gated, checksum-validated recognizers for legal, financial, HR, security and IP-confidential data. Every number on this page comes from a real benchmark run, is shipped with the raw corpus and predictions, and recomputes in your browser.

SYNTHETIC CORPUS: every document is invented and every identifier is provenance-safe. No real registry value appears anywhere on this page.
01

Before and after: precision, recall, F1

Stock Presidio (en_core_web_lg, decision threshold 0.5) against the same engine plus five custom recognizers. Scored on 441 gold spans across 120 synthetic documents. Scoring rule: entity type match plus span overlap. Stock entities stay in both arms; the extension adds coverage without touching them.

F1 per entity: baseline (hollow) to extended (filled)
stock baselineextended
02

False positives: what validation and context gating buy

caught during the build, by the harness

Inflected context words never fire. Presidio's context enhancer compares context entries against token lemmas. The legal recognizer first shipped with "executed" and "parties" in its context list; lemma matching looks for "execute" and "party", so the hard legal cases recalled 0.50. The eval set flagged it, the lemmas were fixed, recall went to 1.00. Default regex flags accept lowercase credentials. PatternRecognizer compiles with IGNORECASE by default, which would read the prose string "akia0example0example" as an AWS key. The credential recognizer pins case-exact flags. Both are the kind of defect only a measured eval set catches before production does.

Exhibits: each one is a planted span from the corpus

Red marks what a pattern-only pass flags. The extended run stays silent on all of them while holding 1.000 recall on the true spans. Open any exhibit in the document browser to see it in context.

03

Throughput, latency, scaling

docs per second, single process (higher is better)
worker scaling, extended arm, spawn pool
the measured trade behind model choice

04

Document browser: real predictions on every document

Every span below is rendered from the recorded predictions of the selected arm, not hand-drawn. Switch arms to watch the naive pattern pass bleed false positives, and overlay the gold labels to check the misses yourself. Click a legend chip to hide or show a type.

05

Method, provenance, and how five becomes fifteen

Run configuration

Detection methods used

pattern
anchored regex per classification, word-boundary disciplined
validation
real ABA checksum on routing candidates; degenerate-placeholder reject on credentials
context
lemma-based context words gate id-shaped matches below the 0.5 threshold unless enterprise context precedes them
dictionary
deny-list codename recognizer with a case validator
model
spaCy NER carries PERSON and friends in the same pass, untouched

Identifier provenance: nothing here belongs to a real registry

Repeatable extension, not one-off recognizers

Each recognizer is a small subclass registered into the analyzer registry: a pattern set, an optional validator, a context list, a decision score. Adding classification number six is a new subclass plus its positive, negative and ambiguous test cases in the corpus; the benchmark, the trap audit and the selftest pick it up with no pipeline changes. That is what keeps the library expandable without redesigning the system.

06

Selftest: do not take this page's word for it

The Python harness ships a 27-assertion selftest (identifier provenance, independent metric rescore, trap liveness, throughput sanity). This page embeds the raw corpus and the raw predictions, so your browser can re-derive the headline numbers itself: confusion counts and F1 for all four arms, the ABA checksum on every routing span, Luhn on every card span, and the zero-hit trap audit. Exact integer agreement is required, float tolerance 1e-9 on cross-language division.

running…