// Open synthetic benchmark
Results, with the limits in view.
Two synthetic sets show the limits: no annotated identifier passes into allowed text on 42 selected cases, but 8 do on 18 new challenge cases. This prototype is not ready for sensitive data.
SEPARATE CHALLENGE SET
8 known misses in a synthetic test.
These 18 cases were written after the initial detector rules, and the rules were not tuned to them. The table counts annotated identifiers still present in text the prototype would allow out. It is not a representative leak rate.
Exact-span recall: names 60%, addresses 50%, secrets 33%, phones 50%, emails 50%. Each type has only a handful of examples.
dmz prototype by type.
A true positive requires the right type and exact text span. FP means a false detection; FN means a missed identifier.
TP = true positive, FP = false positive, FN = missed entity. The dictionary test has one preconfigured synthetic term, Verkefni Gervi. The single address hits a narrow shape rule and blocks the prompt.
PROTOTYPE OUTBOUND BOUNDARY
What would still get through?
On this selected corpus, 0 annotated identifiers remain in allowed outbound text. 2 unfamiliar full names and 1 address block their prompts for review; 10 prompts are blocked overall. This is not evidence of zero leaks on real prompts.
Counted per annotated identifier: masked, blocked with the whole prompt, or still present in allowed text. Nothing was sent to a model. Unannotated context is not assessed, and the corpus is not a representative leak-rate sample.
Comparison: plain regex.
The same 42 cases, without date, checksum, mod-97 or Luhn validation and without a name lexicon. Patterns may overlap; each wrong type counts as a false positive.
Comparison: default Presidio.
Presidio Analyzer 2.2.364 with its default English en_core_web_lg 3.8.0 model. No Icelandic recognizer or custom configuration was added. The same exact spans and types are scored.
How we measured.
Every case has annotated identifiers. The Go detector returns an entity type and UTF-8 byte offsets. The scorer compares exact span and type; partial overlaps do not count.
Test kennitölur use century digit 8. Emails use reserved example domains, cards use test numbers and no customer data is included.