Lobstersian erik varatalu7 min readintermediate
Labeled matches: why is this not in every regex engine?
Summary
The author shows how to label tokens (dates, money, emails, etc.) with a single regex pass using extended operators (`&` for intersection, `~` for complement) in the resharp library. A tiny benchmark compares 10 patterns against spaCy’s NER, reporting ~1.9 GB/s (≈4500× faster) on 8 threads. The post lists the concrete patterns and argues that for deterministic, regular‑language entities regex can…
- Extended regex operators (`&`, `~`) let you express set intersection and complement directly in patterns, enabling precise token categorization without ML.
- A single pre‑compiled regex can label many entity types in one pass, achieving throughput in the low‑GB/s range on multi‑core CPUs.
- For strictly regular entities (e.g., ISO dates, fixed‑format money), regex gives 100 % deterministic accuracy, avoiding the uncertainty of statistical NER.
- The author provides ready‑to‑use patterns for DATE, MONEY, PERCENT, EMAIL, URL, NUM, NAME, VERB, ADJ, ADV, demonstrating practical usage.
When you need high‑throughput, low‑latency text tagging (e.g., log processing, edge services, or resource‑constrained devices), a deterministic regex solution can replace heavyweight NER models, saving CPU, memory, and power while guaranteeing correctness for regular‑language entities.
6/10
