Yelp EngineeringMohammad Mohtasham (Software Engineer); Tao Yu (Software Engineer)7 min readintermediate
ML based ranking using Nrtsearch
Summary
Yelp added an Inference Plugin to Nrtsearch that runs XGBoost and neural‑network models inside the search engine, eliminating a separate scoring service. The plugin extracts features from index documents, loads MLeap bundles from MLflow, and serves predictions on replica nodes with millisecond latency.
- The plugin embeds ML models directly in Nrtsearch, removing network overhead of a standalone inference service.
- Features are pulled from index documents or request parameters, enabling low‑latency per‑document scoring.
- Models are stored as MLeap bundles in MLflow and loaded at node startup; a TensorFlow‑based model server runs in the same JVM.
- Teams can write custom Java scorers via a plugin interface or use a generic Lucene‑expression scorer for rapid adoption.
Search engineers looking to integrate ML ranking need a pattern that reduces latency and operational complexity while staying within the existing search stack.
6/10
