Notes from the field on data, cloud, NLP, search, AI, and whatever else I’m working on.
A teaser for my OpenSearchCon 2026 talk on using local differential privacy (LDP) to capture OpenSearch User Behavior Insights (UBI) data without collecting anyone's individual behavior, so you get better relevance and stronger privacy at the same time.
Why teams move from Apache Solr to OpenSearch, and how the OpenSearch Migration Assistant makes the move a repeatable, low-risk process rather than a hand-rolled reindex. Elasticsearch is a supported source too.
A plain-language introduction to learning to rank (LTR): using machine learning to order search results, the judgments and features that train a model, and how the OpenSearch Learning to Rank plugin puts it to work.
An introduction to OpenSearch User Behavior Insights (UBI): what it is, how it captures the queries and clicks that reveal what users actually want, and how that data powers learning-to-rank, A/B testing, and better search relevance.
Seventeen AWS certifications over eleven and a half years, the long gaps in the middle, what recertification actually looks like, and what I'd tell someone starting now.
A practical guide to hybrid search in OpenSearch, covering the normalization and combination techniques, how to tune the weights without fooling yourself, and when hybrid is not worth the complexity.
A plain-language look at OpenSearch neural sparse search: learned sparse retrieval that sits between BM25 and dense vectors, when it's the right tool, and how to set it up.
Why you don't need an LLM for every NLP task, and how the Model Context Protocol (MCP) lets you delegate foundational work like tokenization and named-entity recognition to Apache OpenNLP: cheaper, faster, and easier to scale.
How ONNX Runtime lets Apache Solr run modern transformer NLP models at index time, through Apache Lucene and Apache OpenNLP, with no new services and no training in Java.
A guest post from the Data Driven WV team on evaluating and improving search relevance across three cases (movies, e-commerce, and sports) using manual judgments, LLM-driven query expansion, and OpenSearch tuning.
How Phileas, an open-source text redaction engine, finds and removes personally identifiable information (PII) from documents and streams, and why consistent, configurable redaction matters more than ever in the age of LLMs.
A writeup from the Data Driven WV team on their search relevance project with Mtn Fog, using human judgment and LLMs to score movie search results and learning where the two disagree.
A practical guide to offline search relevance metrics: precision, recall, MRR, MAP, and NDCG, what each one measures, when to use it, and the tradeoffs to keep in mind.
A practical event-driven architecture built on Lambda, SQS, and EventBridge, with the idempotency, concurrency, and failure handling decisions that determine whether it survives contact with production.
AWS shipped a pile of RAG machinery at re:Invent 2024, but the thing that usually breaks a RAG application is retrieval, and almost nobody is measuring it.
What actually matters when a document pipeline handles PHI on AWS, covering the BAA and its per-service carve-outs, required versus addressable safeguards, KMS key boundaries, audit logging that isn't on by default, and why only one of the two de-identification methods can be automated.
A set of Apache NiFi processors that use Apache OpenNLP to build an automated, end-to-end NLP model pipeline: retrieve training data, convert it, train, evaluate, and conditionally deploy the model when it's good enough.
With LLMs dominating NLP, does a classic Java toolkit like Apache OpenNLP still have a role? A head-to-head NER comparison, and why the answer is yes.
An event-driven stack on GCP built from Cloud Run, a Cloud SQL MySQL instance on a private IP, Pub/Sub, and BigQuery, and the connection math and cost floor that decide whether it holds together.
OpenSearch Serverless went GA in January. A look at the OCU billing model, the cost floor nobody mentions, what it can't do yet, and when a managed domain is still the better answer.
Notes from my OpenSearchCon 2022 talk on why organizations should contribute to OpenSearch, the barriers that slow open source adoption, and how to make contributions that stick.
How I brought Hugging Face transformer models to Java by integrating ONNX Runtime into Apache OpenNLP, with no retraining and no external model server required.
A practical walkthrough of approximate kNN search in Elasticsearch 8.1, including the filtering problem nobody warns you about and what the equivalent will look like in Solr 9.
Elastic relicensed Elasticsearch in January, AWS forked it in April, and OpenSearch 1.0 shipped in July. What that actually means if you run a search cluster.
A containerized pipeline that keeps a search ranking model current with what people are talking about right now, and the measurement that tells you whether it helped.
Five NLP microservices that used to be commercial products are now on GitHub, and some notes on when splitting NLP capabilities into separate services is worth it.
How a neural network classifier and a learning-to-rank model can give authors real-time feedback inside a technical-document workflow, from my Lucidworks Activate 2019 talk.
Using Apache Griffin to measure the quality of data moving through Kafka, with the streaming config it actually takes and an honest look at the rough edges.
Making Apache MiNiFi run as a proper Windows service, using Commons Daemon procrun, and why the hard parts had nothing to do with Java.