Hybrid Search in OpenSearch: Making BM25 and Vectors Work Together

Sooner or later somebody asks why the search box finds “wireless mouse” but not “cordless mouse”. A week later somebody else asks why the shiny new semantic search can’t find the SKU they pasted straight out of an invoice. Those are the same conversation. You have two retrievers that fail in opposite directions, and the obvious fix is to run both.

Running both is the easy part. The part that eats a week is deciding what to do with two sets of scores that have nothing to do with each other. That’s really what hybrid search in OpenSearch is about.

Two retrievers that fail differently

BM25 has been quietly good at its job for thirty years. If the words are in the document, it finds it, ranks it sensibly, and you can read the explain output afterward and understand exactly why. What it will never do is bridge vocabulary. Nobody writes a support article titled “laptop won’t charge”. They write “battery fails to draw power from the adapter”, and those two strings share almost nothing.

Embeddings handle that without breaking a sweat and then fall over somewhere else. They are good at paraphrase and bad at exactness. Model numbers, SKUs, error codes, and people’s names are precisely the tokens an embedding smooths into mush, and they’re also the queries where a near miss is worth nothing. Nobody wants the part that’s similar to the one they need.

Look at a week of real query logs and you’ll see both kinds, often from the same person in the same session. That’s the whole argument for running both.

Two retrievers, two number systems

The trouble starts at the merge. BM25 scores are unbounded and depend on the corpus, the field, and how long the query is. A top hit might score 18.4 on one query and 6.2 on the next, and neither number means anything on its own. Vector similarity usually lands in a narrow band near the top of a 0 to 1 range, where the gap between a great match and a mediocre one might be 0.04. Add those together and the lexical clause wins every single time, for reasons that have nothing to do with which document is better.

So hybrid search is two steps, not one. Normalize each subquery’s scores onto a common scale, then combine them. OpenSearch does that in a search pipeline, in a processor that sits between the query and fetch phases. The shards run the subqueries and hand back scores, the coordinating node rescores everything, and only then does it go fetch the documents it settled on.

Getting it running

Two pieces. A search pipeline holding the normalization processor , and a hybrid query that points at it.

All the interesting decisions live in the pipeline:

curl -X PUT "http://localhost:9200/_search/pipeline/hybrid-pipeline" \
  -H 'Content-Type: application/json' -d '
{
  "description": "Normalize and combine hybrid search results",
  "phase_results_processors": [
    {
      "normalization-processor": {
        "normalization": {
          "technique": "min_max"
        },
        "combination": {
          "technique": "arithmetic_mean",
          "parameters": {
            "weights": [0.4, 0.6]
          }
        }
      }
    }
  ]
}'

The query just lists the subqueries and names the pipeline:

curl -X POST "http://localhost:9200/my-index/_search?search_pipeline=hybrid-pipeline" \
  -H 'Content-Type: application/json' -d '
{
  "size": 10,
  "query": {
    "hybrid": {
      "queries": [
        {
          "match": {
            "passage_text": "laptop will not charge"
          }
        },
        {
          "neural": {
            "passage_embedding": {
              "query_text": "laptop will not charge",
              "model_id": "<your-model-id>",
              "k": 100
            }
          }
        }
      ]
    }
  }
}'

Watch the order of those subqueries. The weights array is positional, so swapping the clauses without swapping the weights leaves you with a config that looks correct and ranks wrong. It’s an easy one to get backwards and it doesn’t announce itself.

Two other constraints are worth knowing before you build much on this. A hybrid query takes at most five subqueries, and it has to be the top-level query. Wrapping it in function_score, constant_score, script_score, or boosting sometimes errors and sometimes just silently skips the normalization pipeline, which is the worse outcome. If you need boosting on top, the usual move is a bool query with should clauses. You give up normalization and keep everything else.

Picking a normalization technique

There are three, and the differences only show up on queries where one retriever is having a bad day. Which is also where your relevance problems live.

min_max is the default and the one you’ll use unless you have a reason not to. It stretches each subquery’s results so the best becomes 1 and the worst becomes 0. Easy to reason about, and easy to wreck with an outlier. One runaway BM25 score squashes everything beneath it into the low end, so results two and three look weak when they aren’t.

l2 divides each score by the L2 norm of that subquery’s score vector. It keeps the relative gaps between results more honest, and it doesn’t force the top hit to exactly

1. That last part matters more than it sounds. Under min_max, a subquery that returned nothing genuinely good still gets to promote its least bad result to a perfect score.

z_score standardizes on the mean and standard deviation instead of the range, which is the textbook answer to outlier-sensitive scaling. It landed in OpenSearch 3.0 and only works with arithmetic_mean. The OpenSearch blog has a good writeup of the reasoning if you want the details.

If you stay on min_max, learn lower_bounds and upper_bounds. They pin the ends of the range per subquery instead of deriving them from whatever happened to come back, which is the direct fix for the least-bad-result problem. Set a floor and weak matches stop pretending to be strong ones.

Picking a combination technique

Now the scores are comparable and you have to actually merge them. The three score-based options really only differ on one question. How much do they forgive a document that one retriever loved and the other ignored?

• arithmetic_mean is the default and the softest. Strong lexically and weak semantically still ranks high. That’s what you want when either signal on its own is a good enough reason to show something.
• geometric_mean punishes disagreement. A near-zero on one side drags the whole thing down no matter how good the other side was. Reach for it when you want the two retrievers to agree before anything gets promoted.
• harmonic_mean sits in between and is ruled by the weaker of the two scores. It’s the cautious pick, for when a bad result costs you more than a missed good one.

Weights are decimal percentages, one per subquery, in the same order as the queries array, and they have to sum to 1.0. That’s the dial you’ll spend your time on.

The fourth option ignores scores entirely. The score ranker processor implements reciprocal rank fusion , which looks only at where each document landed in each list. A document contributes one over its rank plus a rank constant, summed across the subqueries. That’s the whole algorithm.

curl -X PUT "http://localhost:9200/_search/pipeline/rrf-pipeline" \
  -H 'Content-Type: application/json' -d '
{
  "description": "Combine hybrid search results with reciprocal rank fusion",
  "phase_results_processors": [
    {
      "score-ranker-processor": {
        "combination": {
          "technique": "rrf",
          "rank_constant": 60
        }
      }
    }
  ]
}'

RRF is a good place to start when you don’t trust your score distributions yet, which is most of the time early on. Nothing strange happening inside one retriever can blow up the merge, because the merge never sees the scores. The flip side is that it throws away real information. If one retriever is confident and the other is guessing, RRF has no way to tell. rank_constant is where you tune that. Smaller values put more weight on the top few positions and larger ones flatten things out. The default of 60 comes from the original paper and is a perfectly reasonable place to leave it.

Tuning the weights without kidding yourself

This is where these projects tend to go sideways. Somebody tries eight queries by hand, notices that 0.3 lexical and 0.7 vector looks better on those eight, and ships it. That isn’t tuning. That’s picking a number that fits eight examples.

Doing it properly needs a query set that looks like real traffic, including the long tail where most of the pain actually is, relevance judgments for those queries, and a metric you chose before you started staring at results. If you’re already collecting behavioral data, User Behavior Insights hands you the first two almost for free, since it records what people searched for and what they clicked.

Then let the Search Relevance Workbench grind through the options. Its hybrid search optimizer tries every combination of normalization technique, combination technique, and weight in 0.1 steps, plus RRF at a handful of rank constants, scoring each against your judgments.

curl -X PUT "http://localhost:9200/_plugins/_search_relevance/experiments" \
  -H 'Content-Type: application/json' -d '
{
  "querySetId": "<your-query-set-id>",
  "searchConfigurationList": ["<your-search-configuration-id>"],
  "judgmentList": ["<your-judgment-list-id>"],
  "size": 10,
  "type": "HYBRID_OPTIMIZER"
}'

What comes back is usually not a peak. It’s a plateau.

There are two things to take from a shape like that. Hybrid is beating both single retrievers across a wide band, which is the result you were hoping for. And the exact top of the plateau is not worth chasing. The gap between 0.4 and 0.6 there sits inside the noise of most query sets, so a weight that wins by half a point today will not be the winner against next month’s traffic. Pick something near the middle and go do something else.

If you do get a sharp peak, don’t celebrate. Narrow optima usually mean the query set is too small or too homogeneous to be telling you anything. Hold out a slice of your queries and confirm the winner still wins on them. Ten minutes of work, and it catches this.

Did it actually help

Report on the same metric you tuned on. NDCG at 10 is the usual pick for hybrid work, since it’s graded and position weighted, and reshuffling the first page is the entire point of the change.

Compare against a real baseline, meaning your current production ranking tuned as well as you’d honestly tune it. Beating an untuned BM25 config proves that the config was untuned. That’s obvious written down. It is much less obvious at 6pm on the day the numbers finally moved.

Two things quietly change those numbers. The normalization processor only sees what each subquery returned and doesn’t sample any deeper on its own, so size matters more than usual. OpenSearch’s own benchmarking landed on 100 to 200 for corpora up to around 10 million documents, with rising latency and no relevance gain past that. And if you paginate, set pagination_depth and leave it alone between pages, because changing it changes the underlying result set and therefore the order after normalization.

Keep an eye on tail latency too. You’re running two retrievers now, and if one of them encodes the query with a model then you’ve put inference in the request path. The mean will look fine. Go look at p99.

When to skip it

Hybrid search earns its keep in a lot of places. It’s also a model to deploy and version, an embedding field inflating your index, another pipeline to keep in sync, and more things that can be slow at the wrong moment. Some situations where I’d push back on it.

• The lexical side isn’t tuned yet. Field boosts, analyzers, synonyms, phrase matching. All boring, all cheap, and they often recover most of what people are hoping hybrid will hand them. Do that first, then measure again.
• The vocabulary is controlled. Internal catalogs, code search, log search. The users type the words that are in the documents, so semantic recall is solving a problem you don’t have.
• There’s no way to evaluate. No query set, no judgments, no agreed metric. Hybrid search has enough knobs to let you convince yourself of anything, and with nothing to check against, that’s what will happen.
• The latency budget is tight and the hardware is CPU only. Neural sparse search in doc-only mode is worth trying first. You get semantic recall on the inverted index with no vector index and no query-time inference.
• The problem is ranking, not retrieval. If the right documents already come back and just come back in the wrong order, learning to rank or a reranker goes at that directly instead of changing what you retrieve.

None of those rule out hybrid later. A lot of mature setups end up with a hybrid query doing retrieval, a reranking or learning-to-rank layer on top, and behavioral data telling them whether any of it worked. Hybrid is a layer, not the answer.

Summary

BM25 and vector search fail on different queries, and hybrid works because of that. The machinery is a search pipeline that puts both sets of scores on a common scale and then merges them, with min_max, l2, or z_score for the first part and one of the means or RRF for the second. Honestly, the specific settings matter less than having some way to tell whether you’re improving anything. Build the query set and the judgments first. Everything after that is a sweep and a decision.

These processors and parameters have moved around between OpenSearch versions, so check the hybrid search documentation for the version you’re running before wiring anything up.

If you’re weighing hybrid search for your own stack and want a second opinion on whether it’s the right next step, please reach out.