Migrating Apache Solr to OpenSearch with the Migration Assistant

Moving a production search cluster from one engine to another is the kind of project that sounds simple until you actually plan it. You have to recreate your schema and settings, move every document, validate that results still look right, and cut over without breaking the application in front of real users. Doing that by hand is slow and easy to get wrong.

The OpenSearch Migration Assistant exists to make that move a repeatable, low-risk process instead. I’m a maintainer on the project, and this post is a high-level look at why teams migrate from Apache Solr to OpenSearch and how the Migration Assistant helps them do it. If you’re coming from Elasticsearch instead, that’s a supported source too, and I’ll come back to it.

Why move from Solr to OpenSearch

Apache Solr is a capable, battle-tested search engine, and plenty of teams run it happily. The pull toward OpenSearch usually comes down to a few things:

• It ships with OpenSearch Dashboards, security, alerting, and observability in the same project, so you’re not assembling those pieces yourself.
• Its ecosystem includes built-in vector search and k-NN, machine learning, learning to rank, and User Behavior Insights, which matters if you’re building semantic or hybrid search.
• It’s Apache 2.0 licensed and developed in the open under the OpenSearch Software Foundation, with a broad set of contributors and vendors behind it.
• If you’d rather not run it yourself, it’s available as a managed service that takes operational load off your team.

None of that means you have to leave Solr. It means that when the reasons add up, the migration itself shouldn’t be the thing that stops you.

What the Migration Assistant is

The Migration Assistant is a toolkit that moves your data and configuration into OpenSearch and gives you a guided, checkpointed path from start to cutover. You deploy it alongside your clusters, drive it from a migration console, and it handles the mechanical work that would otherwise be a pile of custom scripts.

It’s built around a few components that map to the work a migration actually requires:

• Metadata migration moves the configuration that surrounds your data, such as index settings, templates, and aliases, so the target is shaped correctly before any documents arrive.
• Backfill reindexes your existing documents into OpenSearch, bringing the data up to the target’s current Lucene version along the way.
• Validation lets you compare the source and target so you can confirm the migrated cluster behaves the way you expect before you rely on it.

How a Solr migration works

At a high level, a Solr migration follows the same shape every time. You assess the source and review anything that needs to change on the way over. You migrate the metadata so the target cluster is configured to receive your data. You backfill the documents from Solr into OpenSearch. Then you validate the result, and when you’re satisfied, you cut your application over to OpenSearch.

The point of running this through the Migration Assistant rather than by hand is that each of those steps is a defined stage with approval gates and validation, not a one-shot script you hope worked. If something looks wrong, you find out before your users do.

Preparing a Solr 9 backup in S3

Solr doesn’t have a snapshot format the tooling can read directly the way it reads an Elasticsearch or OpenSearch snapshot, so a Solr migration starts from a regular Solr backup that you place in Amazon S3. From there the Migration Assistant’s Solr reader pulls the Lucene segment files out of the backup, reconstructs the documents, and translates your schema.xml field types into OpenSearch mappings.

The convenient part is that Solr 9 can back up straight to S3. First enable the s3-repository module (for example by adding it to SOLR_MODULES) and define an S3 backup repository in solr.xml:

<backup>
  <repository name="s3" class="org.apache.solr.s3.S3BackupRepository" default="false">
    <str name="s3.bucket.name">my-s3-bucket</str>
    <str name="s3.region">us-east-2</str>
  </repository>
</backup>

The repository uses the default AWS credential provider chain, so it picks up credentials from environment variables, a credentials file, or an IAM role if Solr is running on AWS.

With that in place, trigger a backup of your collection through the Collections API and point it at the s3 repository:

curl "http://localhost:8983/solr/admin/collections?action=BACKUP&collection=mycollection&name=my_backup&repository=s3&location=backups"

That writes the backup into your bucket under the location you gave it. You then point the Migration Assistant at that backup in S3 as the source for the backfill, and the rest of the flow is the same as any other migration.

You describe the migration itself in a workflow configuration, which is YAML. Here’s one that takes the existing Solr backup sitting in S3 and migrates it into OpenSearch:

sourceClusters:
  solr-source:
    endpoint: https://solr.example.com:8983
    allowInsecure: false
    version: SOLR 9.6.1
    snapshotInfo:
      repos:
        default:
          awsRegion: us-east-2
          repoPathUri: s3://my-s3-bucket/backups
      backups:
        customerBackup:
          repoName: default
          externalBackupName: my_backup
          collectionAllowlist:
            - mycollection

targetClusters:
  target:
    endpoint: https://target.example.com:9200
    allowInsecure: false

snapshotMigrationConfigs:
  - fromSource: solr-source
    toTarget: target
    perSnapshotConfig:
      customerBackup:
        - metadataMigrationConfig: {}
          documentBackfillConfig: {}

The snapshotInfo block points at the S3 repository and the specific backup you created, collectionAllowlist picks which collections to bring over, and snapshotMigrationConfigs ties the source backup to the target and runs both the metadata migration and the document backfill.

Elasticsearch and OpenSearch sources

Solr isn’t the only supported starting point. The Migration Assistant also migrates from Elasticsearch and from older OpenSearch clusters, which is the common path for teams doing a major version upgrade.

For those sources there’s an additional capability worth calling out: capture-and-replay. The tooling can record live traffic from the source cluster and replay it against the target, so you can verify the new cluster under realistic load and perform a near zero-downtime cutover with the confidence that it already handled your real queries. That live-traffic path is available for Elasticsearch and OpenSearch sources. Solr migrations use the snapshot-based backfill with a planned cutover.

Deploying on AWS

If you’re running on AWS, there’s a packaged version called Migration Assistant for Amazon OpenSearch Service that deploys the whole thing onto Amazon EKS for you. To give you a sense of what using it actually looks like, here are a few of the steps.

You start by grabbing the bootstrap script, which handles cluster creation, IAM wiring, image mirroring, and installing the Helm chart:

curl -sL -o aws-bootstrap.sh \
  "https://solutions-reference.s3.amazonaws.com/migration-assistant-for-amazon-opensearch-service/latest/aws-bootstrap.sh" \
  && chmod +x aws-bootstrap.sh

You can also get the script straight from the GitHub repository , or from the GitHub releases if you want to pin to a specific version.

Then you run it to stand everything up. This one deploys into a new VPC:

./aws-bootstrap.sh \
  --deploy-create-vpc-cfn \
  --stack-name MA \
  --stage dev \
  --region us-east-2

Once it finishes, you can check that the pods are running and drop into the migration console, which is where you drive the migration itself:

kubectl get pods -n ma
kubectl exec -it migration-console-0 -n ma -- /bin/bash

From inside the console the flow is the same no matter where you’re coming from: confirm the version, load a workflow configuration, run a pilot, validate, and then run the full migration.

Submitting and managing the migration

Once your workflow configuration is ready, you kick the migration off from the migration console with a single command:

workflow submit

That hands the workflow to the orchestrator, which starts working through the phases you configured, such as the metadata migration and then the document backfill. If you’d rather have the command block until the work is done, add --wait.

To keep an eye on things, use:

workflow manage

This opens an interactive view of the migration where you can watch each phase progress, approve the gates that pause the workflow for a human check, and step in if something needs attention. It’s the same command whether you’re migrating from Solr, Elasticsearch, or OpenSearch.

Getting started

The best starting point is the Migration Assistant documentation , which walks through deploying the tooling and running each phase for your source and target versions. The code, issues, and releases live in the opensearch-migrations repository on GitHub. It’s an active project, and contributions and questions are welcome.

Summary

Migrating search engines is mostly a risk-management problem: recreate the configuration, move the data, prove it works, and cut over without surprises. The OpenSearch Migration Assistant turns that from a hand-rolled effort into a guided, validated process, whether you’re coming from Apache Solr, Elasticsearch, or an older OpenSearch cluster.

If you’re weighing a move to OpenSearch and want to talk through what it would take, please reach out.