Tuning AWS OpenSearch

Tune an OpenSearch adapter after its query and recovery behavior meets your requirements. The adapter includes both the AWS origin and the selected target service. A setting useful on a native OpenSearch target is not automatically supported by an Elasticsearch-family mapping, and a managed target may own settings that an operated cluster exposes directly. Begin with Trade-offs in AWS OpenSearch and the directed service profiles. This guide uses real index and application controls. It does not define extra t9:* tags.

Separate the tuning layers

Layer	Examples	Owner
Application	Bulk batch size, concurrent writers, retry queue, query shape	Application team
Index	Refresh interval, mappings, primary shards, replicas	Search team within the directed mapping’s supported surface
Deployment	Nodes, memory, storage, zones and supported engine version	Deployment/platform team or managed control plane
Translation/authentication	Request path and target identity for the selected mapping	Cloud Adapter deployment operator

Changing the AWS management endpoint does not route native search traffic. Use the application’s configured OpenSearch REST endpoint. For a native target, commands reach that engine directly; for an Elastic-family adapter, use its origin-facing search endpoint and qualify each setting against the directed mapping.

Example: product imports on DigitalOcean Managed OpenSearch

A product catalog receives a large nightly import while users continue searching. The first trade-off is write visibility versus indexing work. A longer refresh interval can reduce repeated refresh work, but it means new documents take longer to become searchable. It does not relax a customer-facing promise that a newly saved product appears immediately. Start by measuring the existing refresh setting, import rate, bulk-item failures, query latency and time from an accepted write to search visibility. Compare one-second and five-second refresh intervals on an isolated copy of the index, holding shard count and writer concurrency constant. These are example trial values, not universal recommendations. Keep replicas enabled according to the actual availability requirement. The following experiment creates a disposable index, changes its refresh interval and reads the effective settings back. It uses the native OpenSearch REST surface, so it does not require a local adapter process. Install requests, save the code as search-refresh-check.py, and replace the example endpoint with the test endpoint. For an IAM-signed application path, use the actual signing client for acceptance testing instead of this basic-auth fixture.

Prepare access and a baseline

Use a test identity restricted to tuning-products-check. Keep TLS verification enabled and provide the trusted CA bundle if required by the deployment. Record a query-visibility budget before changing anything. Do not run this against the application’s live index.

Create, change and inspect

import getpass
import requests

session = requests.Session()
session.auth = (input("Test username: "), getpass.getpass("Test password: "))
base = "https://search-test.example.com"
index = "tuning-products-check"

def call(method, path, body=None):
    response = session.request(method, base + path, json=body, timeout=30)
    response.raise_for_status()
    return response.json()

if session.head(base + "/" + index, timeout=30).status_code != 404:
    raise RuntimeError("Index exists or existence check failed")
call("PUT", "/" + index, {"settings": {
    "number_of_shards": 1, "number_of_replicas": 0, "refresh_interval": "1s"
}})
try:
    call("PUT", "/" + index + "/_settings", {
        "index": {"refresh_interval": "5s"}
    })
    effective = call("GET", "/" + index + "/_settings")
    actual = effective[index]["settings"]["index"]["refresh_interval"]
    if actual != "5s":
        raise RuntimeError("Unexpected effective refresh interval: " + actual)
    print({"index": index, "refresh_interval": actual})
finally:
    call("DELETE", "/" + index)
    session.close()

Zero replicas keeps this tiny settings fixture independent of test-cluster node count. It is not part of the recommended production tuning change. The fixture only verifies that the setting can be applied and read back; it does not measure throughput or visibility latency.

Measure the workload

Repeat with a production-shaped disposable index and normal replica topology. Use the same import and search workload for each interval. Record successful documents per second, individual bulk failures, query percentiles and write-to-search delay. Explicit refreshes during the measured interval would change the experiment, so do not add them to hide visibility delays.

Accept or reverse

Accept the longer interval only if its measured ingest benefit matters and the application still meets its visibility budget. Restore the original interval if it does not. Check that failed documents have been reconciled before declaring the import complete. Delete only the disposable index and remove test credentials when finished.

The Update Index Settings API defines the update surface; the refresh documentation distinguishes accepted writes from searchable documents.

Example: bound bulk concurrency

Suppose the import still takes too long. Compare a small number of concurrent writers with a higher bounded number while retaining the same batch byte size and refresh interval. Count successful items individually: a successful HTTP bulk response can contain per-document failures. Use a bounded retry queue and preserve document identity so retries do not generate duplicates. Distinguish retryable throttling from mapping errors or invalid documents. Repeatedly retrying an invalid document wastes capacity and can prevent the importer from finishing. A rejected write is useful feedback that admission exceeds the system’s current capacity. Keep batch sizes bounded by bytes as well as document count. A batch of small catalog rows and a batch of large embedded documents can have the same count but very different memory and network impact. Test mixed search traffic during imports; optimizing maximum ingest in isolation can make customer queries miss their latency budget. The Bulk API documents per-item results and refresh controls. Save representative failure responses with secrets removed, then rerun only the failed work after correcting its cause.

Example: shard and replica decisions on OCI or an operated cluster

For OCI Search with OpenSearch, deployment topology follows the OCI mapping. For customer-operated OpenSearch, your team also chooses node roles, persistent volumes and placement. In both cases, distinguish primary-shard layout from replica count. Primary shards divide an index’s data and work. More shards can spread load but add coordination, memory and recovery work; too many small shards consume resources without solving key or query skew. Plan the initial layout from projected data and workload, then verify it with representative routing and queries. Changing primary layout is a migration/reindex decision, not equivalent to adjusting a dynamic replica setting. Replicas provide additional copies and can serve reads, at the cost of storage and write replication. Their usefulness depends on placement across the failure boundaries you intend to survive. A requested replica count is not proof that copies are assigned or that a failed node can be replaced before the remaining system overloads. For an illustrative comparison, create two disposable index generations from the same source corpus: one with the present shard layout and one with the candidate layout. Keep replica policy, hardware and query mix constant. Measure query latency, import duration, disk usage, allocation state and time to recover after an approved test-node interruption. Do not inject failure into a customer cluster merely to collect a benchmark. Choose the layout that meets both normal and degraded-operation budgets. Include disk headroom for segment work, recovery and two generations during reindexing. A configuration that only fits once the old index is deleted leaves little room for safe migration or reversal.

Example: native OpenSearch versus an Elastic-family adapter

For Elastic Cloud and Azure Native Elastic, separate application tuning from backend-specific controls. Bounded bulk writers and a sensible query workload are relevant across both families. OpenSearch plugin settings, lifecycle policies and vector-index parameters require the specific translated contract. Before changing a setting, identify whether it is passed through, translated or owned directly by the target. Inspect the effective native state with an authorized target identity when that distinction affects the result. Do not edit a target lifecycle policy independently and assume the adapter’s logical policy state has changed with it. For vector workloads, hold the embedding model, corpus, query set and required recall constant while comparing supported parameters. Record both query time and result quality. A faster result with missing required neighbors is a failed trial. Do not copy a parameter name from OpenSearch into Elasticsearch merely because both expose a vector query. Managed capacity changes belong to the Elastic or Azure control plane used by the deployment. Check supported tier/zone changes, billing and rollback before resizing. Adding an application retry loop cannot compensate for an unsupported plugin workflow; return to adapter selection when the actual contract is the issue.

Other backend considerations

Scaleway: use its directed profile to check engine, node and storage choices. An AWS IOPS or topology setting is not proof that the target exposes the same control.
Lucenia: qualify the engine/version and exact plugin/query behavior before tuning. A compatible fork is its own backend contract. Operated deployment still requires backup, security and upgrade ownership.
Private Kubernetes OpenSearch: record node-role allocation, storage class, disk growth, snapshot repository and credential ownership together. Increasing pod memory without checking persistent storage or node-placement constraints can move the bottleneck rather than fix it.

Keep a tuning record

For each experiment, retain the directed adapter, engine version, target topology, initial settings, changed setting, workload, accepted-document count, errors, result-quality checks and recovery observations. Record the accepted visibility and latency budgets before the trial. Keep the original setting and a tested reversal procedure. Use Explain when the request traverses a translating adapter. Use native engine statistics, allocation state and query fixtures for native search behavior. Neither an explanation nor a successful settings response substitutes for measuring the workload after the change.

Tensor9 Cloud Adapter

How It Works

Getting Started

Configuration

Deployment

Making Trade-offs

Operations

Tuning

Debugging

Testing

Amazon Web Services

Google Cloud

Microsoft Azure

Reference

Guides

Tuning AWS OpenSearch

Separate the tuning layers

Example: product imports on DigitalOcean Managed OpenSearch

Example: bound bulk concurrency

Example: shard and replica decisions on OCI or an operated cluster

Example: native OpenSearch versus an Elastic-family adapter

Other backend considerations

Keep a tuning record

​Separate the tuning layers

​Example: product imports on DigitalOcean Managed OpenSearch

​Example: bound bulk concurrency

​Example: shard and replica decisions on OCI or an operated cluster

​Example: native OpenSearch versus an Elastic-family adapter

​Other backend considerations

​Keep a tuning record

Separate the tuning layers

Example: product imports on DigitalOcean Managed OpenSearch

Example: bound bulk concurrency

Example: shard and replica decisions on OCI or an operated cluster

Example: native OpenSearch versus an Elastic-family adapter

Other backend considerations

Keep a tuning record