> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tensor9.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Debugging AWS OpenSearch

> Investigate deployment status, search endpoint access, index behavior and resource pressure for the selected OpenSearch mapping.

Separate domain provisioning from search execution. An AWS OpenSearch management request describes or changes a domain. Indexing and search use an HTTP endpoint with engine-specific authentication and APIs. A healthy deployment status does not prove that a particular index, mapping, query or credential works.

Read the [directed service profile](https://docs.tensor9.com/cloud-adapter/service-catalog/aws/databases-storage/opensearch-domain.md) before interpreting a failure. An OpenSearch-engine target, an Elasticsearch-family mapping and a retained AWS domain have different request routes and operating responsibilities. Follow the profile's documented application route: native engine traffic can go directly to the target, while an Elastic-family mapping can require an origin-facing translation path. Changing an AWS management endpoint does not automatically redirect the application's search client.

## Establish the exact context

Record the adapter version, client version, actual backend engine/version, selected mapping, index or alias, endpoint, authentication method and topology. Record both the logical AWS region used by a signed client and the target deployment's physical region. Include replica settings, storage class and managed-versus-operated ownership when investigating availability.

Keep the original HTTP status and engine error type. For bulk ingestion, inspect [individual item results](https://docs.opensearch.org/latest/api-reference/document-apis/bulk/) as well as the outer HTTP response; a request-level success does not establish that every document was indexed. Avoid replaying a timed-out write before determining its possible effects and the application's document-ID/idempotency rules.

| Symptom | Smallest useful check | Likely next investigation |
| - | - | - |
| Domain creation or resize remains pending | Read the selected provider's operation status, failure details and deployment events | Quota, capacity, permissions, storage or an incomplete configuration change |
| TLS failure or connection timeout | Check the application's actual search hostname, trusted CA, port and network access | Wrong route, private networking or certificate mismatch |
| HTTP 401 or 403 | Identify whether the response comes from the origin-facing adapter or native engine | IAM identity/signing, backend credentials or index permissions |
| Index not found or unexpected empty results | Read the exact index/alias and the application's query target | A different index, alias, routing value, tenant or visibility interval |
| Cluster health is yellow or red | Read cluster health and the affected index's shard allocation state | Replica placement, missing primary shards or resource pressure |
| HTTP 429 or rejected operations | Inspect bounded workload concurrency, rejected work, CPU, heap and disk | Overload, uneven shard placement or insufficient capacity |
| Plugin or endpoint error | Compare that exact operation with the selected engine/version's profile | An unsupported plugin, translation or client API assumption |

On AWS, configuration changes have their own [processing and change-progress states](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-configuration-changes.html). For a retained AWS domain, inspect those states. For another managed service or an operated Kubernetes cluster, inspect its actual operation or rollout events. An AWS-style label cannot replace target-state evidence.

## Start with a bounded read-only engine request

Using the application's authenticated search client, request `GET /_cluster/health` with a short timeout. If that succeeds, narrow the check to the affected index and read its settings or mapping. Health reports shard allocation: green means primaries and replicas are allocated, yellow means some replicas are unassigned, and red means at least one primary is unassigned. It does not validate query answers. See the [native Cluster Health API](https://docs.opensearch.org/latest/api-reference/cluster-api/cluster-health/).

For a target that actually uses HTTP basic authentication, this optional Python example prints only a small health summary. Install `requests` first. Use the authenticated origin-facing endpoint for an Elastic-family mapping when the profile requires it; use the real signing client instead if the application path requires IAM signing.

```python theme={null}
import getpass
import requests

base = input("Configured HTTPS search endpoint: ").rstrip("/")
ca_bundle = input("Trusted CA bundle path: ")
with requests.Session() as session:
    session.auth = (input("Username: "), getpass.getpass("Password: "))
    response = session.get(
        base + "/_cluster/health",
        params={"level": "cluster", "timeout": "5s"},
        timeout=10, verify=ca_bundle,
    )
    response.raise_for_status()
    health = response.json()
    print({key: health.get(key) for key in
           ("status", "number_of_nodes", "unassigned_shards", "timed_out")})
```

A permission refusal to a cluster-wide endpoint does not establish that index searches are broken. Repeat the smallest authorized read for the affected index using the application's identity. Preserve TLS verification and distinguish authentication from authorization instead of changing credentials, network rules and index settings together.

## Compare a native OpenSearch cluster with Elastic Cloud

These are illustrative investigations for two catalog mappings. Use the application's selected backend, engine/client versions, index or alias and configured authentication route. The following HTTP paths are read-only requests to send through that authenticated client; they are not unauthenticated curl instructions.

### Kubernetes OpenSearch: explain an unassigned replica

For **AWS OpenSearch → OpenSearch on Kubernetes**, search traffic goes directly to the native cluster. Suppose the bounded health check reports yellow, the affected index's primaries are allocated, and one replica is unassigned. Narrow the diagnostic to that index, then inspect allocation:

```http theme={null}
GET /products/_settings?flat_settings=true
GET /_cluster/allocation/explain
```

Replace `products` with the affected permitted index. The [native allocation API](https://docs.opensearch.org/latest/api-reference/cluster-api/cluster-allocation/) chooses an unassigned shard when no body is supplied; confirm the returned index and shard before using its explanation for this incident. Its `decider`, `decision` and `explanation` identify the applicable allocation constraints.

Illustrative result: the only eligible data node already holds the primary, so the replica cannot be assigned there. Compare replica count with eligible nodes, failure-domain placement and storage availability. This result calls for a topology or allocation review, not a credential reset or re-import. Yellow health does not by itself prove that product searches failed. If the allocation explanation concerns a different index, request the affected shard explicitly through the native API rather than applying its conclusion to `products`.

### Elastic Cloud: identify which permission or API was refused

For **AWS OpenSearch → Elastic Cloud Hosted**, the application uses the origin-facing translator, which verifies AWS SigV4; backend calls use Elastic credentials or API keys. It is not the same direct OpenSearch route as the Kubernetes example. Suppose the same application's index metadata read succeeds but another operation returns a refusal:

```http theme={null}
GET /products/_mapping
GET /products/_settings?flat_settings=true
```

These successful reads establish only their own index access and supported translation. Preserve the failing operation's exact path and error type. For a supported index/search operation refused by Elastic, ask the backend owner to inspect the configured Elastic identity's privileges. A valid origin IAM signature does not grant an Elastic API key extra rights.

If the failed path instead belongs to OpenSearch PPL or the OpenSearch Security API, the catalog mapping excludes it. Broader credentials will not add that API. Elasticsearch compatibility headers address selected REST-version differences, as described in [Elastic's compatibility documentation](https://www.elastic.co/docs/reference/elasticsearch/rest-apis/compatibility); they do not install OpenSearch plugins. Compare the required feature with this directed profile before changing the application or target. Keep native allocation explanations, query profiles and Tensor9 reports distinct; these native reads do not promise a Tensor9 explanation receipt.

## Worked case: an import succeeds but products are missing

A nightly product import receives HTTP responses, yet several products cannot be found. First inspect each bulk item's status and error. Suppose some items were rejected during a concurrency spike while other items succeeded. Compare rejected work with CPU, memory pressure and shard distribution before adding retries. AWS documents these causes in its [OpenSearch troubleshooting guide](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/handling-errors.html); inspect equivalent native metrics on other targets.

Reduce or pause additional import concurrency while preserving the failed-item record. Retry only the identified failures under the application's idempotency policy. For successfully indexed documents, compare direct document reads with searches against the exact alias, routing value and refresh interval. An accepted write and immediate search visibility are separate observations.

If the issue instead follows an engine change, compare mappings, analyzers, plugin endpoints and client response parsing with the directed profile. OpenSearch `_plugins` APIs do not become universally compatible with Elasticsearch by changing a hostname. Follow [OpenSearch tuning](https://docs.tensor9.com/cloud-adapter/tuning/aws-opensearch.md) for measured shard, refresh, bulk-concurrency and capacity changes after establishing compatibility.

## Explain is separate from native search diagnostics

Use Tensor9 Explain only where the installed adapter version and the exact operation advertise support. Native search traffic does not automatically produce Tensor9 receipts. OpenSearch's own query-explanation or profiling APIs describe engine behavior; they are separate from Tensor9's explanation of an adapted request and can have their own cost and permission requirements.

When an admitted operation supplies `x-t9-explain-id`, retrieve that existing report through the install's synthetic AWS `t9` service with the original owner's IAM credentials and SigV4 signing name `t9`. Preserve the report's revision and assurance. Reads do not rerun indexing or search. The default retention is 24 hours from creation; the published expiry is fixed and reads do not renew it. Check [installed protocol support](https://docs.tensor9.com/cloud-adapter/debugging/explain-sdk.md) before using retrieval examples.

[PostgreSQL query plans](https://docs.tensor9.com/cloud-adapter/debugging/postgres-query-plans.md) remain a planned integration with operation-specific qualification. A PostgreSQL control store does not turn a Lucene/OpenSearch query into PostgreSQL SQL. Where qualified, full SQL plan JSON can contain customer constants, predicates and relation names and is restricted to the exact report owner.

For support, collect redacted configuration, versions, selected profile, endpoint route, index/alias, timestamps, native status/error, a small failed-item summary and relevant provider or cluster events. Attach an existing receipt and expiry when available. Exclude credentials, document bodies and unrestricted query logs.

A [Support Agent](https://docs.tensor9.com/cloud-adapter/support/agents.md) can help separate translation coverage, backend permissions and shard evidence before recommending a cluster change.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.