- Coverage by target cloud
- How the targets compare
- On Google Cloud
- On Google Cloud, Azure, OCI, and Private Kubernetes
- On Azure
- On OCI
- Service level agreements
- Bedrock (LLM inference) to Anthropic Messages API
- Bedrock (LLM inference) to Azure RAI Content Filter Policy
- Bedrock (LLM inference) to Vertex AI - Claude
- Bedrock (LLM inference) to Vertex AI - Gemini
- Bedrock (LLM inference) to Vertex AI - Llama
- Bedrock (LLM inference) to OCI Generative AI
- Bedrock (LLM inference) to OpenAI Chat Completions
- Bedrock (LLM inference) to Scaleway Managed Inference
- Bedrock (LLM inference) to vLLM / Ollama
- Debug this service
Coverage by target cloud
How the targets compare
Each row compares a capability of Bedrock (LLM inference) with its adaptation on each target. A dash means this row is not stated for that target.Max adaptation
On Google Cloud
Via Vertex AI - Gemini
How it works
Tensor9 runs an adapter alongside your application and points the AWS SDK’s Bedrock endpoint to it. The adapter translates Bedrock requests for Google Vertex AI Gemini. Your application keeps its code and SDK, including Bedrock request formats, streaming events, and error codes. Gemini runs in the customer’s Google Cloud project. The adapter authenticates with the appliance’s Google Cloud identity; it needs no API key.
The SDK calls the Tensor9 adapter, which sends requests to the configured model service.
How a conversation maps
A Bedrock chat request is a list of turns, an optional system instruction, and inference settings. The adapter maps all of it across: every turn, the system instruction, and the token limit, temperature, top-p, top-k, and stop settings are preserved. Gemini’s request and response shape differs from Bedrock’s more than most targets do. The service adapter absorbs those differences so your application still sees a normal Bedrock conversation. The model your application names is not necessarily the model that runs; the customer chooses which Gemini model serves the traffic when the appliance is deployed. Because a Gemini model produces every response, the generated content (its wording, its reasoning, and its model-specific strengths) is Gemini’s, and will differ from the Bedrock model your application named. The Bedrock interface around it stays identical: the request shapes, the streaming events, and the error codes are unchanged.
The adapter sends the conversation to the model selected at deployment. Supported inference settings are listed in the text.
Tool use and streaming
If your application uses Bedrock tool use (letting the model call functions you declare), the service adapter supports the full round trip: the model can request a tool call, your application runs it, and the result feeds back into the next turn. Gemini identifies a tool call by the function’s name rather than by a call id, so the adapter keeps the calls straight when the model calls the same tool more than once in a turn. Streaming behaves the same as on Bedrock. As the model produces output, the adapter delivers the same incremental events your SDK already expects, including tool calls.
The application executes tool calls and returns their results. During streaming, the adapter sends these as Bedrock events.
Limitations
△ Where Bedrock and Vertex Gemini differ- Reasoning uses part of the token budget. Gemini 2.5 spends part of the token limit on internal reasoning before it produces visible text, so allow enough tokens for both reasoning and visible output.
- The organization must allow access to generative models. Some Google Cloud organizations allow generative models only through a dedicated service identity, not an interactive user login. An appliance uses its own service identity, which must have the required access before deployment.
- Text inputs only. Image and document content are not served on this path.
- Managed Bedrock features are not served. Agents, Knowledge Bases, and guardrails configured as a service return a clear error rather than a silent or partial result.
Other considerations
Migration. Bedrock inference stores no conversation state in the service. Changing the model service requires configuration changes, with no stored inference data to transfer. Operations. The customer owns the Vertex AI service in their Google Cloud: which models are enabled, the quotas, and the regions. Capacity and cost. Token pricing and throughput are Vertex AI’s, not Bedrock’s.Via Vertex AI - Claude
How it works
Tensor9 runs an adapter alongside your application and points the AWS SDK’s Bedrock endpoint to it. The adapter translates Bedrock requests for Anthropic’s Claude on Google Vertex AI. Your application keeps its code and SDK, including Bedrock request formats, streaming events, and error codes. This runs Claude inside Google Cloud, with no API key: the appliance authenticates with its Google Cloud identity. Bedrock and Claude use similar chat APIs, so the adapter preserves the conversation and settings described below. Prompts stay in Google Cloud.
The SDK calls the Tensor9 adapter, which sends requests to the configured model service.
How a conversation maps
A Bedrock chat request is a list of turns, an optional system instruction, and inference settings. Because the two chat APIs share a shape, the adapter forwards all of it almost unchanged: every turn, the system instruction, and the token limit, temperature, top-p, stop, and top-k settings are preserved. The model your application names is not necessarily the model that runs. The customer chooses which Claude model on Vertex serves the traffic when the appliance is deployed, and the adapter routes every request to it.
The adapter sends the conversation to the model selected at deployment. Supported inference settings are listed in the text.
Tool use and streaming
If your application uses Bedrock tool use (letting the model call functions you declare), the service adapter handles the full round trip: the model can request a tool call, your application runs it, and the result feeds back into the next turn. Because the two chat APIs represent tool calls the same way, this maps directly. Streaming behaves the same as on Bedrock. As the model produces output, the adapter delivers the same incremental events your SDK already expects, including partial tool calls.
The application executes tool calls and returns their results. During streaming, the adapter sends these as Bedrock events.
Limitations
△ Where Bedrock and Claude on Vertex differ- Claude must be enabled in the project. Claude on Vertex AI is offered in specific Google Cloud regions and must be turned on for the project before it can serve traffic. This is a one-time setup when the appliance is deployed.
- Text inputs only. Image and document content are not served on this path.
- Managed Bedrock features are not served. Agents, Knowledge Bases, and guardrails configured as a service return a clear error rather than a silent or partial result.
Other considerations
Migration. Bedrock inference stores no conversation state in the service. Changing the model service requires configuration changes, with no stored inference data to transfer. Operations. The customer owns the Vertex AI service in their Google Cloud, including enabling Claude, the regions, and the quotas. Capacity and cost. Token pricing and throughput are Vertex AI’s, not Bedrock’s.Via Vertex AI - Llama
How it works
Tensor9 runs an adapter alongside your application and points the AWS SDK’s Bedrock endpoint to it. The adapter translates Bedrock requests for open models on Google Vertex AI, including Llama. Your application keeps its code and SDK, including Bedrock request formats, streaming events, and error codes. This runs in Google Cloud, with no API key: the appliance authenticates with its Google Cloud identity. Vertex offers these open models through an OpenAI-style interface, so the adapter maps the conversation the same way it does for any OpenAI-compatible service.
The SDK calls the Tensor9 adapter, which sends requests to the configured model service.
How a conversation maps
A Bedrock chat request is a list of turns, an optional system instruction, and inference settings. The adapter preserves every turn, the system instruction, and the token limit, temperature, top-p, and stop settings. One Bedrock setting, top-k, has no equivalent in the OpenAI-style interface these models use, so it is dropped rather than treated as an error. The model your application names is not necessarily the model that runs. The customer chooses which Vertex open model serves the traffic when the appliance is deployed.
The adapter sends the conversation to the model selected at deployment. Supported inference settings are listed in the text.
Tool use and streaming
If your application uses Bedrock tool use (letting the model call functions you declare), the service adapter handles the full round trip where the served model supports it: the model can request a tool call, your application runs it, and the result feeds back into the next turn. The adapter translates between the Bedrock and OpenAI-style representations of tool calls so your application does not have to. Streaming behaves the same as on Bedrock. As the model produces output, the adapter delivers the same incremental events your SDK already expects.
The application executes tool calls and returns their results. During streaming, the adapter sends these as Bedrock events.
Limitations
△ Where Bedrock and Vertex open models differ- The top-k setting is dropped. The OpenAI-style interface has no equivalent, so a request that sets it still succeeds, without that control applied.
- Capabilities vary by model. Tool use is available only for models that support it; a plain chat model serves text conversations without it.
- Model availability is set by Vertex. Which open models are offered, and in which regions, is decided by Google Vertex AI. The customer picks one that is available where the appliance runs.
- Text inputs only. Image and document content are not served on this path.
- Managed Bedrock features are not served. Agents, Knowledge Bases, and guardrails configured as a service return a clear error rather than a silent or partial result.
Other considerations
Migration. Bedrock inference stores no conversation state in the service. Changing the model service requires configuration changes, with no stored inference data to transfer. Operations. The customer owns the Vertex AI service in their Google Cloud, including which models are enabled and the quotas. Capacity and cost. Token pricing and throughput are Vertex AI’s, not Bedrock’s.On Google Cloud, Azure, OCI, and Private Kubernetes
Via vLLM / Ollama
How it works
Tensor9 runs an adapter alongside your application and points the AWS SDK’s Bedrock endpoint to it. The adapter translates Bedrock requests for a model running in the customer’s cluster. Your application keeps its code and SDK, including Bedrock request formats, streaming events, and error codes. The model runs on the customer’s hardware, served by a standard open model server in their Kubernetes cluster. There is no external model service, no API key, and no prompt or response ever leaves the cluster. The server speaks an OpenAI-style interface, so the adapter maps the conversation the same way it does for any OpenAI-compatible service.
The SDK calls the Tensor9 adapter, which sends requests to the configured model service.
How a conversation maps
A Bedrock chat request is a list of turns, an optional system instruction, and inference settings. The adapter preserves every turn, the system instruction, and the token limit, temperature, top-p, and stop settings. One Bedrock setting, top-k, has no equivalent in the OpenAI-style interface, so it is dropped rather than treated as an error. The model your application names is not necessarily the model that runs. The customer chooses which model the in-cluster server hosts when the appliance is deployed, and the adapter routes every request to it.
The adapter sends the conversation to the model selected at deployment. Supported inference settings are listed in the text.
Tool use and streaming
If your application uses Bedrock tool use (letting the model call functions you declare), the service adapter handles the full round trip where the served model supports it: the model can request a tool call, your application runs it, and the result feeds back into the next turn. The adapter translates between the Bedrock and OpenAI-style representations of tool calls so your application does not have to. Streaming behaves the same as on Bedrock. As the model produces output, the adapter delivers the same incremental events your SDK already expects.
The application executes tool calls and returns their results. During streaming, the adapter sends these as Bedrock events.
Limitations
△ Where Bedrock and a self-hosted model differ- The top-k setting is dropped. The OpenAI-style interface has no equivalent, so a request that sets it still succeeds, without that control applied.
- Capabilities depend on the model you run. Tool use, context length, and quality are set by the open model the customer hosts, not by Bedrock. Choose a model that supports what your application needs.
- Capacity is the customer’s to provide. Throughput is bounded by the GPUs the customer allocates to the server; there is no elastic managed backend behind it.
- Text inputs only. Image and document content are not served on this path.
- Managed Bedrock features are not served. Agents, Knowledge Bases, and guardrails configured as a service return a clear error rather than a silent or partial result.
Other considerations
Migration. Bedrock inference stores no conversation state in the service. Changing the model service requires configuration changes, with no stored inference data to transfer. Operations. The customer runs the model server: choosing the model, provisioning the GPUs, and keeping it available. Tensor9 injects the adapter and points it at that server. Capacity and cost. Throughput and cost are the customer’s own hardware, sized for the workload.On Azure
Via Azure RAI Content Filter Policy
Inference through an Azure model deployment
The Max inference path uses the Tensor9 OpenAI-compatible backend to serve supported Bedrock text-chat requests from an Azure model deployment. Configure the deployment endpoint, API version and API-key or Entra bearer authentication. The backend translates messages, tool calls, stop reasons and streaming fragments between Bedrock and Chat Completions. The application’s Bedrock SDK keeps its supported request and response formats. Model behavior still differs: test prompts, tool schemas, refusals and partial-stream failures against the selected model. The OpenAI path has notop_k field; that setting is omitted. Native model-body support remains limited to the body formats listed in the inference profile.
Content filtering and retrieval
A declared guardrail maps to an Azure responsible-AI content policy on the cognitive account. Attach the policy to the deployment that serves requests. Azure categories and thresholds differ from Bedrock’s, so compare allow/block outcomes using representative content. A knowledge base maps to Azure AI Search infrastructure. Load the corpus, configure ingestion, chunking and embeddings, and connect retrieval results to model requests. Creating the search service does not populate an index or provide the Bedrock Retrieve API.Deployment and validation
Configure the model endpoint and its credentials separately from the content policy and search service. Check network access, model deployment availability and permissions for inference and retrieval. Test the complete application path, including a denied request, a tool call and cancellation of a stream. Managed Bedrock agents are outside this mapping. An application that depends on an agent runtime needs an explicit target implementation; neither a content policy nor a search index supplies that runtime.On OCI
Via OCI Generative AI
How it works
Tensor9 runs an adapter alongside your application and points the AWS SDK’s Bedrock endpoint to it. The adapter translates Bedrock requests for OCI Generative AI. Your application keeps its code and SDK, including Bedrock request formats, streaming events, and error codes. OCI Generative AI hosts two families of chat models: an OpenAI-style family (Llama, Grok, Gemini, and the gpt-oss models) and the Cohere Command family. Each family expects a slightly different request shape. The adapter picks the right shape for whichever model the customer has configured, passes the conversation through, and translates the reply back into a normal Bedrock response.
The SDK calls the Tensor9 adapter, which sends requests to the configured model service.
How a conversation maps
A Bedrock chat request is a list of turns, an optional system instruction, and inference settings. The adapter forwards all of it: every turn, the system instruction, and the token limit, temperature, top-p, top-k, and stop settings are preserved, so the model sees the same conversation it would on AWS. The model your application names in the request is not necessarily the model that runs. The customer chooses which OCI model serves the traffic when the appliance is deployed, and the adapter routes every request to it. Authentication is keyless. The adapter signs each request with the appliance’s own OCI identity, so there is no API key or static secret to manage. The account that is billed is fixed when the appliance is deployed and cannot be changed by a request.
The adapter sends the conversation to the model selected at deployment. Supported inference settings are listed in the text.
Tool use and streaming
If your application uses Bedrock tool use (letting the model call functions you declare), the service adapter handles the full round trip for the OpenAI-style model family: the model can request a tool call, your application runs it, and the result feeds back into the next turn. Streaming behaves the same as on Bedrock. As the model produces output, the adapter delivers the same incremental events your SDK already expects, including partial tool calls. If the OCI model service reports an error partway through a stream, the adapter surfaces it as a normal error rather than a truncated response.
The application executes tool calls and returns their results. During streaming, the adapter sends these as Bedrock events.
Limitations
△ Where Bedrock and OCI Generative AI differ- Tool use is limited to the OpenAI-style family. Tool use with the Cohere Command family is not supported.
- Text inputs only. Image and document inputs are not supported; the adapter serves text conversations and tool use.
- Model availability is region-specific. The Cohere Command family is offered on demand in a different OCI region than the OpenAI-style family, and a given model must be currently offered on demand in the appliance’s region. Choose a model that is available where the appliance runs.
- Managed Bedrock features are not served. Agents, Knowledge Bases, and guardrails configured as a service return a clear error rather than a silent or partial result.
Other considerations
Migration. Bedrock inference stores no conversation state in the service. Changing the model service requires configuration changes, with no stored inference data to transfer. Operations. The customer owns the OCI Generative AI service: which models are enabled, the quotas, and the availability in each region. Capacity and cost. Token pricing and throughput are the OCI model’s, not Bedrock’s. Size the model choice for the workload the same way you would size any inference deployment.Service level agreements
Listed terms cover requests sent through this service adapter. They do not cover direct connections to the target service or replace the cloud provider’s own SLA. Each origin-to-target pair has its own terms or an explicit pending status. See the SLA tables, measurement rules, and scaling conditions.Bedrock (LLM inference) to Anthropic Messages API
What Tensor9 covers. These service levels cover the adapter between the origin API and target API. They do not replace the target provider’s SLA.
How latency is counted. Adapter work, including metadata and coordination, counts. Only separately measured permitted target waits are excluded.
Which terms apply. Your signed agreement names the covered operations, workload limits, remedies, and final service levels for your deployment.
What this adapter does
The adapter validates model identity and request shape, authorizes invocation and translates the AWS runtime envelope to anthropic::1.0.0::messages. Model queueing and inference execution are external spans; stream framing, cancellation, backpressure, token-count conversion and final response/error mapping are adapter work.How the latency targets were chosen
Unary model request translation: 5 ms monthly p99.Converse, CountTokens, InvokeModel. Authorization, model routing and bounded JSON/body conversion.
Streaming model first-chunk translation: 8 ms monthly p99. ConverseStream, InvokeModelWithResponseStream. Admission and conversion through the first bounded response chunk; later chunks are throughput-qualified separately.
Your signed agreement sets the terms for your deployment. A target does not add an operation or option that the compatibility tables mark unavailable.
What counts toward latency
For a request-response row, measure from the agreed ingress boundary to dispatch of the complete response. A row that explicitly names a first response chunk ends at that chunk; its number does not cover the rest of the stream. A long-poll row names the intentional wait and when adapter delay starts. Include parsing, authorization, admission, translation, serialization, adapter-owned storage and coordination, retries and response handling. Subtract only separately measured target-workload waits and external network segments allowed by the measurement rules. The adapter’s own response handling and dispatch remain covered. A database used for adapter metadata is still adapter work, even if a cloud provider hosts it. For concurrent calls, exclude the union of permitted wait intervals, not the sum of overlapping spans. Calculate each request’s adapter duration first, then the monthly p99. Do not subtract one service’s p99 from another’s. Known adapter timeouts are over-budget samples; failed or incomplete requests cannot disappear to improve the percentile. Missing measurements do not become zero latency. An SDK call span alone does not prove how much of its duration can be excluded.Availability and failures
The 99.9% request target measures correct adapter handling, not the percentage of application calls that return success. Correctly forwarding a target quota or permission error is different from producing that error because the adapter sent the wrong request. Adapter-caused failures count even when the target is healthy. With 1,000,000 eligible calls in a month, a 99.9% target permits at most 1,000 adapter-attributable failures. Endpoint probes have their own denominator. Correctness defects remain actionable even when the monthly availability percentage is met.Scaling and target-service capacity
Admission scales across independent requests while per-stream memory and outbound bytes are bounded. Qualification fixes request and response bytes, model, streaming chunk rate and concurrent in-flight streams. Overload must reject before unbounded buffering and one slow stream must not pin unrelated unary calls. Tell Tensor9 the expected steady rate, bursts, concurrency, payload sizes and operation mix. Tensor9 sizes and scales the adapter for the agreed load; you choose and monitor the target service’s capacity with Tensor9’s help. A latency budget is not a requests-per-second rating. Larger requests and higher rates need explicit terms, not silent inheritance of a small-request SLA.Data and behavior guarantees
Payload bytes and content types are preserved, streaming chunks remain ordered, cancellation stops adapter-owned work, and provider safety or model errors retain a typed failure rather than becoming a fabricated success. CountTokens never implies an inference call.Example and diagnosis
InvokeModelWithResponseStream sends a bounded prompt and receives an ordered stream. The SLA meter removes only separately observed model execution/wait time; JSON conversion before the call, chunk framing after it and stalled-client backpressure stay in the adapter measurement. Usetensor9 explain and the documented explain headers to understand the selected adapter and its behavior. Correlate available request diagnostics with the target provider’s latency, throttling and capacity metrics. An explain report helps diagnose a request; it is not by itself a qualified SLA timing measurement. Share the operation, request shape, timestamps and request identifiers with support, with credentials and customer payloads removed.
Bedrock (LLM inference) to Azure RAI Content Filter Policy
SLA status: Not applicable This mapping provisions an Azure content-filter policy. A policy is not a Bedrock inference endpoint; inference adapters have their own operation-specific terms.Bedrock (LLM inference) to Vertex AI - Claude
What Tensor9 covers. These service levels cover the adapter between the origin API and target API. They do not replace the target provider’s SLA.
How latency is counted. Adapter work, including metadata and coordination, counts. Only separately measured permitted target waits are excluded.
Which terms apply. Your signed agreement names the covered operations, workload limits, remedies, and final service levels for your deployment.
What this adapter does
The adapter validates model identity and request shape, authorizes invocation and translates the AWS runtime envelope to google::1.0.0::vertex::claude. Model queueing and inference execution are external spans; stream framing, cancellation, backpressure, token-count conversion and final response/error mapping are adapter work.How the latency targets were chosen
Unary model request translation: 5 ms monthly p99.Converse, CountTokens, InvokeModel. Authorization, model routing and bounded JSON/body conversion.
Streaming model first-chunk translation: 8 ms monthly p99. ConverseStream, InvokeModelWithResponseStream. Admission and conversion through the first bounded response chunk; later chunks are throughput-qualified separately.
Your signed agreement sets the terms for your deployment. A target does not add an operation or option that the compatibility tables mark unavailable.
What counts toward latency
For a request-response row, measure from the agreed ingress boundary to dispatch of the complete response. A row that explicitly names a first response chunk ends at that chunk; its number does not cover the rest of the stream. A long-poll row names the intentional wait and when adapter delay starts. Include parsing, authorization, admission, translation, serialization, adapter-owned storage and coordination, retries and response handling. Subtract only separately measured target-workload waits and external network segments allowed by the measurement rules. The adapter’s own response handling and dispatch remain covered. A database used for adapter metadata is still adapter work, even if a cloud provider hosts it. For concurrent calls, exclude the union of permitted wait intervals, not the sum of overlapping spans. Calculate each request’s adapter duration first, then the monthly p99. Do not subtract one service’s p99 from another’s. Known adapter timeouts are over-budget samples; failed or incomplete requests cannot disappear to improve the percentile. Missing measurements do not become zero latency. An SDK call span alone does not prove how much of its duration can be excluded.Availability and failures
The 99.9% request target measures correct adapter handling, not the percentage of application calls that return success. Correctly forwarding a target quota or permission error is different from producing that error because the adapter sent the wrong request. Adapter-caused failures count even when the target is healthy. With 1,000,000 eligible calls in a month, a 99.9% target permits at most 1,000 adapter-attributable failures. Endpoint probes have their own denominator. Correctness defects remain actionable even when the monthly availability percentage is met.Scaling and target-service capacity
Admission scales across independent requests while per-stream memory and outbound bytes are bounded. Qualification fixes request and response bytes, model, streaming chunk rate and concurrent in-flight streams. Overload must reject before unbounded buffering and one slow stream must not pin unrelated unary calls. Tell Tensor9 the expected steady rate, bursts, concurrency, payload sizes and operation mix. Tensor9 sizes and scales the adapter for the agreed load; you choose and monitor the target service’s capacity with Tensor9’s help. A latency budget is not a requests-per-second rating. Larger requests and higher rates need explicit terms, not silent inheritance of a small-request SLA.Data and behavior guarantees
Payload bytes and content types are preserved, streaming chunks remain ordered, cancellation stops adapter-owned work, and provider safety or model errors retain a typed failure rather than becoming a fabricated success. CountTokens never implies an inference call.Example and diagnosis
InvokeModelWithResponseStream sends a bounded prompt and receives an ordered stream. The SLA meter removes only separately observed model execution/wait time; JSON conversion before the call, chunk framing after it and stalled-client backpressure stay in the adapter measurement. Usetensor9 explain and the documented explain headers to understand the selected adapter and its behavior. Correlate available request diagnostics with the target provider’s latency, throttling and capacity metrics. An explain report helps diagnose a request; it is not by itself a qualified SLA timing measurement. Share the operation, request shape, timestamps and request identifiers with support, with credentials and customer payloads removed.
Bedrock (LLM inference) to Vertex AI - Gemini
What Tensor9 covers. These service levels cover the adapter between the origin API and target API. They do not replace the target provider’s SLA.
How latency is counted. Adapter work, including metadata and coordination, counts. Only separately measured permitted target waits are excluded.
Which terms apply. Your signed agreement names the covered operations, workload limits, remedies, and final service levels for your deployment.
What this adapter does
The adapter validates model identity and request shape, authorizes invocation and translates the AWS runtime envelope to google::1.0.0::vertex::gemini. Model queueing and inference execution are external spans; stream framing, cancellation, backpressure, token-count conversion and final response/error mapping are adapter work.How the latency targets were chosen
Unary model request translation: 5 ms monthly p99.Converse, CountTokens, InvokeModel. Authorization, model routing and bounded JSON/body conversion.
Streaming model first-chunk translation: 8 ms monthly p99. ConverseStream, InvokeModelWithResponseStream. Admission and conversion through the first bounded response chunk; later chunks are throughput-qualified separately.
Your signed agreement sets the terms for your deployment. A target does not add an operation or option that the compatibility tables mark unavailable.
What counts toward latency
For a request-response row, measure from the agreed ingress boundary to dispatch of the complete response. A row that explicitly names a first response chunk ends at that chunk; its number does not cover the rest of the stream. A long-poll row names the intentional wait and when adapter delay starts. Include parsing, authorization, admission, translation, serialization, adapter-owned storage and coordination, retries and response handling. Subtract only separately measured target-workload waits and external network segments allowed by the measurement rules. The adapter’s own response handling and dispatch remain covered. A database used for adapter metadata is still adapter work, even if a cloud provider hosts it. For concurrent calls, exclude the union of permitted wait intervals, not the sum of overlapping spans. Calculate each request’s adapter duration first, then the monthly p99. Do not subtract one service’s p99 from another’s. Known adapter timeouts are over-budget samples; failed or incomplete requests cannot disappear to improve the percentile. Missing measurements do not become zero latency. An SDK call span alone does not prove how much of its duration can be excluded.Availability and failures
The 99.9% request target measures correct adapter handling, not the percentage of application calls that return success. Correctly forwarding a target quota or permission error is different from producing that error because the adapter sent the wrong request. Adapter-caused failures count even when the target is healthy. With 1,000,000 eligible calls in a month, a 99.9% target permits at most 1,000 adapter-attributable failures. Endpoint probes have their own denominator. Correctness defects remain actionable even when the monthly availability percentage is met.Scaling and target-service capacity
Admission scales across independent requests while per-stream memory and outbound bytes are bounded. Qualification fixes request and response bytes, model, streaming chunk rate and concurrent in-flight streams. Overload must reject before unbounded buffering and one slow stream must not pin unrelated unary calls. Tell Tensor9 the expected steady rate, bursts, concurrency, payload sizes and operation mix. Tensor9 sizes and scales the adapter for the agreed load; you choose and monitor the target service’s capacity with Tensor9’s help. A latency budget is not a requests-per-second rating. Larger requests and higher rates need explicit terms, not silent inheritance of a small-request SLA.Data and behavior guarantees
Payload bytes and content types are preserved, streaming chunks remain ordered, cancellation stops adapter-owned work, and provider safety or model errors retain a typed failure rather than becoming a fabricated success. CountTokens never implies an inference call.Example and diagnosis
InvokeModelWithResponseStream sends a bounded prompt and receives an ordered stream. The SLA meter removes only separately observed model execution/wait time; JSON conversion before the call, chunk framing after it and stalled-client backpressure stay in the adapter measurement. Usetensor9 explain and the documented explain headers to understand the selected adapter and its behavior. Correlate available request diagnostics with the target provider’s latency, throttling and capacity metrics. An explain report helps diagnose a request; it is not by itself a qualified SLA timing measurement. Share the operation, request shape, timestamps and request identifiers with support, with credentials and customer payloads removed.
Bedrock (LLM inference) to Vertex AI - Llama
What Tensor9 covers. These service levels cover the adapter between the origin API and target API. They do not replace the target provider’s SLA.
How latency is counted. Adapter work, including metadata and coordination, counts. Only separately measured permitted target waits are excluded.
Which terms apply. Your signed agreement names the covered operations, workload limits, remedies, and final service levels for your deployment.
What this adapter does
The adapter validates model identity and request shape, authorizes invocation and translates the AWS runtime envelope to google::1.0.0::vertex::llama. Model queueing and inference execution are external spans; stream framing, cancellation, backpressure, token-count conversion and final response/error mapping are adapter work.How the latency targets were chosen
Unary model request translation: 5 ms monthly p99.Converse, CountTokens, InvokeModel. Authorization, model routing and bounded JSON/body conversion.
Streaming model first-chunk translation: 8 ms monthly p99. ConverseStream, InvokeModelWithResponseStream. Admission and conversion through the first bounded response chunk; later chunks are throughput-qualified separately.
Your signed agreement sets the terms for your deployment. A target does not add an operation or option that the compatibility tables mark unavailable.
What counts toward latency
For a request-response row, measure from the agreed ingress boundary to dispatch of the complete response. A row that explicitly names a first response chunk ends at that chunk; its number does not cover the rest of the stream. A long-poll row names the intentional wait and when adapter delay starts. Include parsing, authorization, admission, translation, serialization, adapter-owned storage and coordination, retries and response handling. Subtract only separately measured target-workload waits and external network segments allowed by the measurement rules. The adapter’s own response handling and dispatch remain covered. A database used for adapter metadata is still adapter work, even if a cloud provider hosts it. For concurrent calls, exclude the union of permitted wait intervals, not the sum of overlapping spans. Calculate each request’s adapter duration first, then the monthly p99. Do not subtract one service’s p99 from another’s. Known adapter timeouts are over-budget samples; failed or incomplete requests cannot disappear to improve the percentile. Missing measurements do not become zero latency. An SDK call span alone does not prove how much of its duration can be excluded.Availability and failures
The 99.9% request target measures correct adapter handling, not the percentage of application calls that return success. Correctly forwarding a target quota or permission error is different from producing that error because the adapter sent the wrong request. Adapter-caused failures count even when the target is healthy. With 1,000,000 eligible calls in a month, a 99.9% target permits at most 1,000 adapter-attributable failures. Endpoint probes have their own denominator. Correctness defects remain actionable even when the monthly availability percentage is met.Scaling and target-service capacity
Admission scales across independent requests while per-stream memory and outbound bytes are bounded. Qualification fixes request and response bytes, model, streaming chunk rate and concurrent in-flight streams. Overload must reject before unbounded buffering and one slow stream must not pin unrelated unary calls. Tell Tensor9 the expected steady rate, bursts, concurrency, payload sizes and operation mix. Tensor9 sizes and scales the adapter for the agreed load; you choose and monitor the target service’s capacity with Tensor9’s help. A latency budget is not a requests-per-second rating. Larger requests and higher rates need explicit terms, not silent inheritance of a small-request SLA.Data and behavior guarantees
Payload bytes and content types are preserved, streaming chunks remain ordered, cancellation stops adapter-owned work, and provider safety or model errors retain a typed failure rather than becoming a fabricated success. CountTokens never implies an inference call.Example and diagnosis
InvokeModelWithResponseStream sends a bounded prompt and receives an ordered stream. The SLA meter removes only separately observed model execution/wait time; JSON conversion before the call, chunk framing after it and stalled-client backpressure stay in the adapter measurement. Usetensor9 explain and the documented explain headers to understand the selected adapter and its behavior. Correlate available request diagnostics with the target provider’s latency, throttling and capacity metrics. An explain report helps diagnose a request; it is not by itself a qualified SLA timing measurement. Share the operation, request shape, timestamps and request identifiers with support, with credentials and customer payloads removed.
Bedrock (LLM inference) to OCI Generative AI
What Tensor9 covers. These service levels cover the adapter between the origin API and target API. They do not replace the target provider’s SLA.
How latency is counted. Adapter work, including metadata and coordination, counts. Only separately measured permitted target waits are excluded.
Which terms apply. Your signed agreement names the covered operations, workload limits, remedies, and final service levels for your deployment.
What this adapter does
The adapter validates model identity and request shape, authorizes invocation and translates the AWS runtime envelope to oci::1.0.0::genai. Model queueing and inference execution are external spans; stream framing, cancellation, backpressure, token-count conversion and final response/error mapping are adapter work.How the latency targets were chosen
Unary model request translation: 5 ms monthly p99.Converse, CountTokens, InvokeModel. Authorization, model routing and bounded JSON/body conversion.
Streaming model first-chunk translation: 8 ms monthly p99. ConverseStream, InvokeModelWithResponseStream. Admission and conversion through the first bounded response chunk; later chunks are throughput-qualified separately.
Your signed agreement sets the terms for your deployment. A target does not add an operation or option that the compatibility tables mark unavailable.
What counts toward latency
For a request-response row, measure from the agreed ingress boundary to dispatch of the complete response. A row that explicitly names a first response chunk ends at that chunk; its number does not cover the rest of the stream. A long-poll row names the intentional wait and when adapter delay starts. Include parsing, authorization, admission, translation, serialization, adapter-owned storage and coordination, retries and response handling. Subtract only separately measured target-workload waits and external network segments allowed by the measurement rules. The adapter’s own response handling and dispatch remain covered. A database used for adapter metadata is still adapter work, even if a cloud provider hosts it. For concurrent calls, exclude the union of permitted wait intervals, not the sum of overlapping spans. Calculate each request’s adapter duration first, then the monthly p99. Do not subtract one service’s p99 from another’s. Known adapter timeouts are over-budget samples; failed or incomplete requests cannot disappear to improve the percentile. Missing measurements do not become zero latency. An SDK call span alone does not prove how much of its duration can be excluded.Availability and failures
The 99.9% request target measures correct adapter handling, not the percentage of application calls that return success. Correctly forwarding a target quota or permission error is different from producing that error because the adapter sent the wrong request. Adapter-caused failures count even when the target is healthy. With 1,000,000 eligible calls in a month, a 99.9% target permits at most 1,000 adapter-attributable failures. Endpoint probes have their own denominator. Correctness defects remain actionable even when the monthly availability percentage is met.Scaling and target-service capacity
Admission scales across independent requests while per-stream memory and outbound bytes are bounded. Qualification fixes request and response bytes, model, streaming chunk rate and concurrent in-flight streams. Overload must reject before unbounded buffering and one slow stream must not pin unrelated unary calls. Tell Tensor9 the expected steady rate, bursts, concurrency, payload sizes and operation mix. Tensor9 sizes and scales the adapter for the agreed load; you choose and monitor the target service’s capacity with Tensor9’s help. A latency budget is not a requests-per-second rating. Larger requests and higher rates need explicit terms, not silent inheritance of a small-request SLA.Data and behavior guarantees
Payload bytes and content types are preserved, streaming chunks remain ordered, cancellation stops adapter-owned work, and provider safety or model errors retain a typed failure rather than becoming a fabricated success. CountTokens never implies an inference call.Example and diagnosis
InvokeModelWithResponseStream sends a bounded prompt and receives an ordered stream. The SLA meter removes only separately observed model execution/wait time; JSON conversion before the call, chunk framing after it and stalled-client backpressure stay in the adapter measurement. Usetensor9 explain and the documented explain headers to understand the selected adapter and its behavior. Correlate available request diagnostics with the target provider’s latency, throttling and capacity metrics. An explain report helps diagnose a request; it is not by itself a qualified SLA timing measurement. Share the operation, request shape, timestamps and request identifiers with support, with credentials and customer payloads removed.
Bedrock (LLM inference) to OpenAI Chat Completions
What Tensor9 covers. These service levels cover the adapter between the origin API and target API. They do not replace the target provider’s SLA.
How latency is counted. Adapter work, including metadata and coordination, counts. Only separately measured permitted target waits are excluded.
Which terms apply. Your signed agreement names the covered operations, workload limits, remedies, and final service levels for your deployment.
What this adapter does
The adapter validates model identity and request shape, authorizes invocation and translates the AWS runtime envelope to openai::1.0.0::chat. Model queueing and inference execution are external spans; stream framing, cancellation, backpressure, token-count conversion and final response/error mapping are adapter work.How the latency targets were chosen
Unary model request translation: 5 ms monthly p99.Converse, CountTokens, InvokeModel. Authorization, model routing and bounded JSON/body conversion.
Streaming model first-chunk translation: 8 ms monthly p99. ConverseStream, InvokeModelWithResponseStream. Admission and conversion through the first bounded response chunk; later chunks are throughput-qualified separately.
Your signed agreement sets the terms for your deployment. A target does not add an operation or option that the compatibility tables mark unavailable.
What counts toward latency
For a request-response row, measure from the agreed ingress boundary to dispatch of the complete response. A row that explicitly names a first response chunk ends at that chunk; its number does not cover the rest of the stream. A long-poll row names the intentional wait and when adapter delay starts. Include parsing, authorization, admission, translation, serialization, adapter-owned storage and coordination, retries and response handling. Subtract only separately measured target-workload waits and external network segments allowed by the measurement rules. The adapter’s own response handling and dispatch remain covered. A database used for adapter metadata is still adapter work, even if a cloud provider hosts it. For concurrent calls, exclude the union of permitted wait intervals, not the sum of overlapping spans. Calculate each request’s adapter duration first, then the monthly p99. Do not subtract one service’s p99 from another’s. Known adapter timeouts are over-budget samples; failed or incomplete requests cannot disappear to improve the percentile. Missing measurements do not become zero latency. An SDK call span alone does not prove how much of its duration can be excluded.Availability and failures
The 99.9% request target measures correct adapter handling, not the percentage of application calls that return success. Correctly forwarding a target quota or permission error is different from producing that error because the adapter sent the wrong request. Adapter-caused failures count even when the target is healthy. With 1,000,000 eligible calls in a month, a 99.9% target permits at most 1,000 adapter-attributable failures. Endpoint probes have their own denominator. Correctness defects remain actionable even when the monthly availability percentage is met.Scaling and target-service capacity
Admission scales across independent requests while per-stream memory and outbound bytes are bounded. Qualification fixes request and response bytes, model, streaming chunk rate and concurrent in-flight streams. Overload must reject before unbounded buffering and one slow stream must not pin unrelated unary calls. Tell Tensor9 the expected steady rate, bursts, concurrency, payload sizes and operation mix. Tensor9 sizes and scales the adapter for the agreed load; you choose and monitor the target service’s capacity with Tensor9’s help. A latency budget is not a requests-per-second rating. Larger requests and higher rates need explicit terms, not silent inheritance of a small-request SLA.Data and behavior guarantees
Payload bytes and content types are preserved, streaming chunks remain ordered, cancellation stops adapter-owned work, and provider safety or model errors retain a typed failure rather than becoming a fabricated success. CountTokens never implies an inference call.Example and diagnosis
InvokeModelWithResponseStream sends a bounded prompt and receives an ordered stream. The SLA meter removes only separately observed model execution/wait time; JSON conversion before the call, chunk framing after it and stalled-client backpressure stay in the adapter measurement. Usetensor9 explain and the documented explain headers to understand the selected adapter and its behavior. Correlate available request diagnostics with the target provider’s latency, throttling and capacity metrics. An explain report helps diagnose a request; it is not by itself a qualified SLA timing measurement. Share the operation, request shape, timestamps and request identifiers with support, with credentials and customer payloads removed.
Bedrock (LLM inference) to Scaleway Managed Inference
What Tensor9 covers. These service levels cover the adapter between the origin API and target API. They do not replace the target provider’s SLA.
How latency is counted. Adapter work, including metadata and coordination, counts. Only separately measured permitted target waits are excluded.
Which terms apply. Your signed agreement names the covered operations, workload limits, remedies, and final service levels for your deployment.
What this adapter does
The adapter validates model identity and request shape, authorizes invocation and translates the AWS runtime envelope to scaleway::1.0.0::inference. Model queueing and inference execution are external spans; stream framing, cancellation, backpressure, token-count conversion and final response/error mapping are adapter work.How the latency targets were chosen
Unary model request translation: 5 ms monthly p99.Converse, CountTokens, InvokeModel. Authorization, model routing and bounded JSON/body conversion.
Streaming model first-chunk translation: 8 ms monthly p99. ConverseStream, InvokeModelWithResponseStream. Admission and conversion through the first bounded response chunk; later chunks are throughput-qualified separately.
Your signed agreement sets the terms for your deployment. A target does not add an operation or option that the compatibility tables mark unavailable.
What counts toward latency
For a request-response row, measure from the agreed ingress boundary to dispatch of the complete response. A row that explicitly names a first response chunk ends at that chunk; its number does not cover the rest of the stream. A long-poll row names the intentional wait and when adapter delay starts. Include parsing, authorization, admission, translation, serialization, adapter-owned storage and coordination, retries and response handling. Subtract only separately measured target-workload waits and external network segments allowed by the measurement rules. The adapter’s own response handling and dispatch remain covered. A database used for adapter metadata is still adapter work, even if a cloud provider hosts it. For concurrent calls, exclude the union of permitted wait intervals, not the sum of overlapping spans. Calculate each request’s adapter duration first, then the monthly p99. Do not subtract one service’s p99 from another’s. Known adapter timeouts are over-budget samples; failed or incomplete requests cannot disappear to improve the percentile. Missing measurements do not become zero latency. An SDK call span alone does not prove how much of its duration can be excluded.Availability and failures
The 99.9% request target measures correct adapter handling, not the percentage of application calls that return success. Correctly forwarding a target quota or permission error is different from producing that error because the adapter sent the wrong request. Adapter-caused failures count even when the target is healthy. With 1,000,000 eligible calls in a month, a 99.9% target permits at most 1,000 adapter-attributable failures. Endpoint probes have their own denominator. Correctness defects remain actionable even when the monthly availability percentage is met.Scaling and target-service capacity
Admission scales across independent requests while per-stream memory and outbound bytes are bounded. Qualification fixes request and response bytes, model, streaming chunk rate and concurrent in-flight streams. Overload must reject before unbounded buffering and one slow stream must not pin unrelated unary calls. Tell Tensor9 the expected steady rate, bursts, concurrency, payload sizes and operation mix. Tensor9 sizes and scales the adapter for the agreed load; you choose and monitor the target service’s capacity with Tensor9’s help. A latency budget is not a requests-per-second rating. Larger requests and higher rates need explicit terms, not silent inheritance of a small-request SLA.Data and behavior guarantees
Payload bytes and content types are preserved, streaming chunks remain ordered, cancellation stops adapter-owned work, and provider safety or model errors retain a typed failure rather than becoming a fabricated success. CountTokens never implies an inference call.Example and diagnosis
InvokeModelWithResponseStream sends a bounded prompt and receives an ordered stream. The SLA meter removes only separately observed model execution/wait time; JSON conversion before the call, chunk framing after it and stalled-client backpressure stay in the adapter measurement. Usetensor9 explain and the documented explain headers to understand the selected adapter and its behavior. Correlate available request diagnostics with the target provider’s latency, throttling and capacity metrics. An explain report helps diagnose a request; it is not by itself a qualified SLA timing measurement. Share the operation, request shape, timestamps and request identifiers with support, with credentials and customer payloads removed.
Bedrock (LLM inference) to vLLM / Ollama
What Tensor9 covers. These service levels cover the adapter between the origin API and target API. They do not replace the target provider’s SLA.
How latency is counted. Adapter work, including metadata and coordination, counts. Only separately measured permitted target waits are excluded.
Which terms apply. Your signed agreement names the covered operations, workload limits, remedies, and final service levels for your deployment.
What this adapter does
The adapter validates model identity and request shape, authorizes invocation and translates the AWS runtime envelope to tensor9::1.0.0::emu::aws/1.0.0/bedrock::incluster. Model queueing and inference execution are external spans; stream framing, cancellation, backpressure, token-count conversion and final response/error mapping are adapter work.How the latency targets were chosen
Unary model request translation: 5 ms monthly p99.Converse, CountTokens, InvokeModel. Authorization, model routing and bounded JSON/body conversion.
Streaming model first-chunk translation: 8 ms monthly p99. ConverseStream, InvokeModelWithResponseStream. Admission and conversion through the first bounded response chunk; later chunks are throughput-qualified separately.
Your signed agreement sets the terms for your deployment. A target does not add an operation or option that the compatibility tables mark unavailable.
What counts toward latency
For a request-response row, measure from the agreed ingress boundary to dispatch of the complete response. A row that explicitly names a first response chunk ends at that chunk; its number does not cover the rest of the stream. A long-poll row names the intentional wait and when adapter delay starts. Include parsing, authorization, admission, translation, serialization, adapter-owned storage and coordination, retries and response handling. Subtract only separately measured target-workload waits and external network segments allowed by the measurement rules. The adapter’s own response handling and dispatch remain covered. A database used for adapter metadata is still adapter work, even if a cloud provider hosts it. For concurrent calls, exclude the union of permitted wait intervals, not the sum of overlapping spans. Calculate each request’s adapter duration first, then the monthly p99. Do not subtract one service’s p99 from another’s. Known adapter timeouts are over-budget samples; failed or incomplete requests cannot disappear to improve the percentile. Missing measurements do not become zero latency. An SDK call span alone does not prove how much of its duration can be excluded.Availability and failures
The 99.9% request target measures correct adapter handling, not the percentage of application calls that return success. Correctly forwarding a target quota or permission error is different from producing that error because the adapter sent the wrong request. Adapter-caused failures count even when the target is healthy. With 1,000,000 eligible calls in a month, a 99.9% target permits at most 1,000 adapter-attributable failures. Endpoint probes have their own denominator. Correctness defects remain actionable even when the monthly availability percentage is met.Scaling and target-service capacity
Admission scales across independent requests while per-stream memory and outbound bytes are bounded. Qualification fixes request and response bytes, model, streaming chunk rate and concurrent in-flight streams. Overload must reject before unbounded buffering and one slow stream must not pin unrelated unary calls. Tell Tensor9 the expected steady rate, bursts, concurrency, payload sizes and operation mix. Tensor9 sizes and scales the adapter for the agreed load; you choose and monitor the target service’s capacity with Tensor9’s help. A latency budget is not a requests-per-second rating. Larger requests and higher rates need explicit terms, not silent inheritance of a small-request SLA.Data and behavior guarantees
Payload bytes and content types are preserved, streaming chunks remain ordered, cancellation stops adapter-owned work, and provider safety or model errors retain a typed failure rather than becoming a fabricated success. CountTokens never implies an inference call.Example and diagnosis
InvokeModelWithResponseStream sends a bounded prompt and receives an ordered stream. The SLA meter removes only separately observed model execution/wait time; JSON conversion before the call, chunk framing after it and stalled-client backpressure stay in the adapter measurement. Usetensor9 explain and the documented explain headers to understand the selected adapter and its behavior. Correlate available request diagnostics with the target provider’s latency, throttling and capacity metrics. An explain report helps diagnose a request; it is not by itself a qualified SLA timing measurement. Share the operation, request shape, timestamps and request identifiers with support, with credentials and customer payloads removed.