> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tensor9.com/llms.txt
> Use this file to discover all available pages before exploring further.

# SageMaker (Inference)

> AWS SageMaker (Inference). Real-time model inference through SageMaker Runtime endpoints. This entry covers inference only, not notebooks, training jobs or the wider SageMaker platform.

**On this page**

* [Coverage by target cloud](#coverage-by-target-cloud)
* [How the targets compare](#how-the-targets-compare)
  * [Max adaptation](#max-adaptation)
* [On Google Cloud and Azure](#on-google-cloud-and-azure)

## Coverage by target cloud

| Target             | Available |
| ------------------ | --------- |
| Google Cloud       | ✓         |
| Azure              | ✓         |
| OCI                | -         |
| Private Kubernetes | -         |

## How the targets compare

Each row compares a capability of SageMaker (Inference) with its adaptation on each target.
A dash means this row is not stated for that target.

### Max adaptation

| Capability   | SageMaker (Inference) | Google Cloud and Azure |
| ------------ | --------------------- | ---------------------- |
| API coverage | full                  | partial                |

## On Google Cloud and Azure

| Operation                                                                             | Area               | Support      | Depth        | Notes                                                                                                                                                         |
| ------------------------------------------------------------------------------------- | ------------------ | ------------ | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| InvokeEndpoint                                                                        | Inference          | Supported    | Common       | Forwards a request body of up to 6 MiB to one configured model container and returns its response body and content type.                                      |
| InvokeEndpointAsync                                                                   | Inference          | Out of scope | Full surface | Asynchronous inference is unsupported; requests return an error.                                                                                              |
| InvokeEndpointWithResponseStream / InvokeEndpointWithBidirectionalStream              | Inference          | Out of scope | Full surface | Streaming inference is outside this adapter's subset.                                                                                                         |
| ContentType, Accept, CustomAttributes and InferenceId                                 | Inference headers  | Supported    | Common       | These values reach the model container. CustomAttributes in its response are returned to the caller.                                                          |
| Explanation, inference-component, session and prefix-cache controls                   | Inference options  | Out of scope | Full surface | These request controls are not implemented or forwarded to the model. Do not rely on them taking effect.                                                      |
| Batch transform, training jobs, tuning and notebooks                                  | ML platform        | Out of scope | Full surface | These are separate workloads, not synchronous endpoint inference; they remain on AWS.                                                                         |
| TargetVariant / TargetModel / TargetContainerHostname                                 | Model routing      | Out of scope | Full surface | Explicit variant, multi-model and inference-pipeline container selection are refused. Each configured endpoint serves one model container.                    |
| Model, endpoint configuration and endpoint Terraform resources                        | Provisioning       | Partial      | Most usage   | Deploys the model image as a Kubernetes Deployment and Service. Model artifacts, AWS instance sizing and weighted production variants need separate handling. |
| CreateModel / CreateEndpointConfig / CreateEndpoint / UpdateEndpoint / DeleteEndpoint | Runtime management | Out of scope | Full surface | Provision serving resources through the origin stack. Live SageMaker management API calls are not covered by this inference adapter.                          |

#### How inference runs

Your application sends a synchronous `InvokeEndpoint` request through its SageMaker Runtime client. The service adapter forwards the payload to your inference container in the customer's Kubernetes cluster on Azure or Google Cloud, then returns the container's response. The model and its input format stay yours; this mapping does not substitute Vertex AI, Azure Machine Learning or a different model.

Each configured endpoint addresses one model container. Requests naming a different endpoint, a production variant or a multi-model target do not silently run against the default model. Use one model and one endpoint per compiled module for this subset. Configure the client's endpoint name to match the deployed endpoint; custom AWS endpoint aliases are not preserved automatically.

#### Prepare the model and serving capacity

Tensor9 maps the origin stack's model, endpoint configuration and endpoint resources to a Kubernetes Deployment and Service using the model's container image. The endpoint is the served resource; its model and endpoint configuration accompany that provisioning. They do not provide separate model-management or endpoint-management APIs. A model or configuration without an endpoint does not create an inference server. Keep the three resources in one module with one declared model image. Make that image pullable from the customer's environment and provide the model files it needs.

The container must start its inference server itself. This path does not append SageMaker's `serve` argument or copy the model's environment-variable map automatically; package or configure the required startup behavior and environment explicitly.

Unlike SageMaker hosting, this path does not unpack `ModelDataUrl` into `/opt/ml/model` before startup. The artifact URL is passed to the container; the container must fetch it with suitable credentials, or you must supply the files another way. An image expecting preinstalled model files needs adjustment.

AWS instance types, initial instance counts and production-variant weights do not configure equivalent capacity here. Size CPU, memory and any accelerators for the customer's cluster, and validate startup, readiness and inference latency with the actual model. AWS execution roles and endpoint KMS keys are not applied unchanged; configure target identity and storage encryption.

#### Inference-only boundaries

This adapter covers synchronous requests to a provisioned model. It does not cover asynchronous or streaming inference, batch transform, weighted variants, multi-model routing or runtime endpoint management. Changes to serving resources go through the origin stack rather than SageMaker management calls.

Notebooks, training and hyperparameter tuning remain on AWS. The [SageMaker AI (Training and Notebooks)](/service-adapters/aws/available-on-aws-only/sagemaker-ai-training-and-notebooks) entry describes that separate scope. Inference is part of SageMaker AI too; the two index entries distinguish the capabilities Tensor9 adapts, not two unrelated AWS products.

#### AWS reference

AWS documents the [inference container contract](https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-inference-code.html) and the [InvokeEndpoint API](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html). Those describe AWS behavior; the operation table above defines this adapter's narrower scope.

[Service Catalog](/service-adapters/catalog).