SageMaker (Inference)

On this page

• Coverage by target cloud
• How the targets compare
  • Max adaptation
• On Google Cloud and Azure

​

Coverage by target cloud

​

How the targets compare

Each row compares a capability of SageMaker (Inference) with its adaptation on each target. A dash means this row is not stated for that target.

​

Max adaptation

​

On Google Cloud and Azure

​

How inference runs

Your application sends a synchronous InvokeEndpoint request through its SageMaker Runtime client. The service adapter forwards the payload to your inference container in the customer’s Kubernetes cluster on Azure or Google Cloud, then returns the container’s response. The model and its input format stay yours; this mapping does not substitute Vertex AI, Azure Machine Learning or a different model. Each configured endpoint addresses one model container. Requests naming a different endpoint, a production variant or a multi-model target do not silently run against the default model. Use one model and one endpoint per compiled module for this subset. Configure the client’s endpoint name to match the deployed endpoint; custom AWS endpoint aliases are not preserved automatically.

​

Prepare the model and serving capacity

Tensor9 maps the origin stack’s model, endpoint configuration and endpoint resources to a Kubernetes Deployment and Service using the model’s container image. The endpoint is the served resource; its model and endpoint configuration accompany that provisioning. They do not provide separate model-management or endpoint-management APIs. A model or configuration without an endpoint does not create an inference server. Keep the three resources in one module with one declared model image. Make that image pullable from the customer’s environment and provide the model files it needs. The container must start its inference server itself. This path does not append SageMaker’s serve argument or copy the model’s environment-variable map automatically; package or configure the required startup behavior and environment explicitly. Unlike SageMaker hosting, this path does not unpack ModelDataUrl into /opt/ml/model before startup. The artifact URL is passed to the container; the container must fetch it with suitable credentials, or you must supply the files another way. An image expecting preinstalled model files needs adjustment. AWS instance types, initial instance counts and production-variant weights do not configure equivalent capacity here. Size CPU, memory and any accelerators for the customer’s cluster, and validate startup, readiness and inference latency with the actual model. AWS execution roles and endpoint KMS keys are not applied unchanged; configure target identity and storage encryption.

​

Inference-only boundaries

This adapter covers synchronous requests to a provisioned model. It does not cover asynchronous or streaming inference, batch transform, weighted variants, multi-model routing or runtime endpoint management. Changes to serving resources go through the origin stack rather than SageMaker management calls. Notebooks, training and hyperparameter tuning remain on AWS. The SageMaker AI (Training and Notebooks) entry describes that separate scope. Inference is part of SageMaker AI too; the two index entries distinguish the capabilities Tensor9 adapts, not two unrelated AWS products.

​

AWS reference

AWS documents the inference container contract and the InvokeEndpoint API. Those describe AWS behavior; the operation table above defines this adapter’s narrower scope. Service Catalog.