- Coverage by target cloud
- How the backends compare
- On Google Cloud and Private Kubernetes
- On Google Cloud
- On Azure
- On OCI
- On Private Kubernetes
- Debug this service
Coverage by target cloud
How the backends compare
Each row compares a capability of MSK (Kafka) with its adaptation on each backend. A dash means this row is not stated for that backend.Infrastructure-only adaptation
On Google Cloud and Private Kubernetes
Via Strimzi Kafka
How it works
Your producers and consumers keep their Kafka clients and connect directly to Apache Kafka on Kubernetes cluster (via the Strimzi operator) over the Kafka protocol. Tensor9 supplies the bootstrap endpoint; there is no Tensor9 proxy between client and broker. Broker metadata can return further advertised addresses, so clients need network access and TLS trust for those addresses as well as the bootstrap address. Authentication changes with the broker: SASL/SCRAM and mutual TLS are supported. Replace MSK IAM authentication and manage the broker’s access-control lists. Check producer acknowledgement, idempotence and transaction settings against the support table for this broker; a Kafka-compatible endpoint does not guarantee every Kafka administrative or transactional feature.Limitations
- Moving records, offsets and consumer-group state is not supported. A new broker starts without the source records, offsets or consumer groups, and source offsets cannot be applied to a different log. If retained history must move, plan a separate replication process. - You must replace AWS IAM authentication, MSK Connect and Glue Schema Registry separately. Changing the bootstrap servers does not replace those dependencies.
Other considerations
Who runs what. You configure the replication factor and minimum in-sync replicas (ISR), monitor replica health, and operate storage, recovery and upgrades. Topic partitions hold the retained log, and consumer-group offsets record progress. Before cutover. Coordinate producer and consumer cutover. Review topic creation, retention, partition counts and broker configuration on Apache Kafka on Kubernetes cluster (via the Strimzi operator), and set up monitoring, backup or recovery and certificate rotation first. Performance. It depends on the broker, storage and workload; this profile has no Tensor9 broker benchmark.Via Bitnami Kafka
How it works
Your producers and consumers keep their Kafka clients and connect directly to Apache Kafka on Kubernetes cluster (via the Bitnami Kafka chart) over the Kafka protocol. Tensor9 supplies the bootstrap endpoint; there is no Tensor9 proxy between client and broker. Broker metadata can return further advertised addresses, so clients need network access and TLS trust for those addresses as well as the bootstrap address. Authentication changes with the broker: SASL/SCRAM and mutual TLS are supported. Replace MSK IAM authentication and manage the broker’s access-control lists. Check producer acknowledgement, idempotence and transaction settings against the support table for this broker; a Kafka-compatible endpoint does not guarantee every Kafka administrative or transactional feature.Limitations
- Moving records, offsets and consumer-group state is not supported. A new broker starts without the source records, offsets or consumer groups, and source offsets cannot be applied to a different log. If retained history must move, plan a separate replication process. - You must replace AWS IAM authentication, MSK Connect and Glue Schema Registry separately. Changing the bootstrap servers does not replace those dependencies.
Other considerations
Who runs what. You configure the replication factor and minimum in-sync replicas (ISR), monitor replica health, and operate storage, recovery and upgrades. Topic partitions hold the retained log, and consumer-group offsets record progress. Before cutover. Coordinate producer and consumer cutover. Review topic creation, retention, partition counts and broker configuration on Apache Kafka on Kubernetes cluster (via the Bitnami Kafka chart), and set up monitoring, backup or recovery and certificate rotation first. Performance. It depends on the broker, storage and workload; this profile has no Tensor9 broker benchmark.On Google Cloud
Via Managed Service for Apache Kafka
How it works
Your producers and consumers keep their Kafka clients and connect directly to Google Managed Service for Apache Kafka over the Kafka protocol. Tensor9 supplies the bootstrap endpoint; there is no Tensor9 proxy between client and broker. Broker metadata can return further advertised addresses, so clients need network access and TLS trust for those addresses as well as the bootstrap address. Authentication changes with the broker: clients authenticate through Google Cloud IAM over SASL_SSL/OAUTHBEARER; MSK IAM, SASL/SCRAM, and mTLS each need a one-time client-side handler and credential swap. Check producer acknowledgement, idempotence and transaction settings against the support table for this broker; a Kafka-compatible endpoint does not guarantee every Kafka administrative or transactional feature.Limitations
- Moving records, offsets and consumer-group state is not supported. A new broker starts without the source records, offsets or consumer groups, and source offsets cannot be applied to a different log. If retained history must move, plan a separate replication process. - You must replace AWS IAM authentication, MSK Connect and Glue Schema Registry separately. Changing the bootstrap servers does not replace those dependencies.
Other considerations
Who runs what. Google operates durability and HA; capacity is cluster-wide vCPU/memory rather than a broker instance type you pick, and the cluster is reachable only inside your VPC. Topic partitions hold the retained log, and consumer-group offsets record progress. Before cutover. Coordinate producer and consumer cutover. Review topic creation, retention, partition counts and broker configuration on Google Managed Service for Apache Kafka, and set up monitoring, backup or recovery and certificate rotation first. Performance. It depends on the broker, storage and workload; this profile has no Tensor9 broker benchmark.On Azure
How it works
Your Kafka client connects directly to Azure Event Hubs at<namespace>.servicebus.windows.net:9093, the Kafka endpoint of the Standard and Premium tiers. Produce and consume code stay the same; you change the bootstrap address and credentials. Tensor9 does not sit in these connections.
Event Hubs differs from Kafka in administration, transactions, authentication and capacity. Review the differences below before you choose this backend.
Estimating namespace capacity
MSK capacity is declared as a broker count and instance type, with storage and placement per broker. Event Hubs capacity is set on a namespace using throughput units or processing units. Tensor9 estimates namespace capacity from broker count multiplied by vCPUs per broker. Brokers with at least 16 vCPUs each, such askafka.m5.4xlarge, select Premium processing units. Smaller brokers select Standard throughput units. Basic is excluded because it has no Kafka endpoint.
The build reports this conversion as a lossy translation, because machine size does not decide your workload’s message throughput. Load-test the generated capacity and adjust it from measured traffic.
Provision topics as Event Hubs
The generated Terraform creates the namespace. It creates no Event Hubs because an MSK cluster resource does not declare topics: Kafka topics are created by producers or administrators at runtime.aws_msk_configuration contains server.properties settings, not a topic list.
Each Kafka topic needs an Event Hub in the namespace. Create them before producing messages; this backend does not create them on first use.
Configure authentication
Configure clients to use an Event Hubs Shared Access Signature (SAS) policy or Microsoft Entra identity. MSK’s SASL/SCRAM, mutual TLS and IAM configurations do not transfer directly. You configure namespace authentication separately from the generated stack. Credentials are kept in the appliance’s secret store; the generated Terraform contains no SAS key or connection string. Event Hubs requires TLS and encrypts stored data by default, so MSK’sencryption_info is not translated. Configure a customer-managed encryption key separately on Azure if required.
Limitations
Where Event Hubs and MSK differ- Kafka transactions and exactly-once delivery are not fully supported. Producing and consuming over the Kafka protocol work unchanged, but the Event Hubs Kafka endpoint does not offer full Kafka transaction and idempotent-producer behaviour. An application that relies on exactly-once delivery through Kafka transactions cannot use this backend as is.
- Partition counts behave differently: on Standard they are fixed at creation. On Standard, an Event Hub’s partition count cannot change after creation; Premium and Dedicated can increase it but never decrease it, as Kafka does. Size partitions for the consumer parallelism you expect, because on Standard the only way to change it is a new Event Hub.
- Capacity behaves differently: it is bought separately from partitions. On Kafka, partitions set parallelism and much of your throughput headroom. On Event Hubs, throughput units (Standard) or processing units (Premium) are bought separately from partitions, so a partition count copied from MSK does not bring its throughput with it.
- Consumer-group offsets behave differently: there is no
__consumer_offsetstopic. Event Hubs keeps offsets in its own store. Committing and reading offsets through the Kafka client works as usual; tools that read the offsets topic directly, or depend on its retention and compaction, do not work here. - Runtime Kafka administration is only partly supported. Topics are Event Hubs you create up front, and broker administration and ACLs differ from Kafka. Treat code that creates topics or manages ACLs through AdminClient at runtime as needing rework.
- Pinning a Kafka version is not supported. MSK’s
kafka_versionhas no counterpart: Event Hubs sets its own supported Kafka protocol version, and you cannot choose or freeze it. - Custom
server.propertiessettings are not kept. MSK’sconfiguration_info(retention hours, default partition counts and the rest) has no namespace-level equivalent. The settings that exist are per Event Hub and are set after creation. - Placing brokers across availability zones is not supported. An Event Hubs namespace is a regional service with no brokers, so MSK’s per-broker subnets and zone placement have nothing to map to.
Other considerations
- Update the client endpoint and authentication. Repoint the bootstrap servers at the namespace’s Kafka endpoint on port 9093 and switch the SASL configuration to a SAS policy or a Microsoft Entra identity. Produce and consume code, serializers and partitioning logic stay the same.
- Moving from AWS: the namespace starts empty. Messages still in MSK are not copied. Plan a cutover in which producers switch over and consumers drain the old cluster; Kafka’s retention limits how long that can take.
- Revisit capacity after launch. The generated tier and capacity are estimated from your brokers, not measured from your traffic. Watch throughput-unit utilisation once real load arrives; Standard’s throughput units and Premium’s processing units are both adjustable.
- Names stay the same across builds. The namespace name comes from the stack’s stored identity and the original cluster name, so the same stack always builds to the same namespace and a re-plan does not propose replacing infrastructure that has not changed.
On OCI
How it works
Your producers and consumers keep their Kafka clients and connect directly to OCI Streaming over the Kafka protocol. Tensor9 supplies the bootstrap endpoint; there is no Tensor9 proxy between client and broker. Broker metadata can return further advertised addresses, so clients need network access and TLS trust for those addresses as well as the bootstrap address. Authentication changes with the broker: authentication is OCI auth tokens at the stream-pool endpoint, not MSK’s SASL/SCRAM, mTLS, or IAM; your Kafka clients configure OCI’s auth instead. Check producer acknowledgement, idempotence and transaction settings against the support table for this broker; a Kafka-compatible endpoint does not guarantee every Kafka administrative or transactional feature.Limitations
- Moving records, offsets and consumer-group state is not supported. A new broker starts without the source records, offsets or consumer groups, and source offsets cannot be applied to a different log. If retained history must move, plan a separate replication process. - You must replace AWS IAM authentication, MSK Connect and Glue Schema Registry separately. Changing the bootstrap servers does not replace those dependencies.
Other considerations
Who runs what. Oracle operates durability and HA; partitions and retention live under OCI’s quotas rather than a broker you tune directly. Topic partitions hold the retained log, and consumer-group offsets record progress. Before cutover. Coordinate producer and consumer cutover. Review topic creation, retention, partition counts and broker configuration on OCI Streaming, and set up monitoring, backup or recovery and certificate rotation first. Performance. It depends on the broker, storage and workload; this profile has no Tensor9 broker benchmark.On Private Kubernetes
Via Apache Kafka
How it works
Your producers and consumers keep their Kafka clients and connect directly to Apache Kafka on Kubernetes cluster (as a raw Apache Kafka deployment) over the Kafka protocol. Tensor9 supplies the bootstrap endpoint; there is no Tensor9 proxy between client and broker. Broker metadata can return further advertised addresses, so clients need network access and TLS trust for those addresses as well as the bootstrap address. Authentication changes with the broker: SASL/SCRAM and mutual TLS are supported. Replace MSK IAM authentication and manage the broker’s access-control lists. Check producer acknowledgement, idempotence and transaction settings against the support table for this broker; a Kafka-compatible endpoint does not guarantee every Kafka administrative or transactional feature.Limitations
- Moving records, offsets and consumer-group state is not supported. A new broker starts without the source records, offsets or consumer groups, and source offsets cannot be applied to a different log. If retained history must move, plan a separate replication process. - You must replace AWS IAM authentication, MSK Connect and Glue Schema Registry separately. Changing the bootstrap servers does not replace those dependencies.