> ## Documentation Index > Fetch the complete documentation index at: https://docs.tensor9.com/llms.txt > Use this file to discover all available pages before exploring further. # Kinesis Data Firehose > AWS Kinesis Data Firehose. A delivery stream that buffers incoming records and writes them into S3, Redshift, OpenSearch or an HTTP endpoint, with optional format conversion. **On this page** * [Coverage by target cloud](#coverage-by-target-cloud) * [How the targets compare](#how-the-targets-compare) * [Max adaptation](#max-adaptation) * [On Google Cloud](#on-google-cloud) * [Via Pub/Sub](#via-pub/sub) * [On Google Cloud and Private Kubernetes](#on-google-cloud-and-private-kubernetes) * [Via Strimzi Kafka](#via-strimzi-kafka) * [On Azure](#on-azure) * [On OCI](#on-oci) ## Coverage by target cloud | Target | Available | | ------------------ | --------- | | Google Cloud | ✓ | | Azure | ✓ | | OCI | ✓ | | Private Kubernetes | ✓ | ## How the targets compare Each row compares a capability of Kinesis Data Firehose with its adaptation on each target. A dash means this row is not stated for that target. ### Max adaptation | Capability | Kinesis Data Firehose | Google Cloud · Pub/Sub | Google Cloud and Private Kubernetes · Strimzi Kafka | Azure | OCI | | ---------------------------------- | ---------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | Ingest (Direct PUT) | PutRecord / PutRecordBatch | buffered onto Google Cloud Pub/Sub | buffered onto Apache Kafka (Strimzi) on the target cluster | buffered onto Azure Event Hubs | buffered onto OCI Streaming | | Record transformation · per-batch | Lambda | the Lambda equivalent | the Lambda equivalent | the Lambda equivalent | the Lambda equivalent | | Delivery sink · destination API | S3 / Redshift / OpenSearch / Snowflake / Splunk / Datadog / HTTP | Records are delivered through destination APIs; warehouse delivery is a record-loading integration, not a Redshift service replacement. | Records are delivered through destination APIs; warehouse delivery is a record-loading integration, not a Redshift service replacement. | Records are delivered through destination APIs; warehouse delivery is a record-loading integration, not a Redshift service replacement. | Records are delivered through destination APIs; warehouse delivery is a record-loading integration, not a Redshift service replacement. | | Format conversion | Yes | Partial - JSON→Parquet served against a default/inferred schema; the declared Glue Data Catalog schema is AWS-specific and has no target counterpart | Partial - JSON→Parquet served against a default/inferred schema; the declared Glue Data Catalog schema is AWS-specific and has no target counterpart | Partial - JSON→Parquet served against a default/inferred schema; the declared Glue Data Catalog schema is AWS-specific and has no target counterpart | Partial - JSON→Parquet served against a default/inferred schema; the declared Glue Data Catalog schema is AWS-specific and has no target counterpart | | Snowflake / Splunk / Datadog sinks | Yes | Yes - delivered to Snowpipe / HEC / intake endpoints from the target cloud | Yes - delivered to Snowpipe / HEC / intake endpoints from the target cloud | Yes - delivered to Snowpipe / HEC / intake endpoints from the target cloud | Yes - delivered to Snowpipe / HEC / intake endpoints from the target cloud | | Redshift sink | Yes | Partial - A retained Redshift destination or a destination-specific warehouse loader; no database, schema or SQL migration | Partial - A retained Redshift destination or a destination-specific warehouse loader; no database, schema or SQL migration | Partial - A retained Redshift destination or a destination-specific warehouse loader; no database, schema or SQL migration | Partial - A retained Redshift destination or a destination-specific warehouse loader; no database, schema or SQL migration | | API coverage | full | partial | partial | partial | partial | ## On Google Cloud ### Via Pub/Sub | Operation | Area | Support | Depth | Notes | | ------------------------------------------- | --------------- | ------------ | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Buffering (size / interval) | Delivery | Supported | Most usage | the buffer size and interval hints drive when a batch flushes to the sink | | Dynamic partitioning | Delivery | Partial | Full surface | key-based dynamic partitioning of the delivered objects is a subset of Firehose's grammar | | CreateDeliveryStream / DeleteDeliveryStream | Delivery stream | Supported | Common | the delivery stream, its buffering policy, and its sink are provisioned at apply-time | | PutRecord / PutRecordBatch (Direct PUT) | Ingest | Supported | Common | records are buffered onto Google Cloud Pub/Sub; a Kinesis-stream source maps to the Kinesis adapter | | Delivery to destination APIs | Sink | Supported | Common | the delivery agent sends records to each configured destination through that service's API, preserving the destination's delivery behavior | | Sink: Apache Iceberg tables | Sink | Out of scope | Full surface | an Apache Iceberg destination is not offered: Iceberg tables register in the AWS Glue Data Catalog, which is AWS-specific and has no target-native counterpart on these backends | | Sink: OpenSearch Serverless | Sink | Partial | Full surface | the AWS serverless collection endpoint is replaced by the target's OpenSearch or search service, using the same delivery mechanism as the OpenSearch destination | | Sink: Redshift (AWS warehouse) | Sink | Partial | Full surface | A retained Redshift destination requires a delivery integration. Moving record delivery to BigQuery, Synapse or ClickHouse requires a destination-specific object-storage load; it does not migrate Redshift schemas, SQL or warehouse behavior. | | Sink: S3 / object store, HTTP, OpenSearch | Sink | Supported | Common | the S3 sink bridges to Google Cloud Storage (the target object store); an HTTP-endpoint sink forwards natively; an OpenSearch sink bridges to the target's OpenSearch/Elasticsearch equivalent | | Sink: Snowflake / Splunk / Datadog (SaaS) | Sink | Supported | Most usage | records are sent to Snowflake Snowpipe, Splunk HTTP Event Collector (HEC), or Datadog's intake API. These endpoints can receive records from the target cloud | | Format conversion (JSON → Parquet) | Transform | Partial | Full surface | JSON→Parquet conversion uses a default or inferred schema. The declared AWS Glue Data Catalog schema is not translated, so conversion does not use your Glue table definition | | Record transformation (Lambda) | Transform | Supported | Most usage | a transform runs each batch through the Lambda equivalent before delivery | #### Acceptance, buffering and delivery The application sends Firehose PutRecord or PutRecordBatch requests to the Tensor9 adapter. The target design buffers accepted records on Google Cloud Pub/Sub, then a delivery worker sends batches to the configured destination. The object-storage destination for this mapping is Google Cloud Storage. Success from the ingest API means acceptance into the buffer, not confirmation from the final destination. Inspect individual results when a batch has both successful and failed records. Size and interval settings control when a batch is ready for delivery. A low-volume stream may wait for the interval; a busy stream may reach the size threshold first. Worker scheduling, optional transformation and destination response time also affect delivery latency, so the buffering interval is not an end-to-end latency guarantee. #### Transformation and destination setup The intended delivery path can run a batch through the selected Lambda adapter before writing its output. JSON-to-Parquet conversion uses a default or inferred schema; it does not import the declared Glue table schema. Validate field types and the resulting files with their readers. HTTP, search and SaaS destinations require their own credentials, reachable endpoints and delivery formats. A warehouse loader delivers records to the selected warehouse. Configure its destination tables and load format separately; this mapping does not migrate a Redshift database, schema, SQL queries or warehouse workload. #### Failure recovery and migration Delivery must tolerate a destination accepting a batch before the worker records success. A retry can therefore repeat records or create a duplicate object. Downstream processing should identify duplicates. Monitor pending work, retry failures and destination delivery, rather than using successful PutRecord calls as a delivery-health signal. Provision destination storage, credentials and transformation dependencies before switching producers. Allow the old AWS stream to finish delivering its buffered records; those pending records are not copied into the new buffer. Test the selected destination with representative batches, including destination failures and retries. Measure delivery delay and sustained throughput with the actual transform, batch settings and destination. ## On Google Cloud and Private Kubernetes ### Via Strimzi Kafka | Operation | Area | Support | Depth | Notes | | ------------------------------------------- | --------------- | ------------ | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Buffering (size / interval) | Delivery | Supported | Most usage | the buffer size and interval hints drive when a batch flushes to the sink | | Dynamic partitioning | Delivery | Partial | Full surface | key-based dynamic partitioning of the delivered objects is a subset of Firehose's grammar | | CreateDeliveryStream / DeleteDeliveryStream | Delivery stream | Supported | Common | the delivery stream, its buffering policy, and its sink are provisioned at apply-time | | PutRecord / PutRecordBatch (Direct PUT) | Ingest | Supported | Common | records are buffered onto Apache Kafka (Strimzi) on the target cluster; a Kinesis-stream source maps to the Kinesis adapter | | Delivery to destination APIs | Sink | Supported | Common | the delivery agent sends records to each configured destination through that service's API, preserving the destination's delivery behavior | | Sink: Apache Iceberg tables | Sink | Out of scope | Full surface | an Apache Iceberg destination is not offered: Iceberg tables register in the AWS Glue Data Catalog, which is AWS-specific and has no target-native counterpart on these backends | | Sink: OpenSearch Serverless | Sink | Partial | Full surface | the AWS serverless collection endpoint is replaced by the target's OpenSearch or search service, using the same delivery mechanism as the OpenSearch destination | | Sink: Redshift (AWS warehouse) | Sink | Partial | Full surface | A retained Redshift destination requires a delivery integration. Moving record delivery to BigQuery, Synapse or ClickHouse requires a destination-specific object-storage load; it does not migrate Redshift schemas, SQL or warehouse behavior. | | Sink: S3 / object store, HTTP, OpenSearch | Sink | Supported | Common | the S3 sink bridges to the appliance object store (MinIO) (the target object store); an HTTP-endpoint sink forwards natively; an OpenSearch sink bridges to the target's OpenSearch/Elasticsearch equivalent | | Sink: Snowflake / Splunk / Datadog (SaaS) | Sink | Supported | Most usage | records are sent to Snowflake Snowpipe, Splunk HTTP Event Collector (HEC), or Datadog's intake API. These endpoints can receive records from the target cloud | | Format conversion (JSON → Parquet) | Transform | Partial | Full surface | JSON→Parquet conversion uses a default or inferred schema. The declared AWS Glue Data Catalog schema is not translated, so conversion does not use your Glue table definition | | Record transformation (Lambda) | Transform | Supported | Most usage | a transform runs each batch through the Lambda equivalent before delivery | #### Acceptance, buffering and delivery The application sends Firehose PutRecord or PutRecordBatch requests to the Tensor9 adapter. The target design buffers accepted records on Apache Kafka (Strimzi) on the target cluster, then a delivery worker sends batches to the configured destination. The object-storage destination for this mapping is the appliance object store (MinIO). Success from the ingest API means acceptance into the buffer, not confirmation from the final destination. Inspect individual results when a batch has both successful and failed records. Size and interval settings control when a batch is ready for delivery. A low-volume stream may wait for the interval; a busy stream may reach the size threshold first. Worker scheduling, optional transformation and destination response time also affect delivery latency, so the buffering interval is not an end-to-end latency guarantee. #### Transformation and destination setup The intended delivery path can run a batch through the selected Lambda adapter before writing its output. JSON-to-Parquet conversion uses a default or inferred schema; it does not import the declared Glue table schema. Validate field types and the resulting files with their readers. HTTP, search and SaaS destinations require their own credentials, reachable endpoints and delivery formats. A warehouse loader delivers records to the selected warehouse. Configure its destination tables and load format separately; this mapping does not migrate a Redshift database, schema, SQL queries or warehouse workload. #### Failure recovery and migration Delivery must tolerate a destination accepting a batch before the worker records success. A retry can therefore repeat records or create a duplicate object. Downstream processing should identify duplicates. Monitor pending work, retry failures and destination delivery, rather than using successful PutRecord calls as a delivery-health signal. Provision destination storage, credentials and transformation dependencies before switching producers. Allow the old AWS stream to finish delivering its buffered records; those pending records are not copied into the new buffer. Test the selected destination with representative batches, including destination failures and retries. Measure delivery delay and sustained throughput with the actual transform, batch settings and destination. ## On Azure | Operation | Area | Support | Depth | Notes | | ------------------------------------------- | --------------- | ------------ | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Buffering (size / interval) | Delivery | Supported | Most usage | the buffer size and interval hints drive when a batch flushes to the sink | | Dynamic partitioning | Delivery | Partial | Full surface | key-based dynamic partitioning of the delivered objects is a subset of Firehose's grammar | | CreateDeliveryStream / DeleteDeliveryStream | Delivery stream | Supported | Common | the delivery stream, its buffering policy, and its sink are provisioned at apply-time | | PutRecord / PutRecordBatch (Direct PUT) | Ingest | Supported | Common | records are buffered onto Azure Event Hubs; a Kinesis-stream source maps to the Kinesis adapter | | Delivery to destination APIs | Sink | Supported | Common | the delivery agent sends records to each configured destination through that service's API, preserving the destination's delivery behavior | | Sink: Apache Iceberg tables | Sink | Out of scope | Full surface | an Apache Iceberg destination is not offered: Iceberg tables register in the AWS Glue Data Catalog, which is AWS-specific and has no target-native counterpart on these backends | | Sink: OpenSearch Serverless | Sink | Partial | Full surface | the AWS serverless collection endpoint is replaced by the target's OpenSearch or search service, using the same delivery mechanism as the OpenSearch destination | | Sink: Redshift (AWS warehouse) | Sink | Partial | Full surface | A retained Redshift destination requires a delivery integration. Moving record delivery to BigQuery, Synapse or ClickHouse requires a destination-specific object-storage load; it does not migrate Redshift schemas, SQL or warehouse behavior. | | Sink: S3 / object store, HTTP, OpenSearch | Sink | Supported | Common | the S3 sink bridges to Azure Blob Storage (the target object store); an HTTP-endpoint sink forwards natively; an OpenSearch sink bridges to the target's OpenSearch/Elasticsearch equivalent | | Sink: Snowflake / Splunk / Datadog (SaaS) | Sink | Supported | Most usage | records are sent to Snowflake Snowpipe, Splunk HTTP Event Collector (HEC), or Datadog's intake API. These endpoints can receive records from the target cloud | | Format conversion (JSON → Parquet) | Transform | Partial | Full surface | JSON→Parquet conversion uses a default or inferred schema. The declared AWS Glue Data Catalog schema is not translated, so conversion does not use your Glue table definition | | Record transformation (Lambda) | Transform | Supported | Most usage | a transform runs each batch through the Lambda equivalent before delivery | #### Acceptance, buffering and delivery The application sends Firehose PutRecord or PutRecordBatch requests to the Tensor9 adapter. The target design buffers accepted records on Azure Event Hubs, then a delivery worker sends batches to the configured destination. The object-storage destination for this mapping is Azure Blob Storage. Success from the ingest API means acceptance into the buffer, not confirmation from the final destination. Inspect individual results when a batch has both successful and failed records. Size and interval settings control when a batch is ready for delivery. A low-volume stream may wait for the interval; a busy stream may reach the size threshold first. Worker scheduling, optional transformation and destination response time also affect delivery latency, so the buffering interval is not an end-to-end latency guarantee. #### Transformation and destination setup The intended delivery path can run a batch through the selected Lambda adapter before writing its output. JSON-to-Parquet conversion uses a default or inferred schema; it does not import the declared Glue table schema. Validate field types and the resulting files with their readers. HTTP, search and SaaS destinations require their own credentials, reachable endpoints and delivery formats. A warehouse loader delivers records to the selected warehouse. Configure its destination tables and load format separately; this mapping does not migrate a Redshift database, schema, SQL queries or warehouse workload. #### Failure recovery and migration Delivery must tolerate a destination accepting a batch before the worker records success. A retry can therefore repeat records or create a duplicate object. Downstream processing should identify duplicates. Monitor pending work, retry failures and destination delivery, rather than using successful PutRecord calls as a delivery-health signal. Provision destination storage, credentials and transformation dependencies before switching producers. Allow the old AWS stream to finish delivering its buffered records; those pending records are not copied into the new buffer. Test the selected destination with representative batches, including destination failures and retries. Measure delivery delay and sustained throughput with the actual transform, batch settings and destination. ## On OCI | Operation | Area | Support | Depth | Notes | | ------------------------------------------- | --------------- | ------------ | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Buffering (size / interval) | Delivery | Supported | Most usage | the buffer size and interval hints drive when a batch flushes to the sink | | Dynamic partitioning | Delivery | Partial | Full surface | key-based dynamic partitioning of the delivered objects is a subset of Firehose's grammar | | CreateDeliveryStream / DeleteDeliveryStream | Delivery stream | Supported | Common | the delivery stream, its buffering policy, and its sink are provisioned at apply-time | | PutRecord / PutRecordBatch (Direct PUT) | Ingest | Supported | Common | records are buffered onto OCI Streaming; a Kinesis-stream source maps to the Kinesis adapter | | Delivery to destination APIs | Sink | Supported | Common | the delivery agent sends records to each configured destination through that service's API, preserving the destination's delivery behavior | | Sink: Apache Iceberg tables | Sink | Out of scope | Full surface | an Apache Iceberg destination is not offered: Iceberg tables register in the AWS Glue Data Catalog, which is AWS-specific and has no target-native counterpart on these backends | | Sink: OpenSearch Serverless | Sink | Partial | Full surface | the AWS serverless collection endpoint is replaced by the target's OpenSearch or search service, using the same delivery mechanism as the OpenSearch destination | | Sink: Redshift (AWS warehouse) | Sink | Partial | Full surface | A retained Redshift destination requires a delivery integration. Moving record delivery to BigQuery, Synapse or ClickHouse requires a destination-specific object-storage load; it does not migrate Redshift schemas, SQL or warehouse behavior. | | Sink: S3 / object store, HTTP, OpenSearch | Sink | Supported | Common | the S3 sink bridges to OCI Object Storage (the target object store); an HTTP-endpoint sink forwards natively; an OpenSearch sink bridges to the target's OpenSearch/Elasticsearch equivalent | | Sink: Snowflake / Splunk / Datadog (SaaS) | Sink | Supported | Most usage | records are sent to Snowflake Snowpipe, Splunk HTTP Event Collector (HEC), or Datadog's intake API. These endpoints can receive records from the target cloud | | Format conversion (JSON → Parquet) | Transform | Partial | Full surface | JSON→Parquet conversion uses a default or inferred schema. The declared AWS Glue Data Catalog schema is not translated, so conversion does not use your Glue table definition | | Record transformation (Lambda) | Transform | Supported | Most usage | a transform runs each batch through the Lambda equivalent before delivery | #### Acceptance, buffering and delivery The application sends Firehose PutRecord or PutRecordBatch requests to the Tensor9 adapter. The target design buffers accepted records on OCI Streaming, then a delivery worker sends batches to the configured destination. The object-storage destination for this mapping is OCI Object Storage. Success from the ingest API means acceptance into the buffer, not confirmation from the final destination. Inspect individual results when a batch has both successful and failed records. Size and interval settings control when a batch is ready for delivery. A low-volume stream may wait for the interval; a busy stream may reach the size threshold first. Worker scheduling, optional transformation and destination response time also affect delivery latency, so the buffering interval is not an end-to-end latency guarantee. #### Transformation and destination setup The intended delivery path can run a batch through the selected Lambda adapter before writing its output. JSON-to-Parquet conversion uses a default or inferred schema; it does not import the declared Glue table schema. Validate field types and the resulting files with their readers. HTTP, search and SaaS destinations require their own credentials, reachable endpoints and delivery formats. A warehouse loader delivers records to the selected warehouse. Configure its destination tables and load format separately; this mapping does not migrate a Redshift database, schema, SQL queries or warehouse workload. #### Failure recovery and migration Delivery must tolerate a destination accepting a batch before the worker records success. A retry can therefore repeat records or create a duplicate object. Downstream processing should identify duplicates. Monitor pending work, retry failures and destination delivery, rather than using successful PutRecord calls as a delivery-health signal. Provision destination storage, credentials and transformation dependencies before switching producers. Allow the old AWS stream to finish delivering its buffered records; those pending records are not copied into the new buffer. Test the selected destination with representative batches, including destination failures and retries. Measure delivery delay and sustained throughput with the actual transform, batch settings and destination. [Service Catalog](/service-adapters/catalog).