Amazon Kinesis Firehose gives you an easy and reliable way to load your streaming data into data lakes, data stores, and a host of other analytics tools. It is a fully-managed service that can be scaled automatically to match the load and throughput of your data, without requiring any additional administration.

RudderStack supports Amazon Kinesis Firehose as a destination where you can seamlessly send your event data.

Getting started

RudderStack supports sending event data to Kinesis Firehose via the following connection modes:

Once you have confirmed that the source platform supports sending events to Kinesis Firehose, follow these steps:

1. From your RudderStack dashboard, add the source. Then, from the list of destinations, select Amazon Kinesis Firehose.
2. Assign a name to your destination and click Continue.

Connection settings

To successfully set up Kinesis Firehose as a destination, you need to configure the following settings:

• AWS Region: Enter the AWS region in which you have created the Kinesis Firehose stream. This is a required field.
• Role-based Authentication: Enable this setting to use the RudderStack IAM role for authentication. For more information on creating an AWS IAM role for RudderStack, refer to this guide.
  • IAM Role ARN: Enter the ARN of the IAM role.

• If Role-based Authentication is disabled, you need to enter the AWS Access Key ID and AWS Secret Access Key to authorize RudderStack in order to write to the configured stream.

• Mapping event to delivery streams: Use this setting to map the RudderStack Event Name to a Kinesis Firehose Delivery Stream.

Sending events to Kinesis Firehose

RudderStack supports sending the identify, page, and track events to specific Kinesis Firehose streams by configuring them in the dashboard.

You can also send an event type as page, identify, or track. For the track events, you can specify the event name based on the event name in the payload. For example:

• If the event name is page, RudderStack sends all the events with the type set to page.
• If event name is Product Added , RudderStack sends all the track events with the event as Product Added.

Policy permissions

To use the Firehose destination with RudderStack correctly, you must have a Firehose stream created in AWS. For more information on creating a Kinesis Firehose data delivery stream, refer to this AWS documentation.

You also need to create an IAM role and attach the policy containing the necessary permissions (PutRecord) for RudderStack to write to the stream.

• To create an IAM role, follow the instructions on Creating an IAM role.
• For more information on creating an IAM policy, refer to this AWS documentation.

A sample permissions policy that allows a user to send event data into Kinesis Firehose is shown below:

Copy
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "firehose:PutRecord"
      ],
      "Resource": [
        "arn:aws:firehose:{region}:{account-id}:stream/{stream-name}"
      ]
    }
  ]
}

FAQ

How does event mapping work with the delivery stream?

• If there is no delivery stream set for an event in the dashboard settings, RudderStack will not send the event to the Firehose stream.
• If an event is set with a delivery stream, RudderStack sends the payload to the configured delivery stream.
• If you have set the event type, Event Name, and * for mapping purposes, RudderStack gives the topmost priority to the event name, followed by type, and then *. For example, if the type of event is track and Event Name is Product Added, RudderStack does the mapping as shown:

Then all the events go to the stream mapped with Product Added.