No Result
RudderStack Open Source
Data Plane Setup

Kubernetes

Step-by-step instructions for deploying RudderStack on Kubernetes.

You can deploy RudderStack on your Kubernetes cluster using the Helm package manager.

The repository for this Helm chart can be found here.

If you are planning to use RudderStack in production, we strongly recommend using the Kubernetes Helm charts.

Quick overview

helm install deploys RudderStack on the Kubernetes cluster configured with kubectl

The commands are listed below:

git clone git@github.com:rudderlabs/rudderstack-helm.git
cd rudderstack-helm/


# If you're using the RudderStack-hosted control plane:


helm install my-release ./ --set rudderWorkspaceToken="<your_workspace_token>"


# If you're using the self-hosted control plane:


helm install my-release ./ --set backend.controlPlaneJSON=true

Introduction

Installing and setting up RudderStack involves two key steps:

  • Control plane setup
  • Data plane setup

Refer to the RudderStack architecture to know more about the RudderStack control plane and data plane.

Control plane setup

There are two ways you can set up the control plane. This section lists the steps involved in each of them.

Using the RudderStack-hosted control plane

Why do I need to sign up for RudderStack?

RudderStack's dashboard lets you easily set up your data pipelines by configuring your sources and destinations. It is fully hosted by RudderStack and is free for the users. You also get access to some important features like Transformations and a Live Events tab.

  • Note and copy your workspace Token from the top of the page, as shown below. This will be required for setting up the data plane.
RudderStack-Hosted Control Plane

Self-host the control plane

Use this option if you don't wish to sign up for RudderStack.

You can self-host your own control plane using the open-source Control Plane Lite utility.

Note that the control plane set up using the Control Plane Lite utility lacks certain features like Transformations and Live Events tab.

Data plane setup

This section lists the steps to set up the RudderStack Data Plane in your Kubernetes environment.

Prerequisites

  • Kubectl installed and connected to your Kubernetes cluster
  • Helm installed

For RudderStack-hosted control plane

  • Clone the repository containing the RudderStack Helm chart by running the following command:

    git clone git@github.com:rudderlabs/rudderstack-helm.git

  • Navigate to the folder containing the Helm chart.

    cd rudderstack-helm

  • To install the chart with the release name my-release, run the following command after replacing <your_workspace_token> with the workspace token copied from the RudderStack dashboard.

helm install my-release ./ --set rudderWorkspaceToken="<your_workspace_token>"

The above command deploys RudderStack on your default Kubernetes cluster configured with kubectl.

Refer to the Configuration section below for information on the parameters that can be configured during deployment.

  • Once you have successfully followed the steps above, send test events to verify the installation.

For the self-hosted control plane

If you have self-hosted the control plane using the open-source Control Plane Lite utility, follow these instructions to set up the RudderStack data plane in your Kubernetes cluster.

Once you have successfully followed the steps above, send test events to verify the installation.

Upgrading the chart

For RudderStack-hosted control plane

Update the configuration or version of the images and run the following command:

helm upgrade my-release ./ --set rudderWorkspaceToken="<your_workspace_token>"

For self-hosted control plane

Update the configuration or version of the images and run the following command:

helm upgrade my-release ./ --set backend.controlPlaneJSON=true

Uninstalling the chart

To uninstall or delete the my-release deployment, run the following command:

helm uninstall my-release

This removes all the components created by the chart.

Configuration

The following table lists the configurable parameters of the RudderStack chart and their default values:

ParameterDescriptionDefault
rudderWorkspaceTokenWorkspace token from the dashboard-
backend.image.repositoryContainer image repository for the backendrudderlabs/rudder-server
backend.image.versionContainer image tag for the backend. Check the Available versions0.1.9
backend.image.pullPolicyContainer image pull policy for the backend imageAlways
transformer.image.repositoryContainer image repository for the transformerrudderlabs/transformer
transformer.image.versionContainer image tag for the transformer. Check the Available versions0.1.4
transformer.image.imagePullPolicyContainer image pull policy for the transformer imageAlways
backend.extraEnvVarsExtra environments variables to be used by the backend in the deploymentsRefer the values.yaml file
backend.controlPlaneJSONSet this to true for the Data Plane to read the configuration from the workspaceConfig.json file in case you have a self-hosted Control Plane.false

Each of these parameters can be changed in values.yaml. You can also specify each parameter using the --set key=value[,key=value] argument while running the helm install command.

For example:

helm install --name my-release \
  --set backend.image.version=v0.1.6 \
  ./

Note that:

  • The configuration specific to the data plane can be edited in config.yaml.
  • The configuration specific to PostgreSQL can be configured in pg_hba.conf and postgresql.conf.

Instructions for GCP

Make sure you replace the contents of the file rudder-google-application-credentials.json in the repository with the details of your Google service account if you are using Google Cloud Storage or BigQuery for the following cases:

Components

Installing this Helm chart will deploy the following pods and containers in the configured cluster:

POD - {Release name}-rudderstack-0 :

  • rudderstack-backend
  • rudderstack-telegraf-sidecar

POD - {Release name}-rudderstack-postgresql-0 :

  • {Release name}-rudderstack-postgresql

POD - {Release name}-rudderstack-transformer-xxxxxxxxxx-xxxxx:

  • transformer

Contact us

If you come across any issues while setting up RudderStack, you can contact us or start a conversation on our Slack channel.

Contents