Fivetran Connector Terraform Configuration
This document provides a step-by-step guide for setting up a custom Fivetran connector using Terraform to export conversations from Decagon to your data stack.
Requirements
- Terraform installed on your local machine
- Access to Fivetran (API key and API secret)
- Decagon API Key
- The URL of Decagon Google Cloud Function:
Terraform Configuration
Provider Configuration
First, set up the Fivetran provider in your Terraform configuration:
terraform {
required_providers {
fivetran = {
source = "fivetran/fivetran"
version = "~> 1.0"
}
}
}
provider "fivetran" {
api_key = var.fivetran_api_key
api_secret = var.fivetran_api_secret
}
Variables
Define variables for sensitive information:
variable "fivetran_api_key" {
type = string
sensitive = true
}
variable "fivetran_api_secret" {
type = string
sensitive = true
}
variable "decagon_api_key" {
type = string
sensitive = true
}
Fivetran Group Resource
Create a Fivetran group for the Decagon connector:
resource "fivetran_group" "decagon_group" {
name = "Decagon Exports"
}
Fivetran Connector Resource
Configure the Fivetran connector for Decagon:
resource "fivetran_connector" "decagon_connector" {
group_id = fivetran_group.decagon_group.id
service = "google_cloud_function"
destination_schema {
name = "decagon_conversations"
}
config {
function_trigger = "https://us-west1-duet-external.cloudfunctions.net/export-wrapper"
secret {
key = "apiKey"
value = var.decagon_api_key
}
}
}
Usage
-
Save the Terraform configuration in a
.tf
file (e.g.,main.tf
). -
Initialize Terraform:
terraform init
-
Set up your variables in a
terraform.tfvars
file or use environment variables. -
Plan your Terraform changes:
terraform plan
-
Apply the Terraform configuration:
terraform apply
Verifying the Configuration
After applying the Terraform configuration, you can verify the setup by:
-
Checking the Fivetran portal to ensure the connector is created and properly configured.
-
Running an initial sync in Fivetran.
-
Verifying the exported conversations in your destination database:
SELECT * FROM decagon_conversations.conversation;
Note
Remember to handle your sensitive information (API keys, secrets, etc.) securely. Never commit these directly to version control. Consider using Terraform Cloud, AWS Secrets Manager, or similar services for managing sensitive variables in production environments.