Blueprints

Copy flows from development to QA and staging environments or tenants

About this blueprint

System

This system flow copies flows from the current tenant to a new Kestra tenant or environments. The pattern shown here is particularly useful if you want to automatically promote flows from development to QA and staging environments or tenants. Here, we release development flows to a staging tenant every day at 5 PM.

The flow is composed of the following tasks:

  1. Download flows from the source tenant using the Download plugin.
  2. Import flows into the target tenant using the Request plugin.

To authenticate with the Kestra API, you need to provide an API token with the necessary permissions. You can generate a token from the UI by going to the Service Accounts tab under the IAM page, creating a new service account and generating a new token from the API Tokens section. We recommend adding that token as a secret in the system namespace. Here, we've named the secret API_TOKEN.

We've set up retries via pluginDefaults to handle any network issues or temporary failures.

Make sure to adjust the host and newTenant variables in the flow configuration to match your environment and requirements.

yaml
id: copy_flows_to_new_tenant
namespace: system

variables:
  host: preview-ee-kafka.kestra.io
  newTenant: staging

tasks:
  - id: dev_flows
    type: io.kestra.plugin.core.http.Download
    uri: http://{{vars.host}}/api/v1/{{flow.tenantId}}/flows/export/by-query
    method: GET
    contentType: "application/json"
    headers:
      authorization: Bearer {{ secret('API_TOKEN') }}

  - id: import_to_staging
    type: io.kestra.plugin.core.http.Request
    uri: https://{{vars.host}}/api/v1/{{vars.newTenant}}/flows/import
    method: POST
    contentType: multipart/form-data
    formData:
      fileUpload:
        name: "flows.zip"
        content: "{{ outputs.dev_flows.uri }}"
    headers:
      authorization: Bearer {{ secret('API_TOKEN') }}

pluginDefaults:
  - type: io.kestra.plugin.core.http
    forced: true
    values:
      retry:
        type: constant
        interval: PT1S
        maxAttempt: 3

triggers:
  - id: daily_at_5_pm
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "0 17 * * *"

Download

Request

More Related Blueprints

New to Kestra?

Use blueprints to kickstart your first workflows.

Get started with Kestra