PipeCD – User Guide

Docs-V0.56.x: Managing application

Mon, 01 Jan 0001 00:00:00 +0000

Note: You must have at least one activated/running Piped to enable using any of the following features of PipeCD. Please refer to Piped installation docs if you do not have any Piped in your pocket.

Docs-V0.56.x: Managing Piped

Mon, 01 Jan 0001 00:00:00 +0000

In order to use Piped you need to register through PipeCD control plane, so please refer register a Piped docs if you do not have already. After registering successfully, you can monitor your Piped live state via the PipeCD console on the settings page.

Docs-V0.56.x: Managing Control Plane

Mon, 01 Jan 0001 00:00:00 +0000

Docs-V0.56.x: Connect between CI and CD with event watcher

Mon, 01 Jan 0001 00:00:00 +0000

The only way to upgrade your application with PipeCD is modifying configuration files managed by the Git repositories. It brings benefits quite a bit, but it can be painful to manually update them every time in some cases (e.g. continuous deployment to your development environment for debugging, the latest prerelease to the staging environment).

If you’re experiencing any of the above pains, Event watcher is for you. Event watcher works as a helper facility to seamlessly link CI and CD. This feature lets you automatically update files managed by your Piped when an arbitrary event has occurred. While it empowers you to build pretty versatile workflows, the canonical use case is that you trigger a new deployment by image updates, package releases, etc.

This guide walks you through configuring Event watcher and how to push an Event.

Prerequisites

Before we get into configuring EventWatcher, be sure to configure Piped. See here for more details.

Usage

File updating can be done by registering the latest value corresponding to the Event in the Control Plane and comparing it with the current value.

Therefore, you mainly need to:

define which values in which files should be updated when a new Event found.
integrate a step to push an Event to the Control Plane using pipectl into your CI workflow.

1. Defining Events

Use the `.pipe/` directory

NOTE: This way is deprecated and will be removed in the future, so please use the application configuration.

Prepare EventWatcher configuration files under the .pipe/ directory at the root of your Git repository. In that files, you define which values in which files should be updated when the Piped found out a new Event.

For instance, suppose you want to update the Kubernetes manifest defined in helloworld/deployment.yaml when an Event with the name helloworld-image-update occurs:

apiVersion: pipecd.dev/v1beta1
kind: EventWatcher
spec:
  events:
    - name: helloworld-image-update
      replacements:
        - file: helloworld/deployment.yaml
          yamlField: $.spec.template.spec.containers[0].image

The full list of configurable EventWatcher fields are here.

Use the application configuration

Define what to do for which event in the application configuration file of the target application.

matcher: Which event should be handled.
handler: What to do for the event which is specified by matcher.

For instance, suppose you want to update the Kubernetes manifest defined in helloworld/deployment.yaml when an Event with the name helloworld-image-update occurs:

apiVersion: pipecd.dev/v1beta1
kind: KubernetesApp
spec:
  name: helloworld
  eventWatcher:
    - matcher:
        name: helloworld-image-update
      handler:
        type: GIT_UPDATE
        config:
          replacements:
            - file: deployment.yaml
              yamlField: $.spec.template.spec.containers[0].image

The full list of configurable eventWatcher fields are here.

2. Pushing an Event with `pipectl`

To register a new value corresponding to Event such as the above in the Control Plane, you need to perform pipectl. And we highly recommend integrating a step for that into your CI workflow.

You first need to set-up the pipectl:

Install it on your CI system or where you want to run according to this guide.
Grab the API key to which the READ_WRITE role is attached according to this guide.

Once you’re all set up, pushing a new Event to the Control Plane by the following command:

pipectl event register \
    --address={CONTROL_PLANE_API_ADDRESS} \
    --api-key={API_KEY} \
    --name=helloworld-image-update \
    --data=gcr.io/pipecd/helloworld:v0.2.0

You can see the status on the event list page.

After a while, Piped will create a commit as shown below:

     spec:
       containers:
       - name: helloworld
-        image: gcr.io/pipecd/helloworld:v0.1.0
+        image: gcr.io/pipecd/helloworld:v0.2.0

NOTE: Keep in mind that it may take a little while because Piped periodically fetches the new events from the Control Plane. You can change its interval according to here.

[optional] Using labels

Event watcher is a project-wide feature, hence an event name is unique inside a project. That is, you can update multiple repositories at the same time if you use the same event name for different events.

On the contrary, if you want to explicitly distinguish those, we recommend using labels. You can make an event definition unique by using any number of labels with arbitrary keys and values. Suppose you define an event with the labels env: dev and appName: helloworld:

When you use the .pipe/ directory, you can configure like below.

apiVersion: pipecd.dev/v1beta1
kind: EventWatcher
spec:
  events:
    - name: image-update
      labels:
        env: dev
        appName: helloworld
      replacements:
        - file: helloworld/deployment.yaml
          yamlField: $.spec.template.spec.containers[0].image

The other example is like below.

apiVersion: pipecd.dev/v1beta1
kind: ApplicationKind
spec:
  name: helloworld
  eventWatcher:
    - matcher:
        name: image-update
        labels:
          env: dev
          appName: helloworld
      handler:
        type: GIT_UPDATE
        config:
          replacements:
            - file: deployment.yaml
              yamlField: $.spec.template.spec.containers[0].image

The file update will be executed only when the labels are explicitly specified with the --labels flag.

pipectl event register \
    --address=CONTROL_PLANE_API_ADDRESS \
    --api-key=API_KEY \
    --name=image-update \
    --labels env=dev,appName=helloworld \
    --data=gcr.io/pipecd/helloworld:v0.2.0

Note that it is considered a match only when labels are an exact match.

[optional] Using contexts

You can also attach additional metadata to the event. This information can be added as a trailer to the git commit when Event Watcher using the GIT_UPDATE handler. This can be useful when attaching information from the source code repository to the manifest repository.

For example, you can attach the source code commit link to the manifest repository.

pipectl event register \
    --address=CONTROL_PLANE_API_ADDRESS \
    --api-key=API_KEY \
    --name=sample \
    --data=gcr.io/pipecd/helloworld:v0.48.0 \
    --contexts Source-Commit-Hash=xxxxxxx,Source-Commit-URL=https://github.com/pipe-cd/pipecd/commit/xxxxxxx

# In manifest repository
$ git show                                                                        
commit ff46cdc9a3ce87a9a66436269251a4870ac55183 (HEAD -> main, origin/main, origin/HEAD)
Author: ffjlabo <pipecd.dev@gmail.com>
Date:   Wed Oct 30 16:56:36 2024 +0900

    Replace values with "gcr.io/pipecd/helloworld:v0.48.0" set by Event "simple"

    Source-Commit-Hash: xxxxxxx
    Source-Commit-URL: https://github.com/pipe-cd/pipecd/commit/xxxxxxx

[optional] Using commit-hash and commit-url for Deployment Trace

You can attach commit-hash and commit-url of the commit that trigger the event as metadata to the event. Once you added these data to your event on registering, you would have a better way to understand what deployments are triggered by your application commit via PipeCD Deployment Trace feature.

To attach these values

pipectl event register \
    --address=CONTROL_PLANE_API_ADDRESS \
    --api-key=API_KEY \
    --name=sample \
    --data=gcr.io/pipecd/helloworld:v0.48.0 \
    --commit-hash=20c98a503062021720b2fcf2058276b3245xxxxx
    --commit-url=https://github.com/pipe-cd/pipecd/commit/20c98a503062021720b2fcf2058276b3245xxxxx

NOTE: There are few more information of the triggered commit you can attach as event metadata, find more in DeploymentTrace docs.

Examples

Suppose you want to update your configuration file after releasing a new Helm chart.

You define the configuration for event watcher in helloworld/app.pipecd.yaml file like:

apiVersion: pipecd.dev/v1beta1
kind: KubernetesApp
spec:
  input:
    helmChart:
      name: helloworld
      version: 0.1.0
  eventWatcher:
    - matcher:
        name: image-update
        labels:
          env: dev
          appName: helloworld
      handler:
        type: GIT_UPDATE
        config:
          replacements:
            - file: app.pipecd.yaml
              yamlField: $.spec.input.helmChart.version

Push a new version 0.2.0 as data when the Helm release is completed.

pipectl event register \
    --address=CONTROL_PLANE_API_ADDRESS \
    --api-key=API_KEY \
    --name=helm-release \
    --labels env=dev,appName=helloworld \
    --data=0.2.0

Then you’ll see that Piped updates as:

apiVersion: pipecd.dev/v1beta1
kind: KubernetesApp
spec:
  input:
    helmChart:
      name: helloworld
-     version: 0.1.0
+     version: 0.2.0
  eventWatcher:
    - matcher:
        name: image-update
        labels:
          env: dev
          appName: helloworld
      handler:
        type: GIT_UPDATE
        config:
          replacements:
            - file: app.pipecd.yaml
              yamlField: $.spec.input.helmChart.version

Github Actions

If you’re using Github Actions in your CI workflow, actions-event-register is for you! With it, you can easily register events without any installation.

Docs-V0.56.x: Better connect between CI and CD with Deployment Trace

Mon, 01 Jan 0001 00:00:00 +0000

You are a developer who works with application code change, and don’t know what deployment is triggered by your commit on PipeCD UI? This feature is for you.

If you’re using PipeCD Event Watcher to trigger the deployment for your code change, you can attach information of the triggered commit as the event data, PipeCD will use that information and helps you to make a link between your application code commit and the triggered deployments that reflect your code change.

Usage

Via pipectl event register command

Usage:
  pipectl event register [flags]

Flags:
      --commit-author string      The author of commit that triggers the event.
      --commit-hash string        The commit hash that triggers the event.
      --commit-message string     The message of commit that triggers the event.
      --commit-timestamp int      The timestamp of commit that triggers the event.
      --commit-title string       The title of commit that triggers the event.
      --commit-url string         The URL of commit that triggers the event.

Note: You have to attach at least commit-hash and commit-url as the event data in order to use the Deployment Trace feature.

Github Actions

If you’re using Github Actions in your CI workflow, actions-event-register is for you! With it, you can easily register events without any installation.

Docs-V0.56.x: Confidently review your changes with Plan Preview

Mon, 01 Jan 0001 00:00:00 +0000

In order to help developers review the pull request with a better experience and more confidence to approve it to trigger the actual deployments, PipeCD provides a way to preview the deployment plan of all updated applications by that pull request.

Here are what will be included currently in the result of plan-preview process:

which application will be deployed once the pull request got merged
which deployment strategy (QUICK_SYNC or PIPELINE_SYNC) will be used
which resources will be added, deleted, or modified

This feature will available for all application kinds: KUBERNETES, TERRAFORM, CLOUD_RUN, LAMBDA and Amazon ECS.

PlanPreview with GitHub actions pipe-cd/actions-plan-preview

Prerequisites

Ensure the version of your Piped is at least v0.11.0.
Having an API key that has READ_WRITE role to authenticate with PipeCD’s Control Plane. A new key can be generated from settings/api-key page of your PipeCD web.

Usage

Plan-preview result can be requested by using pipectl command-line tool as below:

pipectl plan-preview \
  --address={ PIPECD_CONTROL_PLANE_ADDRESS } \
  --api-key={ PIPECD_API_KEY } \
  --repo-remote-url={ REPO_REMOTE_GIT_SSH_URL } \
  --head-branch={ HEAD_BRANCH } \
  --head-commit={ HEAD_COMMIT } \
  --base-branch={ BASE_BRANCH } \
  --sort-label-keys={ SORT_LABEL_KEYS }

You can run it locally or integrate it to your CI system to run automatically when a new pull request is opened/updated. Use --help to see more options.

pipectl plan-preview --help

Order of the results

By default, the results are sorted by PipedID and Application Name.

If you want to sort the results by labels, add --sort-label-keys option. For example, when you run with --sort-label-keys=env,team, the results will be sorted by PipedID, env label, team label, and then Application Name.

GitHub Actions

If you are using GitHub Actions, you can seamlessly integrate our prepared actions-plan-preview to your workflows. This automatically comments the plan-preview result on the pull request when it is opened or updated. You can also trigger to run plan-preview manually by leave a comment /pipecd plan-preview on the pull request.

Docs-V0.56.x: Insights

Mon, 01 Jan 0001 00:00:00 +0000

Application metrics

The topmost block helps you understand how many applications your project has.

Deployment metrics

Based on your executed deployment data, PipeCD provides charts that help you better understand the delivery performance of your organization.

You can view daily, and monthly data visualizations of your entire project, a specific application, or a group of applications that match a list of labels.

Deployment Frequency

How often does your application/project deploy code to production.

Change Failure Rate

How often deployment failures occur in production that requires an immediate remedy (fix, rollback…).

Lead Time for Changes

How long does it take to go from code committed to code successfully running on production.

WIP

Mean Time To Restore

How long does it generally take to restore service when a service incident occurs.

WIP

Docs-V0.56.x: Metrics

Mon, 01 Jan 0001 00:00:00 +0000

PipeCD comes with a monitoring system including Prometheus, Alertmanager, and Grafana. This page walks you through how to set up and use them.

Monitoring overview

Monitoring Architecture

Both the Control plane and piped agent have their own “admin servers” (the default port number is 9085), which are simple HTTP servers providing operational information such as health status, running version, go profile, and monitoring metrics.

The piped agent collects its metrics and periodically sends them to the Control plane. The Control plane then compacts its resource usage and cluster information with the metrics sent by the piped agent and re-publishes them via its admin server. When the PipeCD monitoring feature is turned on, Prometheus, Alertmanager, and Grafana are deployed with the Control plane, and Prometheus retrieves metrics information from the Control plane’s admin server.

Developers managing the piped agent can also get metrics directly from the piped agent and monitor them with their custom monitoring service.

Enable monitoring system

To enable monitoring system for PipeCD, you first need to set the following value to helm install when installing.

--set monitoring.enabled=true

Dashboards

If you’ve already enabled monitoring system in the previous section, you can access Grafana using port forwarding:

kubectl port-forward -n {NAMESPACE} svc/{PIPECD_RELEASE_NAME}-grafana 3000:80

Control Plane dashboards

There are three dashboards related to Control Plane:

Overview - usage stats of PipeCD
Incoming Requests - gRPC and HTTP requests stats to check for any negative impact on users
Go - processes stats of PipeCD components

Piped dashboards

Visualize the metrics of Piped registered in the Control plane.

Overview - usage stats of piped agents
Process - resource usage of piped agent
Go - processes stats of piped agents.

Cluster dashboards

Because cluster dashboards tracks cluster-wide metrics, defaults to disable. You can enable it with:

--monitoring.clusterStats=true

There are three dashboards that track metrics for:

Node - nodes stats within the Kubernetes cluster where PipeCD runs on
Pod - stats for pods that make PipeCD up
Prometheus - stats for Prometheus itself

Alert notifications

If you want to send alert notifications to external services like Slack, you need to set an alertmanager configuration file.

For example, let’s say you use Slack as a receiver. Create values.yaml and put the following configuration to there.

prometheus:
  alertmanagerFiles:
    alertmanager.yml:
      global:
        slack_api_url: {YOUR_WEBHOOK_URL}
      route:
        receiver: slack-notifications
      receivers:
        - name: slack-notifications
          slack_configs:
            - channel: '#your-channel'

And give it to the helm install command when installing.

--values=values.yaml

See here for more details on AlertManager’s configuration.

Piped agent metrics

Metric	Type	Description
`cloudprovider_kubernetes_tool_calls_total`	counter	Number of calls made to run the tool like kubectl, kustomize.
`deployment_status`	gauge	The current status of deployment. 1 for current status, 0 for others.
`livestatestore_kubernetes_api_requests_total`	counter	Number of requests sent to kubernetes api server.
`livestatestore_kubernetes_resource_events_total`	counter	Number of resource events received from kubernetes server.
`plan_preview_command_handled_total`	counter	Total number of plan-preview commands handled at piped.
`plan_preview_command_handling_seconds`	histogram	Histogram of handling seconds of plan-preview commands.
`plan_preview_command_received_total`	counter	Total number of plan-preview commands received at piped.

Control plane metrics

All Piped’s metrics are sent to the control plane so that they are also available on the control plane’s metrics server.

Metric	Type	Description
`cache_get_operation_total`	counter	Number of cache get operation while processing.
`grpcapi_create_deployment_total`	counter	Number of successful CreateDeployment RPC with project label.
`http_request_duration_milliseconds`	histogram	Histogram of request latencies in milliseconds.
`http_requests_total`	counter	Total number of HTTP requests.
`insight_application_total`	gauge	Number of applications currently controlled by control plane.

Health Checking

The below components expose their endpoint for health checking.

server
ops
piped
launcher (only when you run with designating the launcher-admin-port option.)

The spec of the health check endpoint is as below.

Path: /healthz
Port: the same as admin server’s port. 9085 by default.

Docs-V0.56.x: Command-line tool: pipectl

Mon, 01 Jan 0001 00:00:00 +0000

Besides using web UI, PipeCD also provides a command-line tool, pipectl, which allows you to run commands against your project’s resources. You can use pipectl to add and sync applications, wait for a deployment status.

Installation

The Pipectl command-line tool can be installed using one of the following methods:

Directly download and set up binary

Download the appropriate version for your platform from PipeCD Releases.

We recommend using the latest version of pipectl to avoid unforeseen issues. Run the following script:
```
# OS="darwin" or "linux"
curl -Lo ./pipectl https://github.com/pipe-cd/pipecd/releases/download/v0.56.0/pipectl_v0.56.0_{OS}_amd64
```
Make the pipectl binary executable.
```
chmod +x ./pipectl
```

Move the binary to your PATH.

sudo mv ./pipectl /usr/local/bin/pipectl

Test to ensure the version you installed is up-to-date.
```
pipectl version
```

Using Asdf

About Asdf

Add pipectl plugin to asdf. (If you have not yet asdf add plugin add pipectl.)
```
asdf plugin add pipectl
```
Install pipectl. Available versions are here.
```
asdf install pipectl {VERSION}
```
Set a version.
```
asdf global pipectl {VERSION}
```
Test to ensure the version you installed is up-to-date.
```
pipectl version
```

Using Aqua

About Aqua

Add pipectl to aqua.yaml. (If you want to select a version, use aqua g -i -s pipe-cd/pipecd/pipectl)
```
aqua g -i pipe-cd/pipecd/pipectl
```
Install pipectl.
```
aqua i
```
Test to ensure the version you installed is up-to-date.
```
pipectl version
```

Using Homebrew

About Homebrew

Add the pipe-cd/tap and fetch new formulae from GitHub.
```
brew tap pipe-cd/tap
brew update
```
Install pipectl.
```
brew install pipectl
```
Test to ensure the version you installed is up-to-date.
```
pipectl version
```

Run in Docker container

We are storing every version of docker image for pipectl on GitHub Container Registry. Available versions are here.

docker run --rm ghcr.io/pipe-cd/pipectl:{VERSION} -h

Authentication

In order for pipectl to authenticate with PipeCD’s Control Plane, it needs an API key, which can be created from Settings/API Key tab on the web UI. There are two kinds of key role: READ_ONLY and READ_WRITE. Depending on the command, it might require an appropriate role to execute.

Adding a new API key from Settings tab

When executing a command of pipectl you have to specify either a string of API key via --api-key flag or a path to the API key file via --api-key-file flag.

Usage

Help

Run help to know the available commands:

$ pipectl --help

The command line tool for PipeCD.

Usage:
  pipectl [command]

Available Commands:
  application  Manage application resources.
  deployment   Manage deployment resources.
  encrypt      Encrypt the plaintext entered in either stdin or the --input-file flag.
  event        Manage event resources.
  help         Help about any command
  init         Generate an application config (app.pipecd.yaml) easily and interactively.
  piped        Manage piped resources.
  plan-preview Show plan preview against the specified commit.
  version      Print the information of current binary.

Flags:
  -h, --help                               help for pipectl
      --log-encoding string                The encoding type for logger [json|console|humanize]. (default "humanize")
      --log-level string                   The minimum enabled logging level. (default "info")
      --metrics                            Whether metrics is enabled or not. (default true)
      --profile                            If true enables uploading the profiles to Stackdriver.
      --profile-debug-logging              If true enables logging debug information of profiler.
      --profiler-credentials-file string   The path to the credentials file using while sending profiles to Stackdriver.

Use "pipectl [command] --help" for more information about a command.

Adding a new application

Add a new application into the project:

pipectl application add \
    --address=CONTROL_PLANE_API_ADDRESS \
    --api-key=API_KEY \
    --app-name=simple \
    --app-kind=KUBERNETES \
    --piped-id=PIPED_ID \
    --platform-provider=kubernetes-default \
    --repo-id=examples \
    --app-dir=kubernetes/simple

Run help to know what command flags should be specified:

$ pipectl application add --help

Add a new application.

Usage:
  pipectl application add [flags]

Flags:
      --app-dir string            The relative path from the root of repository to the application directory.
      --app-kind string           The kind of application. (KUBERNETES|TERRAFORM|LAMBDA|CLOUDRUN|ECS)
      --app-name string           The application name.
      --platform-provider string  The platform provider name. One of the registered providers in the piped configuration. The previous name of this field is cloud-provider.
      --config-file-name string   The configuration file name. (default "app.pipecd.yaml")
      --description string        The description of the application.
  -h, --help                      help for add
      --piped-id string           The ID of piped that should handle this application.
      --repo-id string            The repository ID. One the registered repositories in the piped configuration.

Global Flags:
      --address string                     The address to Control Plane api.
      --api-key string                     The API key used while authenticating with Control Plane.
      --api-key-file string                Path to the file containing API key used while authenticating with Control Plane.
      --cert-file string                   The path to the TLS certificate file.
      --insecure                           Whether disabling transport security while connecting to Control Plane.
      --log-encoding string                The encoding type for logger [json|console|humanize]. (default "humanize")
      --log-level string                   The minimum enabled logging level. (default "info")
      --metrics                            Whether metrics is enabled or not. (default true)
      --profile                            If true enables uploading the profiles to Stackdriver.
      --profile-debug-logging              If true enables logging debug information of profiler.
      --profiler-credentials-file string   The path to the credentials file using while sending profiles to Stackdriver.

Syncing an application

Send a request to sync an application and exit immediately when the deployment is triggered:

pipectl application sync \
    --address={CONTROL_PLANE_API_ADDRESS} \
    --api-key={API_KEY} \
    --app-id={APPLICATION_ID}

Send a request to sync an application and wait until the triggered deployment reaches one of the specified statuses:

pipectl application sync \
    --address={CONTROL_PLANE_API_ADDRESS} \
    --api-key={API_KEY} \
    --app-id={APPLICATION_ID} \
    --wait-status=DEPLOYMENT_SUCCESS,DEPLOYMENT_FAILURE

Getting an application

Display the information of a given application in JSON format:

pipectl application get \
    --address={CONTROL_PLANE_API_ADDRESS} \
    --api-key={API_KEY} \
    --app-id={APPLICATION_ID}

Listing applications

Find and display the information of matching applications in JSON format:

pipectl application list \
    --address={CONTROL_PLANE_API_ADDRESS} \
    --api-key={API_KEY} \
    --app-name={APPLICATION_NAME} \
    --app-kind=KUBERNETES \

Disable an application

Disable an application with given id:

pipectl application disable \
    --address={CONTROL_PLANE_API_ADDRESS} \
    --api-key={API_KEY} \
    --app-id={APPLICATION_ID}

Deleting an application

Delete an application with given id:

pipectl application delete \
    --address={CONTROL_PLANE_API_ADDRESS} \
    --api-key={API_KEY} \
    --app-id={APPLICATION_ID}

List deployments

Show the list of deployments based on filters.

pipectl deployment list \
    --address={CONTROL_PLANE_API_ADDRESS} \
    --api-key={API_KEY} \
    --app-id={APPLICATION_ID}

Waiting a deployment status

Wait until a given deployment reaches one of the specified statuses:

pipectl deployment wait-status \
    --address={CONTROL_PLANE_API_ADDRESS} \
    --api-key={API_KEY} \
    --deployment-id={DEPLOYMENT_ID} \
    --status=DEPLOYMENT_SUCCESS

Get deployment stages log

Get deployment stages log.

pipectl deployment logs \
    --address={CONTROL_PLANE_API_ADDRESS} \
    --api-key={API_KEY} \
    --deployment-id={DEPLOYMENT_ID}

Registering an event for EventWatcher

pipectl event register \
    --address={CONTROL_PLANE_API_ADDRESS} \
    --api-key={API_KEY} \
    --name=example-image-pushed \
    --data=gcr.io/pipecd/example:v0.1.0

See more on usage of Event Watcher.

Encrypting the data you want to use when deploying

Encrypt the plaintext entered either in stdin or via the --input-file flag.

You can encrypt it the same way you do from the web.

From stdin:

pipectl encrypt \
    --address={CONTROL_PLANE_API_ADDRESS} \
    --api-key={API_KEY} \
    --piped-id={PIPED_ID} <{PATH_TO_SECRET_FILE}

From the --input-file flag:

pipectl encrypt \
    --address={CONTROL_PLANE_API_ADDRESS} \
    --api-key={API_KEY} \
    --piped-id={PIPED_ID} \
    --input-file={PATH_TO_SECRET_FILE}

Note: The docs for pipectl available command is maybe outdated, we suggest users use the help command for the updated usage while using pipectl.

Generating an application config (app.pipecd.yaml)

Generate an app.pipecd.yaml interactively:

$ pipectl init
Which platform? Enter the number [0]Kubernetes [1]ECS: 1
Name of the application: myApp
...

After the above interaction, you can get the config YAML:

apiVersion: pipecd.dev/v1beta1
kind: ECSApp
spec:
  name: myApp
  input:
    serviceDefinitionFile: serviceDef.yaml
    taskDefinitionFile: taskDef.yaml
    targetGroups:
      primary:
        targetGroupArn: arn:aws:elasticloadbalancing:ap-northeast-1:123456789012:targetgroup/xxx/xxx
        containerName: web
        containerPort: 80
  description: Generated by `pipectl init`. See https://pipecd.dev/docs/user-guide/configuration-reference/ for more.

See Feature Status.

You want more?

We always want to add more needed commands into pipectl. Please let us know what command you want to add by creating issues in the pipe-cd/pipecd repository. We also welcome your pull request to add the command.

Docs-V0.56.x: PipeCD Terraform provider

Mon, 01 Jan 0001 00:00:00 +0000

Besides using web UI and command line tool, PipeCD community also provides Terraform module, terraform-provider-pipecd, which allows you to manage PipeCD resources. This provider enables us to add, update, and delete PipeCD resources as Infrastructure as Code (IaC). Storing resources as code in a version control system like Git repository ensures more reliability, security, and makes it more friendly for engineers to manage PipeCD resources with the power of Git leverage.

Usage

Setup Terraform provider

Add terraform block to declare that you use PipeCD Terraform provider. You need to input a control plane’s host name and API key via provider settings or environment variables. API key is available on the web UI.

terraform {
  required_providers {
    pipecd = {
      source  = "pipe-cd/pipecd"
      version = "0.1.0"
    }
  }
  required_version = ">= 1.4"
}

provider "pipecd" {
  # pipecd_host    = "" // optional, if not set, read from environments as PIPECD_HOST
  # pipecd_api_key = "" // optional, if not set, read from environments as PIPECD_API_KEY
}

Manage Piped agent

Add pipecd_piped resource to manage a Piped agent.

resource "pipecd_piped" "mypiped" {
  name        = "mypiped"
  description = "This is my piped"
  id          = "my-piped-id"
}

Adding a new application

Add pipecd_application resource to manage an application.

// CloudRun Application
resource "pipecd_application" "main" {
  kind              = "CLOUDRUN"
  name              = "example-application"
  description       = "This is the simple application"
  platform_provider = "cloudrun-inproject"
  piped_id          = "your-piped-id"
  git = {
    repository_id = "examples"
    remote        = "git@github.com:pipe-cd/examples.git"
    branch        = "master"
    path          = "cloudrun/simple"
    filename      = "app.pipecd.yaml"
  }
}

You want more?

We always want to add more needed resources into the Terraform provider. Please let the maintainers know what resources you want to add by creating issues in the pipe-cd/terraform-provider-pipecd repository. We also welcome your pull request to contribute!

Docs-V0.56.x: Configuration reference

Mon, 01 Jan 0001 00:00:00 +0000

Kubernetes Application

apiVersion: pipecd.dev/v1beta1
kind: KubernetesApp
spec:
  input:
  pipeline:
  ...

Field	Type	Description	Required
name	string	The application name.	Yes (if you want to create PipeCD application through the application configuration file)
labels	map[string]string	Additional attributes to identify applications.	No
description	string	Notes on the Application.	No
input	KubernetesDeploymentInput	Input for Kubernetes deployment such as kubectl version, helm version, manifests filter…	No
trigger	DeploymentTrigger	Configuration for trigger used to determine should we trigger a new deployment or not.	No
planner	DeploymentPlanner	Configuration for planner used while planning deployment.	No
commitMatcher	CommitMatcher	Forcibly use QuickSync or Pipeline when commit message matched the specified pattern.	No
quickSync	KubernetesQuickSync	Configuration for quick sync.	No
pipeline	Pipeline	Pipeline for deploying progressively.	No
service	KubernetesService	Which Kubernetes resource should be considered as the Service of application. Empty means the first Service resource will be used.	No
workloads	[]KubernetesWorkload	Which Kubernetes resources should be considered as the Workloads of application. Empty means all Deployment resources.	No
trafficRouting	KubernetesTrafficRouting	How to change traffic routing percentages.	No
encryption	SecretEncryption	List of encrypted secrets and targets that should be decrypted before using.	No
attachment	Attachment	List of attachment sources and targets that should be attached to manifests before using.	No
timeout	duration	The maximum length of time to execute deployment before giving up. Default is 6h.	No
notification	DeploymentNotification	Additional configuration used while sending notification to external services.	No
postSync	PostSync	Additional configuration used as extra actions once the deployment is triggered.	No
variantLabel	KubernetesVariantLabel	The label will be configured to variant manifests used to distinguish them.	No
eventWatcher	[]EventWatcher	List of configurations for event watcher.	No
driftDetection	DriftDetection	Configuration for drift detection.	No

Annotations

Kubernetes resources can be managed by some annotations provided by PipeCD.

Annotation key	Target resource(s)	Possible values	Description
`pipecd.dev/ignore-drift-detection`	any	“true”	Whether the drift detection should ignore this resource.
`pipecd.dev/server-side-apply`	any	“true”	Use server side apply instead of client side apply.
`pipecd.dev/sync-by-replace`	any	“enabled”	Use `replace` instead of `apply`.
`pipecd.dev/force-sync-by-replace`	any	“enabled”	Use `replace --force` instead of `apply`.

Terraform application

apiVersion: pipecd.dev/v1beta1
kind: TerraformApp
spec:
  input:
  pipeline:
  ...

Field	Type	Description	Required
name	string	The application name.	Yes if you set the application through the application configuration file
labels	map[string]string	Additional attributes to identify applications.	No
description	string	Notes on the Application.	No
input	TerraformDeploymentInput	Input for Terraform deployment such as terraform version, workspace…	No
trigger	DeploymentTrigger	Configuration for trigger used to determine should we trigger a new deployment or not.	No
planner	DeploymentPlanner	Configuration for planner used while planning deployment.	No
quickSync	TerraformQuickSync	Configuration for quick sync.	No
pipeline	Pipeline	Pipeline for deploying progressively.	No
encryption	SecretEncryption	List of encrypted secrets and targets that should be decrypted before using.	No
attachment	Attachment	List of attachment sources and targets that should be attached to manifests before using.	No
timeout	duration	The maximum length of time to execute deployment before giving up. Default is 6h.	No
notification	DeploymentNotification	Additional configuration used while sending notification to external services.	No
postSync	PostSync	Additional configuration used as extra actions once the deployment is triggered.	No
eventWatcher	[]EventWatcher	List of configurations for event watcher.	No

Cloud Run application

apiVersion: pipecd.dev/v1beta1
kind: CloudRunApp
spec:
  input:
  pipeline:
  ...

Field	Type	Description	Required
name	string	The application name.	Yes if you set the application through the application configuration file
labels	map[string]string	Additional attributes to identify applications.	No
description	string	Notes on the Application.	No
input	CloudRunDeploymentInput	Input for Cloud Run deployment such as docker image…	No
trigger	DeploymentTrigger	Configuration for trigger used to determine should we trigger a new deployment or not.	No
planner	DeploymentPlanner	Configuration for planner used while planning deployment.	No
quickSync	CloudRunQuickSync	Configuration for quick sync.	No
pipeline	Pipeline	Pipeline for deploying progressively.	No
encryption	SecretEncryption	List of encrypted secrets and targets that should be decrypted before using.	No
attachment	Attachment	List of attachment sources and targets that should be attached to manifests before using.	No
timeout	duration	The maximum length of time to execute deployment before giving up. Default is 6h.	No
notification	DeploymentNotification	Additional configuration used while sending notification to external services.	No
postSync	PostSync	Additional configuration used as extra actions once the deployment is triggered.	No
eventWatcher	[]EventWatcher	List of configurations for event watcher.	No

Lambda application

apiVersion: pipecd.dev/v1beta1
kind: LambdaApp
spec:
  pipeline:
  ...

Field	Type	Description	Required
name	string	The application name.	Yes if you set the application through the application configuration file
labels	map[string]string	Additional attributes to identify applications.	No
description	string	Notes on the Application.	No
input	LambdaDeploymentInput	Input for Lambda deployment such as path to function manifest file…	No
architectures	[]string	Specific architecture for which a function supports (Default x86_64).	No
trigger	DeploymentTrigger	Configuration for trigger used to determine should we trigger a new deployment or not.	No
planner	DeploymentPlanner	Configuration for planner used while planning deployment.	No
quickSync	LambdaQuickSync	Configuration for quick sync.	No
pipeline	Pipeline	Pipeline for deploying progressively.	No
encryption	SecretEncryption	List of encrypted secrets and targets that should be decrypted before using.	No
attachment	Attachment	List of attachment sources and targets that should be attached to manifests before using.	No
timeout	duration	The maximum length of time to execute deployment before giving up. Default is 6h.	No
notification	DeploymentNotification	Additional configuration used while sending notification to external services.	No
postSync	PostSync	Additional configuration used as extra actions once the deployment is triggered.	No
eventWatcher	[]EventWatcher	List of configurations for event watcher.	No

ECS application

apiVersion: pipecd.dev/v1beta1
kind: ECSApp
spec:
  input:
  pipeline:
  ...

Field	Type	Description	Required
name	string	The application name.	Yes if you set the application through the application configuration file
labels	map[string]string	Additional attributes to identify applications.	No
description	string	Notes on the Application.	No
input	ECSDeploymentInput	Input for ECS deployment such as path to TaskDefinition, Service…	No
trigger	DeploymentTrigger	Configuration for trigger used to determine should we trigger a new deployment or not.	No
planner	DeploymentPlanner	Configuration for planner used while planning deployment.	No
quickSync	ECSQuickSync	Configuration for quick sync.	No
pipeline	Pipeline	Pipeline for deploying progressively.	No
encryption	SecretEncryption	List of encrypted secrets and targets that should be decrypted before using.	No
attachment	Attachment	List of attachment sources and targets that should be attached to manifests before using.	No
timeout	duration	The maximum length of time to execute deployment before giving up. Default is 6h.	No
notification	DeploymentNotification	Additional configuration used while sending notification to external services.	No
postSync	PostSync	Additional configuration used as extra actions once the deployment is triggered.	No
eventWatcher	[]EventWatcher	List of configurations for event watcher.	No

Analysis Template Configuration

apiVersion: pipecd.dev/v1beta1
kind: AnalysisTemplate
spec:
  metrics:
    grpc_error_rate_percentage:
      interval: 1m
      provider: prometheus-dev
      failureLimit: 1
      expected:
        max: 10
      query: awesome_query

Field	Type	Description	Required
metrics	map[string]AnalysisMetrics	Template for metrics.	No

Event Watcher Configuration (deprecated)

apiVersion: pipecd.dev/v1beta1
kind: EventWatcher
spec:
  events:
    - name: helloworld-image-update
      replacements:
        - file: helloworld/deployment.yaml
          yamlField: $.spec.template.spec.containers[0].image

Field	Type	Description	Required
name	string	The event name.	Yes
labels	map[string]string	Additional attributes of event. This can make an event definition unique even if the one with the same name exists.	No
replacements	[]EventWatcherReplacement	List of places where will be replaced when the new event matches.	Yes

EventWatcherReplacement

One of yamlField or regex is required.

Field	Type	Description	Required
file	string	The relative path from the repository root to the file to be updated.	Yes
yamlField	string	The yaml path to the field to be updated. It requires to start with `$` which represents the root element. e.g. `$.foo.bar[0].baz`.	No
regex	string	The regex string that specify what should be replaced. The only first capturing group enclosed by `()` will be replaced with the new value. e.g. `host.xz/foo/bar:(v[0-9].[0-9].[0-9])`, `host.xz/foo/bar:([0-9a-z]+)`	No

CommitMatcher

Field	Type	Description	Required
quickSync	string	Regular expression string to forcibly do QuickSync when it matches the commit message.	No
pipeline	string	Regular expression string to forcibly do Pipeline when it matches the commit message.	No

SecretEncryption

Field	Type	Description	Required
encryptedSecrets	map[string]string	List of encrypted secrets.	No
decryptionTargets	[]string	List of files to be decrypted before using.	No

Attachment

Field	Type	Description	Required
sources	map[string]string	List of attaching files with key is its refer name.	No
targets	[]string	List of files which should contain the attachments.	No

DeploymentPlanner

Field	Type	Description	Required
alwaysUsePipeline	bool	Always use the defined pipeline to deploy the application in all deployments. Default is `false`.	No

DeploymentTrigger

Field	Type	Description	Required
onCommit	OnCommit	Controls triggering new deployment when new Git commits touched the application.	No
onCommand	OnCommand	Controls triggering new deployment when received a new `SYNC` command.	No
onOutOfSync	OnOutOfSync	Controls triggering new deployment when application is at `OUT_OF_SYNC` state.	No
onChain	OnChain	Controls triggering new deployment when the application is counted as a node of some chains.	No

OnCommit

Field	Type	Description	Required
disabled	bool	Whether to exclude application from triggering target when new Git commits touched it. Default is `false`.	No
paths	[]string	List of directories or files where any changes of them will be considered as touching the application. Regular expression can be used. Empty means watching all changes under the application directory.	No
ignores	[]string	List of directories or files where any changes of them will NOT be considered as touching the application. Regular expression can be used. This config has a higher priority compare to `paths`.	No

OnCommand

Field	Type	Description	Required
disabled	bool	Whether to exclude application from triggering target when received a new `SYNC` command. Default is `false`.	No

OnOutOfSync

Field	Type	Description	Required
disabled	bool	Whether to exclude application from triggering target when application is at `OUT_OF_SYNC` state. Default is `true`.	No
minWindow	duration	Minimum amount of time must be elapsed since the last deployment. This can be used to avoid triggering unnecessary continuous deployments based on `OUT_OF_SYNC` status. Default is `5m`.	No

OnChain

Field	Type	Description	Required
disabled	bool	Whether to exclude application from triggering target when application is counted as a node of some chains. Default is `true`.	No

Pipeline

Field	Type	Description	Required
stages	[]PipelineStage	List of deployment pipeline stages.	No

PipelineStage

Field	Type	Description	Required
id	string	The unique ID of the stage. This field is `deprecated`.	No
name	string	One of the provided stage names.	Yes
desc	string	The description about the stage.	No
timeout	duration	The maximum time the stage can be taken to run.	No
with	StageOptions	Specific configuration for the stage. This must be one of these StageOptions.	No

DeploymentNotification

Field	Type	Description	Required
mentions	[]NotificationMention	List of users to be notified for each event.	No

NotificationMention

Field	Type	Description	Required
event	string	The event to be notified to users.	Yes
slack	[]string	Deprecated: Please use `slackUsers` instead. List of user IDs for mentioning in Slack. See here for more information on how to check them.	No
slackUsers	[]string	List of user IDs for mentioning in Slack. See here for more information on how to check them.	No
slackGroups	[]string	List of group IDs for mentioning in Slack. See here for more information on how to check them.	No

KubernetesDeploymentInput

Field	Type	Description	Required
manifests	[]string	List of manifest files in the application directory used to deploy. Empty means all manifest files in the directory will be used.	No
kubectlVersion	string	Version of kubectl will be used. Empty means the version set on piped config or default version will be used.	No
kustomizeVersion	string	Version of kustomize will be used. Empty means the default version will be used.	No
kustomizeOptions	map[string]string	List of options that should be used by Kustomize commands.	No
helmVersion	string	Version of helm will be used. Empty means the default version will be used.	No
helmChart	HelmChart	Where to fetch helm chart.	No
helmOptions	HelmOptions	Configurable parameters for helm commands.	No
namespace	string	The namespace where manifests will be applied.	No
autoRollback	bool	Automatically reverts all deployment changes on failure. Default is `true`.	No
autoCreateNamespace	bool	Automatically create a new namespace if it does not exist. Default is `false`.	No

HelmChart

Field	Type	Description	Required
gitRemote	string	Git remote address where the chart is placing. Empty means the same repository.	No
ref	string	The commit SHA or tag value. Only valid when gitRemote is not empty.	No
path	string	Relative path from the repository root to the chart directory.	No
repository	string	The name of a registered Helm Chart Repository.	No
name	string	The chart name.	No
version	string	The chart version. Only valid when using a chart repository (repository field).	No

HelmOptions

Field	Type	Description	Required
releaseName	string	The release name of helm deployment. By default, the release name is equal to the application name.	No
setValues	map[string]string	List of values.	No
valueFiles	[]string	List of value files should be loaded. Only local files stored under the application directory or remote files served at the http(s) endpoint are allowed.	No
setFiles	map[string]string	List of file path for values.	No
apiVersions	[]string	Kubernetes api versions used for Capabilities.APIVersions.	No
kubeVersion	string	Kubernetes version used for Capabilities.KubeVersion.	No

KubernetesVariantLabel

Field	Type	Description	Required
key	string	The key of the label. Default is `pipecd.dev/variant`.	No
primaryValue	string	The label value for PRIMARY variant. Default is `primary`.	No
canaryValue	string	The label value for CANARY variant. Default is `canary`.	No
baselineValue	string	The label value for BASELINE variant. Default is `baseline`.	No

KubernetesQuickSync

Field	Type	Description	Required
addVariantLabelToSelector	bool	Whether the PRIMARY variant label should be added to manifests if they were missing. Default is `false`.	No
prune	bool	Whether the resources that are no longer defined in Git should be removed or not. Default is `false`	No

KubernetesService

Field	Type	Description	Required
name	string	The name of Service manifest.	No

KubernetesWorkload

Field	Type	Description	Required
kind	string	The kind name of workload manifests. Currently, only `Deployment` is supported. In the future, we also want to support `ReplicationController`, `DaemonSet`, `StatefulSet`.	No
name	string	The name of workload manifest.	No

KubernetesTrafficRouting

Field	Type	Description	Required
method	string	Which traffic routing method will be used. Available values are `istio`, `smi`, `podselector`. Default is `podselector`.	No
istio	IstioTrafficRouting	Istio configuration when the method is `istio`.	No

IstioTrafficRouting

Field	Type	Description	Required
editableRoutes	[]string	List of routes in the VirtualService that can be changed to update traffic routing. Empty means all routes should be updated.	No
host	string	The service host.	No
virtualService	IstioVirtualService	The reference to VirtualService manifest. Empty means the first VirtualService resource will be used.	No

IstioVirtualService

Field	Type	Description	Required
name	string	The name of VirtualService manifest.	No

TerraformDeploymentInput

Field	Type	Description	Required
workspace	string	The terraform workspace name. Empty means `default` workspace.	No
terraformVersion	string	The version of terraform should be used. Empty means the pre-installed version will be used.	No
vars	[]string	List of variables that will be set directly on terraform commands with `-var` flag. The variable must be formatted by `key=value`.	No
varFiles	[]string	List of variable files that will be set on terraform commands with `-var-file` flag.	No
commandFlags	TerraformCommandFlags	List of additional flags will be used while executing terraform commands.	No
commandEnvs	TerraformCommandEnvs	List of additional environment variables will be used while executing terraform commands.	No
autoRollback	bool	Automatically reverts all changes from all stages when one of them failed.	No

TerraformCommandFlags

Field	Type	Description	Required
shared	[]string	List of additional flags used for all Terraform commands.	No
init	[]string	List of additional flags used for Terraform `init` command.	No
plan	[]string	List of additional flags used for Terraform `plan` command.	No
apply	[]string	List of additional flags used for Terraform `apply` command.	No

TerraformCommandEnvs

Field	Type	Description	Required
shared	[]string	List of additional environment variables used for all Terraform commands.	No
init	[]string	List of additional environment variables used for Terraform `init` command.	No
plan	[]string	List of additional environment variables used for Terraform `plan` command.	No
apply	[]string	List of additional environment variables used for Terraform `apply` command.	No

TerraformQuickSync

Field	Type	Description	Required

CloudRunDeploymentInput

Field	Type	Description	Required
serviceManifestFile	string	The name of service manifest file placing in application directory. Default is `service.yaml`.	No
autoRollback	bool	Automatically reverts to the previous state when the deployment is failed. Default is `true`.	No

CloudRunQuickSync

Field	Type	Description	Required

LambdaDeploymentInput

Field	Type	Description	Required
functionManifestFile	string	The name of function manifest file placing in application directory. Default is `function.yaml`.	No
autoRollback	bool	Automatically reverts to the previous state when the deployment is failed. Default is `true`.	No

Specific function.yaml

One of image, s3Bucket, or source is required.

If you use s3Bucket, s3Key and s3ObjectVersion are required.
If you use s3Bucket or source, handler and runtime are required.

See Configuring Lambda application for more details.

Field	Type	Description	Required
name	string	Name of the Lambda function	Yes
role	string	IAM role ARN	Yes
image	string	URI of the container image	No
s3Bucket	string	S3 bucket name for code package	No
s3Key	string	S3 key for code package	No
s3ObjectVersion	string	S3 object version for code package	No
source	source	Git settings	No
handler	string	Lambda function handler	No
runtime	string	Runtime environment	No
architectures	[]Architecture	Supported architectures	No
ephemeralStorage	EphemeralStorage	Ephemeral storage configuration	No
memory	int32	Memory allocation (in MB)	Yes
timeout	int32	Function timeout (in seconds)	Yes
tags	map[string]string	Key-value pairs for tags	No
environments	map[string]string	Environment variables	No
vpcConfig	VPCConfig	VPC configuration	No
layers	[]string	ARNs of layers to depend on	No

Source

Field	Type	Description	Required
git	string	Git repository URL	Yes
ref	string	Git branch/tag/reference	Yes
path	string	Path within the repository	Yes

Architecture

Field	Type	Description	Required
name	string	Name of the architecture	Yes

EphemeralStorage

Field	Type	Description	Required
size	int32	Size of the ephemeral storage	Yes

VPCConfig

Field	Type	Description	Required
securityGroupIds	[]string	List of security group IDs	No
subnetIds	[]string	List of subnet IDs	No

LambdaQuickSync

Field	Type	Description	Required

ECSDeploymentInput

Field	Type	Description	Required
serviceDefinitionFile	string	The path ECS Service configuration file. Allow file in both `yaml` and `json` format. The default value is `service.json`. See here and Restrictions for parameters.	No
taskDefinitionFile	string	The path to ECS TaskDefinition configuration file. Allow file in both `yaml` and `json` format. The default value is `taskdef.json`. See here and Restrictions for parameters.	No
targetGroups	ECSTargetGroupInput	The target groups configuration, will be used to routing traffic to created task sets.	Yes (if you want to perform progressive delivery)
runStandaloneTask	bool	Run standalone tasks during deployments. About standalone task, see here. The default value is `true`.
accessType	string	How the ECS service is accessed. One of `ELB` or `SERVICE_DISCOVERY`. See examples here. The default value is `ELB`.

Restrictions of Service Definition

There are some restrictions in configuring a service definition file.

As long as desiredCount is 0 or not set, desiredCount of your service will NOT be updated in deployments.
- If desiredCount is 0 or not set for a new service, the service’s desiredCount will be 0.
capacityProviderStrategy is not supported.
clientToken is not supported.
deploymentController is required and must be EXTERNAL.
loadBalancers is not supported. Use targetGroups in ECSDeploymentInput instead.
platformFamily is not supported.
taskDefinition is not supported. PipeCD uses the definition in taskDefinitionFile in ECSDeploymentInput.

Restrictions of Task Definition

There are some restrictions in configuring a task definition file.

tags is not supported.

ECSTargetGroupInput

Field	Type	Description	Required
primary	ECSTargetGroupObject	The PRIMARY target group, will be used to register the PRIMARY ECS task set.	Yes
canary	ECSTargetGroupObject	The CANARY target group, will be used to register the CANARY ECS task set if exist. It’s required to enable PipeCD to perform the multi-stage deployment.	No

ECSTargetGroupObject

Field	Type	Description	Required
targetGroupArn	string	The name of the container (as it appears in a container definition) to　associate with the load balancer	Yes
containerName	string	The full Amazon Resource Name (ARN) of the Elastic Load Balancing target group or groups associated with a service or task set.	Yes
containerPort	int	The port on the container to associate with the load balancer.	Yes
LoadBalancerName	string	The name of the load balancer to associate with the Amazon ECS service or task set.	No

Note: The available values are identical to those found in the aws-sdk-go-v2 Types.LoadBalancer. For more details, please refer to this link .

ECSQuickSync

Field	Type	Description	Required
recreate	bool	Whether to delete old tasksets before creating new ones or not. Default to false.	No

AnalysisMetrics

Field	Type	Description	Required
provider	string	The unique name of provider defined in the Piped Configuration.	Yes
strategy	string	The strategy name. One of `THRESHOLD` or `PREVIOUS` or `CANARY_BASELINE` or `CANARY_PRIMARY` is available. Defaults to `THRESHOLD`.	No
query	string	A query performed against the Analysis Provider. The stage will be skipped if no data points were returned.	Yes
expected	AnalysisExpected	The statically defined expected query result. This field is ignored if there was no data point as a result of the query.	Yes if the strategy is `THRESHOLD`
interval	duration	Run a query at specified intervals.	Yes
failureLimit	int	Acceptable number of failures. e.g. If 1 is set, the `ANALYSIS` stage will end with failure after two queries results failed. Defaults to 1.	No
skipOnNoData	bool	If true, it considers as a success when no data returned from the analysis provider. Defaults to false.	No
deviation	string	The stage fails on deviation in the specified direction. One of `LOW` or `HIGH` or `EITHER` is available. This can be used only for `PREVIOUS`, `CANARY_BASELINE` or `CANARY_PRIMARY`. Defaults to `EITHER`.	No
baselineArgs	map[string][string]	The custom arguments to be populated for the Baseline query. They can be referred as `{{ .VariantCustomArgs.xxx }}`.	No
canaryArgs	map[string][string]	The custom arguments to be populated for the Canary query. They can be referred as `{{ .VariantCustomArgs.xxx }}`.	No
primaryArgs	map[string][string]	The custom arguments to be populated for the Primary query. They can be referred as `{{ .VariantCustomArgs.xxx }}`.	No
timeout	duration	How long after which the query times out.	No
template	AnalysisTemplateRef	Reference to the template to be used.	No

AnalysisExpected

Field	Type	Description	Required
min	float64	Failure, if the query result is less than this value.	No
max	float64	Failure, if the query result is larger than this value.	No

AnalysisTemplateRef

Field	Type	Description	Required
name	string	The template name to refer.	Yes
appArgs	map[string]string	The arguments for custom-args.	No

AnalysisLog

Field	Type	Description	Required

AnalysisHttp

Field	Type	Description	Required

SkipOptions

Field	Type	Description	Required
commitMessagePrefixes	[]string	List of commit message’s prefixes. The stage will be skipped when the prefix of the commit’s message matches any of them. Empty means the stage will not be skipped by this condition.	No
paths	[]string	List of paths to directories or files. When all commit changes match them, the stage will be skipped. Empty means the stage will not be skipped by this condition. Regular expression can be used.	No

StageOptions

KubernetesPrimaryRolloutStageOptions

Field	Type	Description	Required
suffix	string	Suffix that should be used when naming the PRIMARY variant’s resources. Default is `primary`.	No
createService	bool	Whether the PRIMARY service should be created. Default is `false`.	No
addVariantLabelToSelector	bool	Whether the PRIMARY variant label should be added to manifests if they were missing. Default is `false`.	No
prune	bool	Whether the resources that are no longer defined in Git should be removed or not. Default is `false`	No

KubernetesCanaryRolloutStageOptions

Field	Type	Description	Required
replicas	int	How many pods for CANARY workloads. Default is `1` pod. Alternatively, can be specified a string suffixed by “%” to indicate a percentage value compared to the pod number of PRIMARY	No
suffix	string	Suffix that should be used when naming the CANARY variant’s resources. Default is `canary`.	No
createService	bool	Whether the CANARY service should be created. Default is `false`.	No
patches	[]KubernetesResourcePatch	List of patches used to customize manifests for CANARY variant.	No

KubernetesCanaryCleanStageOptions

Field	Type	Description	Required

KubernetesBaselineRolloutStageOptions

Field	Type	Description	Required
replicas	int	How many pods for BASELINE workloads. Default is `1` pod. Alternatively, can be specified a string suffixed by “%” to indicate a percentage value compared to the pod number of PRIMARY	No
suffix	string	Suffix that should be used when naming the BASELINE variant’s resources. Default is `baseline`.	No
createService	bool	Whether the BASELINE service should be created. Default is `false`.	No

KubernetesBaselineCleanStageOptions

Field	Type	Description	Required

KubernetesTrafficRoutingStageOptions

This stage routes traffic with the method specified in KubernetesTrafficRouting. When using podselector method as a traffic routing method, routing is done by updating the Service selector. Therefore, note that all traffic will be routed to the primary if the the primary variant’s service is rolled out by running the K8S_PRIMARY_ROLLOUT stage.

Field	Type	Description	Required
all	string	Which variant should receive all traffic. Available values are “primary”, “canary”, “baseline”. Default is `primary`.	No
primary	Percentage	The percentage of traffic should be routed to PRIMARY variant.	No
canary	Percentage	The percentage of traffic should be routed to CANARY variant.	No
baseline	Percentage	The percentage of traffic should be routed to BASELINE variant.	No

TerraformPlanStageOptions

Field	Type	Description	Required
exitOnNoChanges	bool	Whether exiting the pipeline when the result has no changes	No

TerraformApplyStageOptions

Field	Type	Description	Required

CloudRunPromoteStageOptions

Field	Type	Description	Required
percent	Percentage	Percentage of traffic should be routed to the new version.	No

LambdaCanaryRolloutStageOptions

Field	Type	Description	Required

LambdaPromoteStageOptions

Field	Type	Description	Required
percent	Percentage	Percentage of traffic should be routed to the new version.	No

ECSPrimaryRolloutStageOptions

Field	Type	Description	Required

ECSCanaryRolloutStageOptions

Field	Type	Description	Required
scale	Percentage	The percentage of workloads should be rolled out as CANARY variant’s workload.	Yes

ECSTrafficRoutingStageOptions

Field	Type	Description	Required
primary	Percentage	The percentage of traffic should be routed to PRIMARY variant.	No
canary	Percentage	The percentage of traffic should be routed to CANARY variant.	No

Note: By default, the sum of traffic is rounded to 100. If both primary and canary numbers are not set, the PRIMARY variant will receive 100% while the CANARY variant will receive 0% of the traffic.

AnalysisStageOptions

Field	Type	Description	Required
duration	duration	Maximum time to perform the analysis.	Yes
metrics	[]AnalysisMetrics	Configuration for analysis by metrics.	No
skipOn	SkipOptions	When to skip this stage.	No

WaitStageOptions

Field	Type	Description	Required
duration	duration	Time to wait.	Yes
skipOn	SkipOptions	When to skip this stage.	No

WaitApprovalStageOptions

Field	Type	Description	Required
timeout	duration	The maximum length of time to wait before giving up. Default is 6h.	No
approvers	[]string	List of username who has permission to approve.	Yes
minApproverNum	int	Number of minimum needed approvals to make this stage complete. Default is 1.	No
skipOn	SkipOptions	When to skip this stage.	No

CustomSyncStageOptions (deprecated)

Field	Type	Description	Required
timeout	duration	The maximum time the stage can be taken to run. Default is `6h`	No
envs	map[string]string	Environment variables used with scripts.	No
run	string	Script run on this stage.	Yes

ScriptRunStageOptions

Field	Type	Description	Required
run	string	Script run on this stage.	Yes
env	map[string]string	Environment variables used with scripts.	No
timeout	duration	The maximum time the stage can be taken to run. Default is `6h`	No
skipOn	SkipOptions	When to skip this stage.	No

PostSync

Field	Type	Description	Required
chain	DeploymentChain	Deployment chain configuration, used to determine and build deployments that should be triggered once the current deployment is triggered.	No

DeploymentChain

Field	Type	Description	Required
applications	[]DeploymentChainApplication	The list of applications which should be triggered once deployment of this application rolled out successfully.	Yes

DeploymentChainApplication

Field	Type	Description	Required
name	string	The name of PipeCD application, note that application name is not unique in PipeCD datastore	No
kind	string	The kind of the PipeCD application, which should be triggered as a node in deployment chain. The value will be one of: KUBERNETES, TERRAFORM, CLOUDRUN, LAMBDA, ECS.	No

EventWatcher

Field	Type	Description	Required
matcher	EventWatcherMatcher	Which event will be handled.	Yes
handler	EventWatcherHandler	What to do for the event which matched by the above matcher.	Yes

EventWatcherMatcher

Field	Type	Description	Required
name	string	The event name.	Yes
labels	map[string]string	Additional attributes of event. This can make an event definition unique even if the one with the same name exists.	No

EventWatcherHandler

Field	Type	Description	Required
type	string	The handler type. Currently, only `GIT_UPDATE` is supported.	Yes
config	EventWatcherHandlerConfig	Configuration for the event watcher handler.	Yes

EventWatcherHandlerConfig

Field	Type	Description	Required
commitMessage	string	The commit message used to push after replacing values. Default message is used if not given.	No
makePullRequest	bool	Whether to create a new branch or not when commit changes in event watcher. Default is `false`.	No
replacements	[]EventWatcherReplacement	List of places where will be replaced when the new event matches.	Yes

DriftDetection

Field	Type	Description	Required
ignoreFields	[]string	List of fields path in manifests, which its diff should be ignored. This is available for only `KubernetesApp`.	No

PipeCD rich defined types

Percentage

A wrapper of type int to represent percentage data. Basically, you can pass 10 or "10" or 10% and they will be treated as 10% in PipeCD.

KubernetesResourcePatch

Field	Type	Description	Required
target	KubernetesResourcePatchTarget	Which manifest, which field will be the target of patch operations.	Yes
ops	[]KubernetesResourcePatchOp	List of operations should be applied to the above target.	No

KubernetesResourcePatchTarget

Field	Type	Description	Required
kind	string	The resource kind. e.g. `ConfigMap`	Yes
name	string	The resource name. e.g. `config-map-name`	Yes
documentRoot	string	In case you want to manipulate the YAML or JSON data specified in a field of the manfiest, specify that field’s path. The string value of that field will be used as input for the patch operations. Otherwise, the whole manifest will be the target of patch operations. e.g. `$.data.envoy-config`	No

KubernetesResourcePatchOp

Field	Type	Description	Required
op	string	The operation type. This must be one of `yaml-replace`, `yaml-add`, `yaml-remove`, `json-replace`, `text-regex`. Default is `yaml-replace`.	No
path	string	The path string pointing to the manipulated field. For yaml operations it looks like `$.foo.array[0].bar`.	No
value	string	The value string whose content will be used as new value for the field.	No

Docs-V0.56.x: Examples

Mon, 01 Jan 0001 00:00:00 +0000

One of the best ways to see what PipeCD can do, and learn how to deploy your applications with it, is to see some real examples.

We have prepared some examples for each kind of application, please visit the PipeCD examples page for details.

PipeCD – User Guide

Docs-V0.56.x: Managing application

Docs-V0.56.x: Managing Piped

Docs-V0.56.x: Managing Control Plane

Docs-V0.56.x: Connect between CI and CD with event watcher

Prerequisites

Usage

1. Defining Events

Use the .pipe/ directory

Use the application configuration

2. Pushing an Event with pipectl

[optional] Using labels

[optional] Using contexts

[optional] Using commit-hash and commit-url for Deployment Trace

Examples

Github Actions

Docs-V0.56.x: Better connect between CI and CD with Deployment Trace

Usage

Github Actions

Docs-V0.56.x: Confidently review your changes with Plan Preview

Prerequisites

Usage

Order of the results

GitHub Actions

Docs-V0.56.x: Insights

Application metrics

Deployment metrics

Deployment Frequency

Change Failure Rate

Lead Time for Changes

Mean Time To Restore

Docs-V0.56.x: Metrics

Monitoring overview

Enable monitoring system

Dashboards

Control Plane dashboards

Piped dashboards

Cluster dashboards

Alert notifications

Piped agent metrics

Control plane metrics

Health Checking

Docs-V0.56.x: Command-line tool: pipectl

Installation

Directly download and set up binary

Using Asdf

Using Aqua

Using Homebrew

Run in Docker container

Authentication

Usage

Help

Adding a new application

Syncing an application

Getting an application

Listing applications

Disable an application

Deleting an application

List deployments

Waiting a deployment status

Get deployment stages log

Registering an event for EventWatcher

Encrypting the data you want to use when deploying

Generating an application config (app.pipecd.yaml)

You want more?

Docs-V0.56.x: PipeCD Terraform provider

Usage

Setup Terraform provider

Manage Piped agent

Adding a new application

You want more?

Docs-V0.56.x: Configuration reference

Kubernetes Application

Annotations

Terraform application

Cloud Run application

Lambda application

ECS application

Analysis Template Configuration

Event Watcher Configuration (deprecated)

Use the `.pipe/` directory

2. Pushing an Event with `pipectl`