PipeCD – Managing Piped

Docs-V0.56.x: Remote upgrade and remote config

Mon, 01 Jan 0001 00:00:00 +0000

Remote upgrade

The remote upgrade is the ability to restart the currently running Piped with another version from the web console. This reduces the effort involved in updating Piped to newer versions. All Pipeds that are running by the provided Piped container image can be enabled to use this feature. It means Pipeds running on a Kubernetes cluster, a virtual machine, a serverless service can be upgraded remotely from the web console.

Basically, in order to use this feature you must run Piped with /launcher command instead of /piped command as usual. Please check the installation guide on each environment to see the details.

After starting Piped with the remote-upgrade feature, you can go to the Settings page then click on UPGRADE button on the top-right corner. A dialog will be shown for selecting which Pipeds you want to upgrade and what version they should run.

Select a list of Pipeds to upgrade from Settings page

Remote config

Although the remote-upgrade allows you remotely restart your Pipeds to run any new version you want, if your Piped is loading its config locally where Piped is running, you still need to manually restart Piped after adding any change on that config data. Remote-config is for you to remove that kind of manual operation.

Remote-config is the ability to load Piped config data from a remote location such as a Git repository. Not only that, but it also watches the config periodically to detect any changes on that config and restarts Piped to reflect the new configuration automatically.

This feature requires the remote-upgrade feature to be enabled simultaneously. Please check Runtime Options and the installation guide on each environment to know how to configure Piped to load a remote config file.

Summary

By remote-upgrade you can upgrade your Piped to a newer version by clicking on the web console
By remote-config you can enforce your Piped to use the latest config data just by updating its config file stored in a Git repository

Docs-V0.56.x: Adding a git repository

Mon, 01 Jan 0001 00:00:00 +0000

In the piped configuration file, we specify the list of Git repositories that should be handled by the piped. A Git repository contains one or more deployable applications where each application is put inside a directory called the application directory. That directory contains an application configuration file as well as application manifests. The piped periodically checks for new commits and fetches the needed manifests from those repositories for executing the deployment.

A single piped can be configured to handle one or more Git repositories. In order to enable a new Git repository, let’s add a new GitRepository block to the repositories field in the piped configuration file.

For example, with the following snippet, piped will take the master branch of pipe-cd/examples repository as a target Git repository for doing deployments.

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  ...
  repositories:
    - repoId: examples
      remote: git@github.com:pipe-cd/examples.git
      branch: master

In most of the cases, we want to deal with private Git repositories. For accessing those private repositories, piped needs a private SSH key, which can be configured while installing with secret.sshKey in the Helm chart.

helm install dev-piped pipecd/piped --version={VERSION} \
  --set-file config.data={PATH_TO_PIPED_CONFIG_FILE} \
  --set-file secret.data.piped-key={PATH_TO_PIPED_KEY_FILE} \
  --set-file secret.data.ssh-key={PATH_TO_PRIVATE_SSH_KEY_FILE}

You can see this configuration reference for more configurable fields about Git commands.

Currently, piped allows configuring only one private SSH key for all specified Git repositories. So you can configure the same SSH key for all of those private repositories, or break them into separate pipeds. In the near future, we also want to update piped to support loading multiple SSH keys.

Docs-V0.56.x: Adding a platform provider

Mon, 01 Jan 0001 00:00:00 +0000

PipeCD supports multiple platforms and multiple application kinds which run on those platforms. Platform provider defines which platform and where the application should be deployed to. So while registering a new application, the name of a configured platform provider is required.

Currently, PipeCD is supporting these five kinds of platform providers: KUBERNETES, ECS, TERRAFORM, CLOUDRUN, LAMBDA. A new platform provider can be enabled by adding a PlatformProvider struct to the piped configuration file. A piped can have one or multiple platform provider instances from the same or different platform provider kind.

The next sections show the specific configuration for each kind of platform provider.

Configuring Kubernetes platform provider

By default, piped deploys Kubernetes application to the cluster where the piped is running in. An external cluster can be connected by specifying the masterURL and kubeConfigPath in the configuration.

And, the default resources (defined at here) from all namespaces of the Kubernetes cluster will be watched for rendering the application state in realtime and detecting the configuration drift. In case you want to restrict piped to watch only a single namespace, let specify the namespace in the KubernetesAppStateInformer field. You can also add other resources or exclude resources to/from the watching targets by that field.

Below configuration snippet just specifies a name and type of platform provider. It means the platform provider kubernetes-dev will connect to the Kubernetes cluster where the piped is running in, and this platform provider watches all of the predefined resources from all namespaces inside that cluster.

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  ...
  platformProviders:
    - name: kubernetes-dev
      type: KUBERNETES

See ConfigurationReference for the full configuration.

Configuring Terraform platform provider

A terraform platform provider contains a list of shared terraform variables that will be applied while running the deployment of its applications.

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  ...
  platformProviders:
    - name: terraform-dev
      type: TERRAFORM
      config:
        vars:
          - "project=pipecd"

See ConfigurationReference for the full configuration.

Configuring Cloud Run platform provider

Adding a Cloud Run provider requires the name of the Google Cloud project and the region name where Cloud Run service is running. A service account file for accessing to Cloud Run is also required if the machine running the piped does not have enough permissions to access.

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  ...
  platformProviders:
    - name: cloudrun-dev
      type: CLOUDRUN
      config:
        project: {GCP_PROJECT}
        region: {CLOUDRUN_REGION}
        credentialsFile: {PATH_TO_THE_SERVICE_ACCOUNT_FILE}

See ConfigurationReference for the full configuration.

Configuring Lambda platform provider

Adding a Lambda provider requires the region name where Lambda service is running.

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  ...
  platformProviders:
    - name: lambda-dev
      type: LAMBDA
      config:
        region: {LAMBDA_REGION}
        profile: default
        credentialsFile: {PATH_TO_THE_CREDENTIAL_FILE}

You will generally need your AWS credentials to authenticate with Lambda. Piped provides multiple methods of loading these credentials. It attempts to retrieve credentials in the following order:

From the environment variables. Available environment variables are AWS_ACCESS_KEY_ID or AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY or AWS_SECRET_KEY.
From the given credentials file. (the credentialsFile field in above sample)
From the pod running in EKS cluster via STS (SecurityTokenService).
From the EC2 Instance Role.

Therefore, you don’t have to set credentialsFile if you use the environment variables or the EC2 Instance Role. Keep in mind the IAM role/user that you use with your Piped must possess the IAM policy permission for at least Lambda.Function and Lambda.Alias resources controll (list/read/write).

See ConfigurationReference for the full configuration.

Configuring ECS platform provider

Adding a ECS provider requires the region name where ECS cluster is running.

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  ...
  platformProviders:
    - name: ecs-dev
      type: ECS
      config:
        region: {ECS_CLUSTER_REGION}
        profile: default
        credentialsFile: {PATH_TO_THE_CREDENTIAL_FILE}

Just same as Lambda platform provider, there are several ways to authorize Piped agent to enable it performs deployment jobs. It attempts to retrieve credentials in the following order:

From the environment variables. Available environment variables are AWS_ACCESS_KEY_ID or AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY or AWS_SECRET_KEY.
From the given credentials file. (the credentialsFile field in above sample)
From the pod running in EKS cluster via STS (SecurityTokenService).
From the EC2 Instance Role.

See ConfigurationReference for the full configuration.

Docs-V0.56.x: Adding a Helm chart repository or registry

Mon, 01 Jan 0001 00:00:00 +0000

PipeCD supports Kubernetes applications that are using Helm for templating and packaging. In addition to being able to deploy a Helm chart that is sourced from the same Git repository (local chart) or from a different Git repository (remote git chart), an application can use a chart sourced from a Helm chart repository.

Adding Helm chart repository

A Helm chart repository is a location backed by an HTTP server where packaged charts can be stored and shared. Before an application can be configured to use a chart from a Helm chart repository, that chart repository must be enabled in the related piped by adding the ChartRepository struct to the piped configuration file.

# piped configuration file
apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  ...
  chartRepositories:
    - name: pipecd
      address: https://charts.pipecd.dev

For example, the above snippet enables the official chart repository of PipeCD project. After that, you can configure the Kubernetes application to load a chart from that chart repository for executing the deployment.

# Application configuration file.
apiVersion: pipecd.dev/v1beta1
kind: KubernetesApp
spec:
  input:
    # Helm chart sourced from a Helm Chart Repository.
    helmChart:
      repository: pipecd
      name: helloworld
      version: v0.5.0

In case the chart repository is backed by HTTP basic authentication, the username and password strings are required in configuration.

Adding Helm chart registry

A Helm chart registry is a mechanism enabled by default in Helm 3.8.0 and later that allows the OCI registry to be used for storage and distribution of Helm charts.

Before an application can be configured to use a chart from a registry, that registry must be enabled in the related piped by adding the ChartRegistry struct to the piped configuration file if authentication is enabled at the registry.

# piped configuration file
apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  ...
  chartRegistries:
    - type: OCI
      address: registry.example.com
      username: sample-username
      password: sample-password

Docs-V0.56.x: Adding an analysis provider

Mon, 01 Jan 0001 00:00:00 +0000

To enable Automated deployment analysis feature, you have to set the needed information for Piped to connect to the Analysis Provider.

Currently, PipeCD supports the following providers:

Prometheus

Piped queries the range query endpoint to obtain metrics used to evaluate the deployment.

You need to define the Prometheus server address accessible to Piped.

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  analysisProviders:
    - name: prometheus-dev
      type: PROMETHEUS
      config:
        address: https://your-prometheus.dev

The full list of configurable fields are here.

Datadog

Piped queries the MetricsApi.QueryMetrics endpoint to obtain metrics used to evaluate the deployment.

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  analysisProviders:
    - name: datadog-dev
      type: DATADOG
      config:
        apiKeyFile: /etc/piped-secret/datadog-api-key
        applicationKeyFile: /etc/piped-secret/datadog-application-key

The full list of configurable fields are here.

If you choose Helm as the installation method, we recommend using --set-file to mount the key files while performing the upgrading process.

--set-file secret.data.datadog-api-key={PATH_TO_API_KEY_FILE} \
--set-file secret.data.datadog-application-key={PATH_TO_APPLICATION_KEY_FILE}

Docs-V0.56.x: Configuring event watcher

Mon, 01 Jan 0001 00:00:00 +0000

To enable EventWatcher, you have to configure your piped at first.

Grant write permission

The SSH key used by Piped must be a key with write-access because piped needs to commit and push to your git repository when any incoming event matches.

Specify Git repositories to be observed

Piped watches events only for the Git repositories specified in the gitRepos list. You need to add all repositories you want to enable EventWatcher.

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  eventWatcher:
    gitRepos:
      - repoId: repo-1
      - repoId: repo-2
      - repoId: repo-3

[optional] Specify EventWatcher files Piped will use

NOTE: This way is valid only for defining events using .pipe/.

If multiple Pipeds handle a single repository, you can prevent conflicts by splitting into the multiple EventWatcher files and setting includes/excludes to specify the files that should be monitored by this Piped.

Say for instance, if you only want the Piped to use the EventWatcher files under .pipe/dev/:

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  eventWatcher:
    gitRepos:
      - repoId: repo-1
        commitMessage: Update values by Event watcher
        includes:
          - dev/*.yaml

excludes is prioritized if both includes and excludes are given.

The full list of configurable fields are here.

[optional] Settings for git user

By default, every git commit uses piped as a username and pipecd.dev@gmail.com as an email. You can change it with the git field.

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  git:
    username: foo
    email: foo@example.com

Docs-V0.56.x: Configuring notifications

Mon, 01 Jan 0001 00:00:00 +0000

PipeCD events (deployment triggered, planned, completed, analysis result, piped started…) can be sent to external services like Slack or a Webhook service. While forwarding those events to a chat service helps developers have a quick and convenient way to know the deployment’s current status, forwarding to a Webhook service may be useful for triggering other related tasks like CI jobs.

PipeCD events are emitted and sent by the piped component. So all the needed configurations can be specified in the piped configuration file. Notification configuration including:

a list of Routes which used to match events and decide where the event should be sent to
a list of Receivers which used to know how to send events to the external service

Notification Route matches events based on their metadata like name, group, app, labels. Below is the list of supporting event names and their groups.

Event	Group	Description
DEPLOYMENT_TRIGGERED	DEPLOYMENT
DEPLOYMENT_PLANNED	DEPLOYMENT
DEPLOYMENT_STARTED	DEPLOYMENT
DEPLOYMENT_APPROVED	DEPLOYMENT
DEPLOYMENT_WAIT_APPROVAL	DEPLOYMENT
DEPLOYMENT_ROLLING_BACK	DEPLOYMENT	PipeCD sends a notification when a deployment is completed, while it does not send a notification when a deployment status changes to DEPLOYMENT_ROLLING_BACK because it is not a completion status. See #4547
DEPLOYMENT_SUCCEEDED	DEPLOYMENT
DEPLOYMENT_FAILED	DEPLOYMENT
DEPLOYMENT_CANCELLED	DEPLOYMENT
DEPLOYMENT_TRIGGER_FAILED	DEPLOYMENT
APPLICATION_SYNCED	APPLICATION_SYNC
APPLICATION_OUT_OF_SYNC	APPLICATION_SYNC
APPLICATION_HEALTHY	APPLICATION_HEALTH
APPLICATION_UNHEALTHY	APPLICATION_HEALTH
PIPED_STARTED	PIPED
PIPED_STOPPED	PIPED

Sending notifications to Slack

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  notifications:
    routes:
      # Sending all event which contains labels `env: dev` to dev-slack-channel.
      - name: dev-slack
        labels:
          env: dev
        receiver: dev-slack-channel
      # Only sending deployment started and completed events which contains
      # labels `env: prod` and `team: pipecd` to prod-slack-channel.
      - name: prod-slack
        events:
          - DEPLOYMENT_TRIGGERED
          - DEPLOYMENT_SUCCEEDED
        labels:
          env: prod
          team: pipecd
        receiver: prod-slack-channel
      - name: integration-slack
        receiver: integration-slack-api
    receivers:
      - name: dev-slack-channel
        slack:
          hookURL: https://slack.com/dev
      - name: prod-slack-channel
        slack:
          hookURL: https://slack.com/prod
      - name: integration-slack-api
        slack:
          oauthTokenData: "token"
          channelID: "testid"
      - name: hookurl-with-mentioned-accounts
        slack:
          hookURL: https://slack.com/dev,
          mentionedAccounts:
            - '@user1'
            - 'user2'
      - name: integration-slack-api-with-mentioned-accounts
        slack:
          oauthTokenData: token
          channelID: testid
          mentionedAccounts:
            - '@user1'
            - 'user2'
      - name: integration-slack-api-with-oauthTokenData-and-mentioned-groups
        slack:
          oauthTokenData: token
          channelID: testid
          mentionedGroups:
            - 'group1'
            - '<!subteam^group2>'
      - name: integration-slack-api-with-oauthTokenData-and-mentioned-both-accounts-and-groups
        slack:
          oauthTokenData: token
          channelID: testid
          mentionedAccounts:
            - 'user1'
            - '@user2'
          mentionedGroups:
            - 'groupID1'
            - '<!subteam^groupID2>'

Deployment was triggered, planned and completed successfully

A piped has been started

For detailed configuration, please check the configuration reference for Notifications section.

Sending notifications to external services via webhook

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  notifications:
    routes:
      # Sending all events an external service.
      - name: all-events-to-a-external-service
        receiver: a-webhook-service
    receivers:
      - name: a-webhook-service
        webhook:
          url: {WEBHOOK_SERVICE_URL}
          signatureValue: {RANDOM_SIGNATURE_STRING}

For detailed configuration, please check the configuration reference for NotificationReceiverWebhook section.

Docs-V0.56.x: Configuration reference

Mon, 01 Jan 0001 00:00:00 +0000

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  projectID: ...
  pipedID: ...
  ...

Piped Configuration

Field	Type	Description	Required
projectID	string	The identifier of the PipeCD project where this piped belongs to.	Yes
pipedID	string	The generated ID for this piped.	Yes
pipedKeyFile	string	The path to the file containing the generated key string for this piped.	Yes
pipedKeyData	string	Base64 encoded string of Piped key. Either pipedKeyFile or pipedKeyData must be set.	Yes
apiAddress	string	The address used to connect to the Control Plane’s API in format `host:port`.	Yes
syncInterval	duration	How often to check whether an application should be synced. Default is `1m`.	No
appConfigSyncInterval	duration	How often to check whether application configuration files should be synced. Default is `1m`.	No
git	Git	Git configuration needed for Git commands.	No
repositories	[]Repository	List of Git repositories this piped will handle.	No
chartRepositories	[]ChartRepository	List of Helm chart repositories that should be added while starting up.	No
chartRegistries	[]ChartRegistry	List of helm chart registries that should be logged in while starting up.	No
platformProviders	[]PlatformProvider	List of platform providers can be used by this piped.	No
analysisProviders	[]AnalysisProvider	List of analysis providers can be used by this piped.	No
eventWatcher	EventWatcher	Optional Event watcher settings.	No
secretManagement	SecretManagement	The using secret management method.	No
notifications	Notifications	Sending notifications to Slack, Webhook…	No
appSelector	map[string]string	List of labels to filter all applications this piped will handle. Currently, it is only be used to filter the applications suggested for adding from the control plane.	No

Git

Field	Type	Description	Required
username	string	The username that will be configured for `git` user. Default is `piped`.	No
email	string	The email that will be configured for `git` user. Default is `pipecd.dev@gmail.com`.	No
sshConfigFilePath	string	Where to write ssh config file. Default is `$HOME/.ssh/config`.	No
host	string	The host name. Default is `github.com`.	No
hostName	string	The hostname or IP address of the remote git server. Default is the same value with Host.	No
sshKeyFile	string	The path to the private ssh key file. This will be used to clone the source code of the specified git repositories.	No
sshKeyData	string	Base64 encoded string of SSH key.	No
password	string	The base64 encoded password for git used while cloning above Git repository.	No

GitRepository

Field	Type	Description	Required
repoID	string	Unique identifier to the repository. This must be unique in the piped scope.	Yes
remote	string	Remote address of the repository used to clone the source code. e.g. `git@github.com:org/repo.git`	Yes
branch	string	The branch will be handled.	Yes

ChartRepository

Field	Type	Description	Required
type	string	The repository type. Currently, HTTP and GIT are supported. Default is HTTP.	No
name	string	The name of the Helm chart repository. Note that is not a Git repository but a Helm chart repository.	Yes if type is HTTP
address	string	The address to the Helm chart repository.	Yes if type is HTTP
username	string	Username used for the repository backed by HTTP basic authentication.	No
password	string	Password used for the repository backed by HTTP basic authentication.	No
insecure	bool	Whether to skip TLS certificate checks for the repository or not.	No
gitRemote	string	Remote address of the Git repository used to clone Helm charts.	Yes if type is GIT
sshKeyFile	string	The path to the private ssh key file used while cloning Helm charts from above Git repository.	No

ChartRegistry

Field	Type	Description	Required
type	string	The registry type. Currently, only OCI is supported. Default is OCI.	No
address	string	The address to the registry.	Yes
username	string	Username used for the registry authentication.	No
password	string	Password used for the registry authentication.	No

PlatformProvider

Field	Type	Description	Required
name	string	The name of the platform provider.	Yes
type	string	The platform provider type. Must be one of the following values: `KUBERNETES`, `TERRAFORM`, `ECS`, `CLOUDRUN`, `LAMBDA`.	Yes
config	PlatformProviderConfig	Specific configuration for the specified type of platform provider.	No

PlatformProviderConfig

Must be one of the following structs:

PlatformProviderKubernetesConfig

Field	Type	Description	Required
masterURL	string	The master URL of the kubernetes cluster. Empty means in-cluster.	No
kubectlVersion	string	Version of kubectl which will be used to connect to your cluster. Empty means the version set on piped config or default version will be used.	No
kubeConfigPath	string	The path to the kubeconfig file. Empty means in-cluster.	No
appStateInformer	KubernetesAppStateInformer	Configuration for application resource informer.	No

PlatformProviderTerraformConfig

Field	Type	Description	Required
vars	[]string	List of variables that will be set directly on terraform commands with `-var` flag. The variable must be formatted by `key=value`.	No
driftDetectionEnabled	bool	Enable drift detection. This is a temporary option and will be possibly removed in the future release. Default is `true`	No

PlatformProviderCloudRunConfig

Field	Type	Description	Required
project	string	The GCP project hosting the Cloud Run service.	Yes
region	string	The region of running Cloud Run service.	Yes
credentialsFile	string	The path to the service account file for accessing Cloud Run service.	No

PlatformProviderLambdaConfig

Field	Type	Description	Required
region	string	The region of running Lambda service.	Yes
credentialsFile	string	The path to the credential file for logging into AWS cluster. If this value is not provided, piped will read credential info from environment variables. It expects the format ~/.aws/credentials.	No
roleARN	string	The IAM role arn to use when assuming an role. Required if you want to use the AWS SecurityTokenService.	No
tokenFile	string	The path to the WebIdentity token the SDK should use to assume a role with. Required if you want to use the AWS SecurityTokenService.	No
profile	string	The profile to use for logging into AWS cluster. The default value is `default`.	No
awsAPIPollingInterval	duration	The interval of periodical calls of AWS APIs. Currently, this is an interval of refreshing the live state of Lambda functions. Default is 15s.	No

PlatformProviderECSConfig

Field	Type	Description	Required
region	string	The region of running ECS cluster.	Yes
credentialsFile	string	The path to the credential file for logging into AWS cluster. If this value is not provided, piped will read credential info from environment variables. It expects the format ~/.aws/credentials	No
roleARN	string	The IAM role arn to use when assuming an role. Required if you want to use the AWS SecurityTokenService.	No
tokenFile	string	The path to the WebIdentity token the SDK should use to assume a role with. Required if you want to use the AWS SecurityTokenService.	No
profile	string	The profile to use for logging into AWS cluster. The default value is `default`.	No

KubernetesAppStateInformer

Field	Type	Description	Required
namespace	string	Only watches the specified namespace. Empty means watching all namespaces.	No
includeResources	[]KubernetesResourcematcher	List of resources that should be added to the watching targets.	No
excludeResources	[]KubernetesResourcematcher	List of resources that should be ignored from the watching targets.	No

KubernetesResourceMatcher

Field	Type	Description	Required
apiVersion	string	The APIVersion of the kubernetes resource.	Yes
kind	string	The kind name of the kubernetes resource. Empty means all kinds are matching.	No

AnalysisProvider

Field	Type	Description	Required
name	string	The unique name of the analysis provider.	Yes
type	string	The provider type. Currently, only PROMETHEUS, DATADOG are available.	Yes
config	AnalysisProviderConfig	Specific configuration for the specified type of analysis provider.	Yes

AnalysisProviderConfig

Must be one of the following structs:

AnalysisProviderPrometheusConfig

Field	Type	Description	Required
address	string	The Prometheus server address.	Yes
usernameFile	string	The path to the username file.	No
passwordFile	string	The path to the password file.	No

AnalysisProviderDatadogConfig

Field	Type	Description	Required
address	string	The address of Datadog API server. Only “datadoghq.com”, “us3.datadoghq.com”, “datadoghq.eu”, “ddog-gov.com” are available. Defaults to “datadoghq.com”	No
apiKeyFile	string	The path to the api key file.	Yes
applicationKeyFile	string	The path to the application key file.	Yes
apiKeyData	string	Base64 API Key for Datadog API server. Either apiKeyData or apiKeyFile must be set	No
applicationKeyData	string	Base64 Application Key for Datadog API server. Either applicationKeyFile or applicationKeyData must be set	No

EventWatcher

Field	Type	Description	Required
checkInterval	duration	Interval to fetch the latest event and compare it with one defined in EventWatcher config files. Defaults to `1m`.	No
gitRepos	[]EventWatcherGitRepo	The configuration list of git repositories to be observed. Only the repositories in this list will be observed by Piped.	No

EventWatcherGitRepo

Field	Type	Description	Required
repoId	string	Id of the git repository. This must be unique within the repos’ elements.	Yes
commitMessage	string	The commit message used to push after replacing values. Default message is used if not given.	No
includes	[]string	The paths to EventWatcher files to be included. Patterns can be used like `foo/*.yaml`.	No
excludes	[]string	The paths to EventWatcher files to be excluded. Patterns can be used like `foo/*.yaml`. This is prioritized if both includes and this are given.	No

SecretManagement

Field	Type	Description	Required
type	string	Which management method should be used. Default is `KEY_PAIR`.	Yes
config	SecretManagementConfig	Configuration for using secret management method.	Yes

SecretManagementConfig

Must be one of the following structs:

SecretManagementKeyPair

Field	Type	Description	Required
privateKeyFile	string	Path to the private RSA key file.	Yes
privateKeyData	string	Base64 encoded string of private RSA key. Either privateKeyFile or privateKeyData must be set.	No
publicKeyFile	string	Path to the public RSA key file.	Yes
publicKeyData	string	Base64 encoded string of public RSA key. Either publicKeyFile or publicKeyData must be set.	No

SecretManagementGCPKMS

WIP

Notifications

Field	Type	Description	Required
routes	[]NotificationRoute	List of notification routes.	No
receivers	[]NotificationReceiver	List of notification receivers.	No

NotificationRoute

Field	Type	Description	Required
name	string	The name of the route.	Yes
receiver	string	The name of receiver who will receive all matched events.	Yes
events	[]string	List of events that should be routed to the receiver.	No
ignoreEvents	[]string	List of events that should be ignored.	No
groups	[]string	List of event groups should be routed to the receiver.	No
ignoreGroups	[]string	List of event groups should be ignored.	No
apps	[]string	List of applications where their events should be routed to the receiver.	No
ignoreApps	[]string	List of applications where their events should be ignored.	No
labels	map[string]string	List of labels where their events should be routed to the receiver.	No
ignoreLabels	map[string]string	List of labels where their events should be ignored.	No

NotificationReceiver

Field	Type	Description	Required
name	string	The name of the receiver.	Yes
slack	NotificationReceiverSlack	Configuration for slack receiver.	No
webhook	NotificationReceiverWebhook	Configuration for webhook receiver.	No

NotificationReceiverSlack

Field	Type	Description	Required
hookURL	string	The hookURL of a slack channel.	Yes
oauthToken	string	The token for Slack API use. (deprecated)	No
oauthTokenData	string	Base64 encoded string of The token for Slack API use.	No
oauthTokenFile	string	The path to the oauthToken file	No
channelID	string	The channel id which slack api send to.	No
mentionedAccounts	[]string	The accounts to which slack api refers. This field supports both `@username` and `username` writing styles.	No
mentionedGroups	[]string	The groups to which slack api refers. This field supports both `<!subteam^groupname>` and `groupname` writing styles.	No

NotificationReceiverWebhook

Field	Type	Description	Required
url	string	The URL where notification event will be sent to.	Yes
signatureKey	string	The HTTP header key used to store the configured signature in each event. Default is “PipeCD-Signature”.	No
signatureValue	string	The value of signature included in header of each event request. It can be used to verify the received events.	No
signatureValueFile	string	The path to the signature value file.	No

Docs-V0.56.x: Using Pprof in Piped

Mon, 01 Jan 0001 00:00:00 +0000

Piped provides built-in support for pprof, a tool for visualization and analysis of profiling data. It’s a part of the standard Go library.

In Piped, several routes are registered to serve the profiling data in a format understood by the pprof tool. Here are the routes:

/debug/pprof/: This route serves an index page that lists the available profiling data.
/debug/pprof/profile: This route serves CPU profiling data.
/debug/pprof/trace: This route serves execution trace data.

You can access these routes to get the profiling data. For example, to get the CPU profiling data, you can access the /debug/pprof/profile route.

Note that using these features in a production environment may impact performance.

This document explains the basic usage of pprof in Piped. For more detailed information or specific use cases, please refer to the official Go documentation.

How to use pprof

Access the pprof index page
```
curl http://localhost:9085/debug/pprof/
```
This will return an HTML page that lists the available profiling data.
Get the CPU Profile
```
curl http://localhost:9085/debug/pprof/profile > cpu.pprof
```
This will save the CPU profiling data to a file named cpu.pprof. You can then analyze this data using the pprof tool:
```
go tool pprof cpu.pprof
```
Get the Execution Trace
```
curl http://localhost:9085/debug/pprof/trace > trace.out
```
This will save the execution trace data to a file named trace.out. You can then view this trace using the go tool trace command:
```
go tool trace trace.out
```
Please replace localhost:9085 with the actual address and port of your Piped’s admin server.

Docs-V0.56.x: Runtime Options

Mon, 01 Jan 0001 00:00:00 +0000

You can configure some options when running Piped and launcher.

Options for Piped

Usage:
  piped piped [flags]

Flags:
      --add-login-user-to-passwd                   Whether to add login user to $HOME/passwd. This is typically for applications running as a random user ID.
      --admin-port int                             The port number used to run a HTTP server for admin tasks such as metrics, healthz. (default 9085)
      --app-manifest-cache-count int               The number of app manifests to cache. The cache-key contains the commit hash. The default is 150. (default 150)     
      --cert-file string                           The path to the TLS certificate file.
      --config-aws-secret string                   The ARN of secret that contains Piped config and be stored in AWS Secrets Manager.
      --config-aws-ssm-parameter string            The name of parameter of Piped config stored in AWS Systems Manager Parameter Store. SecureString is also supported.
      --config-data string                         The base64 encoded string of the configuration data.
      --config-file string                         The path to the configuration file.
      --config-gcp-secret string                   The resource ID of secret that contains Piped config and be stored in GCP SecretManager.
      --enable-default-kubernetes-cloud-provider   Whether the default kubernetes provider is enabled or not. This feature is deprecated.
      --grace-period duration                      How long to wait for graceful shutdown. (default 30s)
  -h, --help                                       help for piped
      --insecure                                   Whether disabling transport security while connecting to control-plane.
      --launcher-version string                    The version of launcher which initialized this Piped.
      --tools-dir string                           The path to directory where to install needed tools such as kubectl, helm, kustomize. (default "~/.piped/tools")

Global Flags:
      --log-encoding string                The encoding type for logger [json|console|humanize]. (default "humanize")
      --log-level string                   The minimum enabled logging level. (default "info")
      --metrics                            Whether metrics is enabled or not. (default true)
      --profile                            If true enables uploading the profiles to Stackdriver.
      --profile-debug-logging              If true enables logging debug information of profiler.
      --profiler-credentials-file string   The path to the credentials file using while sending profiles to Stackdriver.

Options for launcher

Usage:
  launcher launcher [flags]

Flags:
      --aws-secret-id string                  The ARN of secret that contains Piped config in AWS Secrets Manager service.
      --aws-ssm-parameter string              The name of parameter of Piped config stored in AWS Systems Manager Parameter Store. SecureString is also supported.
      --cert-file string                      The path to the TLS certificate file.
      --check-interval duration               Interval to periodically check desired config/version to restart Piped. Default is 1m. (default 1m0s)
      --config-data string                    The base64 encoded string of the configuration data.
      --config-file string                    The path to the configuration file.
      --config-from-aws-secret                Whether to load Piped config that is being stored in AWS Secrets Manager service.
      --config-from-aws-ssm-parameter-store   Whether to load Piped config that is being stored in AWS Systems Manager Parameter Store.
      --config-from-gcp-secret                Whether to load Piped config that is being stored in GCP SecretManager service.
      --config-from-git-repo                  Whether to load Piped config that is being stored in a git repository.
      --default-version string                The version should be run when no desired version was specified. Empty means using the same version with Launcher.
      --gcp-secret-id string                  The resource ID of secret that contains Piped config in GCP SecretManager service.
      --git-branch string                     Branch of git repository to for Piped config.
      --git-piped-config-file string          Relative path within git repository to locate Piped config file.
      --git-repo-url string                   The remote URL of git repository to fetch Piped config.
      --git-ssh-key-data string               Base64 encoded value of SSH private key to fetch Piped config from the private git repository.
      --git-ssh-key-file string               The path to SSH private key to fetch Piped config from private git repository.
      --grace-period duration                 How long to wait for graceful shutdown. (default 30s)
  -h, --help                                  help for launcher
      --home-dir string                       The working directory of Launcher.
      --insecure                              Whether disabling transport security while connecting to control-plane.
      --launcher-admin-port int               The port number used to run a HTTP server for admin tasks such as metrics, healthz.

Global Flags:
      --log-encoding string                The encoding type for logger [json|console|humanize]. (default "humanize")
      --log-level string                   The minimum enabled logging level. (default "info")
      --metrics                            Whether metrics is enabled or not. (default true)
      --profile                            If true enables uploading the profiles to Stackdriver.
      --profile-debug-logging              If true enables logging debug information of profiler.
      --profiler-credentials-file string   The path to the credentials file using while sending profiles to Stackdriver.