PipeCD – Install Piped

Docs-V0.56.x: Installing on Kubernetes cluster

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

Having piped’s ID and Key strings

Ensure that the piped has been registered and you are having its PIPED_ID and PIPED_KEY strings.
If you are not having them, this page guides you how to register a new one.

Preparing SSH key

If your Git repositories are private, piped requires a private SSH key to access those repositories.
Please checkout this documentation for how to generate a new SSH key pair. Then add the public key to your repositories. (If you are using GitHub, you can add it to Deploy Keys at the repository’s Settings page.)

In the cluster-wide mode

This way requires installing cluster-level resources. Piped installed with this way can perform deployment workloads against any other namespaces than the where Piped runs on.

Preparing a piped configuration file as the following

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  projectID: {PROJECT_ID}
  pipedID: {PIPED_ID}
  pipedKeyFile: /etc/piped-secret/piped-key
  # Write in a format like "host:443" because the communication is done via gRPC.
  apiAddress: {CONTROL_PLANE_API_ADDRESS}
  git:
    sshKeyFile: /etc/piped-secret/ssh-key
  repositories:
    - repoId: {REPO_ID_OR_NAME}
      remote: git@github.com:{GIT_ORG}/{GIT_REPO}.git
      branch: {GIT_BRANCH}
  syncInterval: 1m

See ConfigurationReference for the full configuration.

Installing by using Helm (3.8.0 or later)

# This command just installs a Piped with the specified version.
# Whenever you want to upgrade that Piped to a new version or update its config data
# you have to restart it by re-running this command.

helm upgrade -i dev-piped oci://ghcr.io/pipe-cd/chart/piped --version=v0.56.0 --namespace={NAMESPACE} \
  --set-file config.data={PATH_TO_PIPED_CONFIG_FILE} \
  --set-file secret.data.piped-key={PATH_TO_PIPED_KEY_FILE} \
  --set-file secret.data.ssh-key={PATH_TO_PRIVATE_SSH_KEY_FILE}

# Enable remote-upgrade feature of Piped.
# https://pipecd.dev/docs/user-guide/managing-piped/remote-upgrade-remote-config/#remote-upgrade
# This allows upgrading Piped to a new version from the web console.
# But we still need to restart Piped when we want to update its config data.

helm upgrade -i dev-piped oci://ghcr.io/pipe-cd/chart/piped --version=v0.56.0 --namespace={NAMESPACE} \
  --set launcher.enabled=true \
  --set-file config.data={PATH_TO_PIPED_CONFIG_FILE} \
  --set-file secret.data.piped-key={PATH_TO_PIPED_KEY_FILE} \
  --set-file secret.data.ssh-key={PATH_TO_PRIVATE_SSH_KEY_FILE}

# Enable both remote-upgrade and remote-config features of Piped.
# https://pipecd.dev/docs/user-guide/managing-piped/remote-upgrade-remote-config/#remote-config
# Beside of the ability to upgrade Piped to a new version from the web console,
# remote-config allows loading the Piped config stored in a remote location such as a Git repository.
# Whenever the config data is changed, it loads the new config and restarts Piped to use that new config.

helm upgrade -i dev-piped oci://ghcr.io/pipe-cd/chart/piped --version=v0.56.0 --namespace={NAMESPACE} \
  --set launcher.enabled=true \
  --set launcher.configFromGitRepo.enabled=true \
  --set launcher.configFromGitRepo.repoUrl=git@github.com:{GIT_ORG}/{GIT_REPO}.git \
  --set launcher.configFromGitRepo.branch={GIT_BRANCH} \
  --set launcher.configFromGitRepo.configFile={RELATIVE_PATH_TO_PIPED_CONFIG_FILE_IN_GIT_REPO} \
  --set launcher.configFromGitRepo.sshKeyFile=/etc/piped-secret/ssh-key \
  --set-file secret.data.piped-key={PATH_TO_PIPED_KEY_FILE} \
  --set-file secret.data.ssh-key={PATH_TO_PRIVATE_SSH_KEY_FILE}

Note: Be sure to set --set args.insecure=true if your Control Plane has not TLS-enabled yet.

See values.yaml for the full values.

In the namespaced mode

The previous way requires installing cluster-level resources. If you want to restrict Piped’s permission within the namespace where Piped runs on, this way is for you. Most parts are identical to the previous way, but some are slightly different.

Adding a new cloud provider like below to the previous piped configuration file

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  projectID: {PROJECT_ID}
  pipedID: {PIPED_ID}
  pipedKeyFile: /etc/piped-secret/piped-key
  # Write in a format like "host:443" because the communication is done via gRPC.
  apiAddress: {CONTROL_PLANE_API_ADDRESS}
  git:
    sshKeyFile: /etc/piped-secret/ssh-key
  repositories:
    - repoId: REPO_ID_OR_NAME
      remote: git@github.com:{GIT_ORG}/{GIT_REPO}.git
      branch: {GIT_BRANCH}
  syncInterval: 1m
  # This is needed to restrict to limit the access range to within a namespace.
  platformProviders:
    - name: my-kubernetes
      type: KUBERNETES
      config:
        appStateInformer:
          namespace: {NAMESPACE}

Installing by using Helm (3.8.0 or later)

# This command just installs a Piped with the specified version.
# Whenever you want to upgrade that Piped to a new version or update its config data
# you have to restart it by re-running this command.

helm upgrade -i dev-piped oci://ghcr.io/pipe-cd/chart/piped --version=v0.56.0 --namespace={NAMESPACE} \
  --set-file config.data={PATH_TO_PIPED_CONFIG_FILE} \
  --set-file secret.data.piped-key={PATH_TO_PIPED_KEY_FILE} \
  --set-file secret.data.ssh-key={PATH_TO_PRIVATE_SSH_KEY_FILE} \
  --set args.enableDefaultKubernetesCloudProvider=false \
  --set rbac.scope=namespace

# Enable remote-upgrade feature of Piped.
# https://pipecd.dev/docs/user-guide/managing-piped/remote-upgrade-remote-config/#remote-upgrade
# This allows upgrading Piped to a new version from the web console.
# But we still need to restart Piped when we want to update its config data.

helm upgrade -i dev-piped oci://ghcr.io/pipe-cd/chart/piped --version=v0.56.0 --namespace={NAMESPACE} \
  --set launcher.enabled=true \
  --set-file config.data={PATH_TO_PIPED_CONFIG_FILE} \
  --set-file secret.data.piped-key={PATH_TO_PIPED_KEY_FILE} \
  --set-file secret.data.ssh-key={PATH_TO_PRIVATE_SSH_KEY_FILE} \
  --set args.enableDefaultKubernetesCloudProvider=false \
  --set rbac.scope=namespace

# Enable both remote-upgrade and remote-config features of Piped.
# https://pipecd.dev/docs/user-guide/managing-piped/remote-upgrade-remote-config/#remote-config
# Beside of the ability to upgrade Piped to a new version from the web console,
# remote-config allows loading the Piped config stored in a remote location such as a Git repository.
# Whenever the config data is changed, it loads the new config and restarts Piped to use that new config.

helm upgrade -i dev-piped oci://ghcr.io/pipe-cd/chart/piped --version=v0.56.0 --namespace={NAMESPACE} \
  --set launcher.enabled=true \
  --set launcher.configFromGitRepo.enabled=true \
  --set launcher.configFromGitRepo.repoUrl=git@github.com:{GIT_ORG}/{GIT_REPO}.git \
  --set launcher.configFromGitRepo.branch={GIT_BRANCH} \
  --set launcher.configFromGitRepo.configFile={RELATIVE_PATH_TO_PIPED_CONFIG_FILE_IN_GIT_REPO} \
  --set launcher.configFromGitRepo.sshKeyFile=/etc/piped-secret/ssh-key \
  --set-file secret.data.piped-key={PATH_TO_PIPED_KEY_FILE} \
  --set-file secret.data.ssh-key={PATH_TO_PRIVATE_SSH_KEY_FILE} \
  --set args.enableDefaultKubernetesCloudProvider=false \
  --set rbac.scope=namespace

In case on OpenShift less than 4.2

OpenShift uses an arbitrarily assigned user ID when it starts a container. Starting from OpenShift 4.2, it also inserts that user into /etc/passwd for using by the application inside the container, but before that version, the assigned user is missing in that file. That blocks workloads of ghcr.io/pipe-cd/piped image. Therefore if you are running on OpenShift with a version before 4.2, please use ghcr.io/pipe-cd/piped-okd image with the following command:

Installing by using Helm (3.8.0 or later)

# This command just installs a Piped with the specified version.
# Whenever you want to upgrade that Piped to a new version or update its config data
# you have to restart it by re-running this command.

helm upgrade -i dev-piped oci://ghcr.io/pipe-cd/chart/piped --version=v0.56.0 --namespace={NAMESPACE} \
  --set-file config.data={PATH_TO_PIPED_CONFIG_FILE} \
  --set-file secret.data.piped-key={PATH_TO_PIPED_KEY_FILE} \
  --set-file secret.data.ssh-key={PATH_TO_PRIVATE_SSH_KEY_FILE} \
  --set args.enableDefaultKubernetesCloudProvider=false \
  --set rbac.scope=namespace
  --set args.addLoginUserToPasswd=true \
  --set securityContext.runAsNonRoot=true \
  --set securityContext.runAsUser={UID} \
  --set securityContext.fsGroup={FS_GROUP} \
  --set securityContext.runAsGroup=0 \
  --set image.repository="ghcr.io/pipe-cd/piped-okd"

# Enable remote-upgrade feature of Piped.
# https://pipecd.dev/docs/user-guide/managing-piped/remote-upgrade-remote-config/#remote-upgrade
# This allows upgrading Piped to a new version from the web console.
# But we still need to restart Piped when we want to update its config data.

helm upgrade -i dev-piped oci://ghcr.io/pipe-cd/chart/piped --version=v0.56.0 --namespace={NAMESPACE} \
  --set launcher.enabled=true \
  --set-file config.data={PATH_TO_PIPED_CONFIG_FILE} \
  --set-file secret.data.piped-key={PATH_TO_PIPED_KEY_FILE} \
  --set-file secret.data.ssh-key={PATH_TO_PRIVATE_SSH_KEY_FILE} \
  --set args.enableDefaultKubernetesCloudProvider=false \
  --set rbac.scope=namespace
  --set args.addLoginUserToPasswd=true \
  --set securityContext.runAsNonRoot=true \
  --set securityContext.runAsUser={UID} \
  --set securityContext.fsGroup={FS_GROUP} \
  --set securityContext.runAsGroup=0 \
  --set launcher.image.repository="ghcr.io/pipe-cd/launcher-okd"

# Enable both remote-upgrade and remote-config features of Piped.
# https://pipecd.dev/docs/user-guide/managing-piped/remote-upgrade-remote-config/#remote-config
# Beside of the ability to upgrade Piped to a new version from the web console,
# remote-config allows loading the Piped config stored in a remote location such as a Git repository.
# Whenever the config data is changed, it loads the new config and restarts Piped to use that new config.

helm upgrade -i dev-piped oci://ghcr.io/pipe-cd/chart/piped --version=v0.56.0 --namespace={NAMESPACE} \
  --set launcher.enabled=true \
  --set launcher.configFromGitRepo.enabled=true \
  --set launcher.configFromGitRepo.repoUrl=git@github.com:{GIT_ORG}/{GIT_REPO}.git \
  --set launcher.configFromGitRepo.branch={GIT_BRANCH} \
  --set launcher.configFromGitRepo.configFile={RELATIVE_PATH_TO_PIPED_CONFIG_FILE_IN_GIT_REPO} \
  --set launcher.configFromGitRepo.sshKeyFile=/etc/piped-secret/ssh-key \
  --set-file secret.data.piped-key={PATH_TO_PIPED_KEY_FILE} \
  --set-file secret.data.ssh-key={PATH_TO_PRIVATE_SSH_KEY_FILE} \
  --set args.enableDefaultKubernetesCloudProvider=false \
  --set rbac.scope=namespace
  --set args.addLoginUserToPasswd=true \
  --set securityContext.runAsNonRoot=true \
  --set securityContext.runAsUser={UID} \
  --set securityContext.fsGroup={FS_GROUP} \
  --set securityContext.runAsGroup=0 \
  --set launcher.image.repository="ghcr.io/pipe-cd/launcher-okd"

Docs-V0.56.x: Installing on Google Cloud VM

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

Having piped’s ID and Key strings

Ensure that the piped has been registered and you are having its PIPED_ID and PIPED_KEY strings.
If you are not having them, this page guides you how to register a new one.

Preparing SSH key

If your Git repositories are private, piped requires a private SSH key to access those repositories.
Please checkout this documentation for how to generate a new SSH key pair. Then add the public key to your repositories. (If you are using GitHub, you can add it to Deploy Keys at the repository’s Settings page.)

Installation

Preparing a piped configuration file as the following:

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  projectID: {PROJECT_ID}
  pipedID: {PIPED_ID}
  pipedKeyData: {BASE64_ENCODED_PIPED_KEY}
  # Write in a format like "host:443" because the communication is done via gRPC.
  apiAddress: {CONTROL_PLANE_API_ADDRESS}

  git:
    sshKeyData: {BASE64_ENCODED_PRIVATE_SSH_KEY}

  repositories:
    - repoId: {REPO_ID_OR_NAME}
      remote: git@github.com:{GIT_ORG}/{GIT_REPO}.git
      branch: {GIT_BRANCH}

  # Optional
  # Uncomment this if you want to enable this Piped to handle Cloud Run application.
  # platformProviders:
  #  - name: cloudrun-in-project
  #    type: CLOUDRUN
  #    config:
  #      project: {GCP_PROJECT_ID}
  #      region: {GCP_PROJECT_REGION}

  # Optional
  # Uncomment this if you want to enable this Piped to handle Terraform application.
  #  - name: terraform-gcp
  #    type: TERRAFORM

  # Optional
  # Uncomment this if you want to enable SecretManagement feature.
  # https://pipecd.dev//docs/user-guide/managing-application/secret-management/
  # secretManagement:
  #   type: KEY_PAIR
  #   config:
  #     privateKeyData: {BASE64_ENCODED_PRIVATE_KEY}
  #     publicKeyData: {BASE64_ENCODED_PUBLIC_KEY}

See ConfigurationReference for the full configuration.

Creating a new secret in SecretManager to store above configuration data securely
```
gcloud secrets create vm-piped-config --data-file={PATH_TO_CONFIG_FILE}
```

Creating a new Service Account for Piped and giving it needed roles

gcloud iam service-accounts create vm-piped \
  --description="Using by Piped running on Google Cloud VM" \
  --display-name="vm-piped"

# Allow Piped to access the created secret.
gcloud secrets add-iam-policy-binding vm-piped-config \
  --member="serviceAccount:vm-piped@{GCP_PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor"

# Allow Piped to write its log messages to Google Cloud Logging service.
gcloud projects add-iam-policy-binding {GCP_PROJECT_ID} \
  --member="serviceAccount:vm-piped@{GCP_PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/logging.logWriter"

# Optional
# If you want to use this Piped to handle Cloud Run application
# run the following command to give it the needed roles.
# https://cloud.google.com/run/docs/reference/iam/roles#additional-configuration
#
# gcloud projects add-iam-policy-binding {GCP_PROJECT_ID} \
#   --member="serviceAccount:vm-piped@{GCP_PROJECT_ID}.iam.gserviceaccount.com" \
#   --role="roles/run.developer"
#
# gcloud iam service-accounts add-iam-policy-binding {GCP_PROJECT_NUMBER}-compute@developer.gserviceaccount.com \
#   --member="serviceAccount:vm-piped@{GCP_PROJECT_ID}.iam.gserviceaccount.com" \
#   --role="roles/iam.serviceAccountUser"

Running Piped on a Google Cloud VM

# Enable remote-upgrade feature of Piped.
# https://pipecd.dev/docs/user-guide/managing-piped/remote-upgrade-remote-config/#remote-upgrade
# This allows upgrading Piped to a new version from the web console.

  gcloud compute instances create-with-container vm-piped \
    --container-image="ghcr.io/pipe-cd/launcher:v0.56.0" \
    --container-arg="launcher" \
    --container-arg="--config-from-gcp-secret=true" \
    --container-arg="--gcp-secret-id=projects/{GCP_PROJECT_ID}/secrets/vm-piped-config/versions/{SECRET_VERSION}" \
    --network="{VPC_NETWORK}" \
    --subnet="{VPC_SUBNET}" \
    --scopes="cloud-platform" \
    --service-account="vm-piped@{GCP_PROJECT_ID}.iam.gserviceaccount.com"

# This just installs a Piped with the specified version.
# Whenever you want to upgrade that Piped to a new version or update its config data you have to restart it.

  gcloud compute instances create-with-container vm-piped \
    --container-image="ghcr.io/pipe-cd/piped:v0.56.0" \
    --container-arg="piped" \
    --container-arg="--config-gcp-secret=projects/{GCP_PROJECT_ID}/secrets/vm-piped-config/versions/{SECRET_VERSION}" \
    --network="{VPC_NETWORK}" \
    --subnet="{VPC_SUBNET}" \
    --scopes="cloud-platform" \
    --service-account="vm-piped@{GCP_PROJECT_ID}.iam.gserviceaccount.com"

After that, you can see on PipeCD web at Settings page that Piped is connecting to the Control Plane. You can also view Piped log as described here.

Docs-V0.56.x: Installing on Cloud Run

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

Having piped’s ID and Key strings

Ensure that the piped has been registered and you are having its PIPED_ID and PIPED_KEY strings.
If you are not having them, this page guides you how to register a new one.

Preparing SSH key

If your Git repositories are private, piped requires a private SSH key to access those repositories.
Please checkout this documentation for how to generate a new SSH key pair. Then add the public key to your repositories. (If you are using GitHub, you can add it to Deploy Keys at the repository’s Settings page.)

Installation

Preparing a piped configuration file as the following:

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  projectID: {PROJECT_ID}
  pipedID: {PIPED_ID}
  pipedKeyData: {BASE64_ENCODED_PIPED_KEY}
  # Write in a format like "host:443" because the communication is done via gRPC.
  apiAddress: {CONTROL_PLANE_API_ADDRESS}

  git:
    sshKeyData: {BASE64_ENCODED_PRIVATE_SSH_KEY}

  repositories:
    - repoId: {REPO_ID_OR_NAME}
      remote: git@github.com:{GIT_ORG}/{GIT_REPO}.git
      branch: {GIT_BRANCH}

  # Optional
  # Enable this Piped to handle Cloud Run application.
  platformProviders:
    - name: cloudrun-in-project
      type: CLOUDRUN
      config:
        project: {GCP_PROJECT_ID}
        region: {GCP_PROJECT_REGION}

  # Optional
  # Uncomment this if you want to enable this Piped to handle Terraform application.
  #  - name: terraform-gcp
  #    type: TERRAFORM

  # Optional
  # Uncomment this if you want to enable SecretManagement feature.
  # https://pipecd.dev//docs/user-guide/managing-application/secret-management/
  # secretManagement:
  #   type: KEY_PAIR
  #   config:
  #     privateKeyData: {BASE64_ENCODED_PRIVATE_KEY}
  #     publicKeyData: {BASE64_ENCODED_PUBLIC_KEY}

See ConfigurationReference for the full configuration.

Creating a new secret in SecretManager to store above configuration data securely
```
gcloud secrets create cloudrun-piped-config --data-file={PATH_TO_CONFIG_FILE}
```
then make sure that Cloud Run has the ability to access that secret as this guide.

Running Piped in Cloud Run

Prepare a Cloud Run service manifest file as below.

# Enable remote-upgrade feature of Piped.
# https://pipecd.dev/docs/user-guide/managing-piped/remote-upgrade-remote-config/#remote-upgrade
# This allows upgrading Piped to a new version from the web console.

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: piped
  annotaions:
    run.googleapis.com/ingress: internal
    run.googleapis.com/ingress-status: internal
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/maxScale: '1'           # This must be 1.
        autoscaling.knative.dev/minScale: '1'           # This must be 1.
        run.googleapis.com/cpu-throttling: "false"      # This is required.
    spec:
      containerConcurrency: 1                           # This must be 1 to ensure Piped work correctly.
      containers:
        - image: gcr.io/pipecd/launcher:v0.56.0
          args:
            - launcher
            - --launcher-admin-port=9086
            - --config-file=/etc/piped-config/config.yaml
          ports:
            - containerPort: 9086
          volumeMounts:
            - mountPath: /etc/piped-config
              name: piped-config
          resources:
            limits:
              cpu: 1000m
              memory: 2Gi
      volumes:
        - name: piped-config
          secret:
            secretName: cloudrun-piped-config
            items:
              - path: config.yaml
                key: latest

# This just installs a Piped with the specified version.
# Whenever you want to upgrade that Piped to a new version or update its config data you have to restart it.

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: piped
  annotaions:
    run.googleapis.com/ingress: internal
    run.googleapis.com/ingress-status: internal
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/maxScale: '1'           # This must be 1.
        autoscaling.knative.dev/minScale: '1'           # This must be 1.
        run.googleapis.com/cpu-throttling: "false"      # This is required.
    spec:
      containerConcurrency: 1                           # This must be 1.
      containers:
        - image: gcr.io/pipecd/piped:v0.56.0
          args:
            - piped
            - --config-file=/etc/piped-config/config.yaml
          ports:
            - containerPort: 9085
          volumeMounts:
            - mountPath: /etc/piped-config
              name: piped-config
          resources:
            limits:
              cpu: 1000m
              memory: 2Gi
      volumes:
        - name: piped-config
          secret:
            secretName: cloudrun-piped-config
            items:
              - path: config.yaml
                key: latest

Run Piped service on Cloud Run with the following command:

gcloud beta run services replace cloudrun-piped-service.yaml

Note: Make sure that the created secret is accessible from this Piped service. See more here.

Docs-V0.56.x: Installing on ECS Fargate

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

Having piped’s ID and Key strings

Ensure that the piped has been registered and you are having its PIPED_ID and PIPED_KEY strings.
If you are not having them, this page guides you how to register a new one.

Preparing SSH key

If your Git repositories are private, piped requires a private SSH key to access those repositories.
Please checkout this documentation for how to generate a new SSH key pair. Then add the public key to your repositories. (If you are using GitHub, you can add it to Deploy Keys at the repository’s Settings page.)

Installation

Preparing a piped configuration file as follows:

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  projectID: {PROJECT_ID}
  pipedID: {PIPED_ID}
  pipedKeyData: {BASE64_ENCODED_PIPED_KEY}
  # Write in a format like "host:443" because the communication is done via gRPC.
  apiAddress: {CONTROL_PLANE_API_ADDRESS}

  git:
    sshKeyData: {BASE64_ENCODED_PRIVATE_SSH_KEY}

  repositories:
    - repoId: {REPO_ID_OR_NAME}
      remote: git@github.com:{GIT_ORG}/{GIT_REPO}.git
      branch: {GIT_BRANCH}

  # Optional
  # Enable this Piped to handle ECS application.
  platformProviders:
    - name: ecs-dev
      type: ECS
      config:
        region: {ECS_RUNNING_REGION}

  # Optional
  # Uncomment this if you want to enable this Piped to handle Terraform application.
  #  - name: terraform-dev
  #    type: TERRAFORM

  # Optional
  # Uncomment this if you want to enable SecretManagement feature.
  # https://pipecd.dev//docs/user-guide/managing-application/secret-management/
  # secretManagement:
  #   type: KEY_PAIR
  #   config:
  #     privateKeyData: {BASE64_ENCODED_PRIVATE_KEY}
  #     publicKeyData: {BASE64_ENCODED_PUBLIC_KEY}

See ConfigurationReference for the full configuration.

Store the above configuration data to AWS to enable using it while creating piped task. Both AWS Secrets Manager and AWS Systems Manager Parameter Store are available to address this task.

  aws secretsmanager create-secret --name PipedConfig \
    --description "Configuration of piped running as ECS Fargate task" \
    --secret-string `base64 piped-config.yaml`

  aws ssm put-parameter \
    --name PipedConfig \
    --value `base64 piped-config.yaml` \
    --type SecureString

Prepare task definition for your piped task. Basically, you can just define your piped TaskDefinition as normal TaskDefinition, the only thing that needs to be beware is, in case you used AWS Secrets Manager to store piped configuration, to enable your piped accesses it’s configuration we created as a secret on above, you need to add secretsmanager:GetSecretValue policy to your piped task executionRole. Read more in Required IAM permissions for Amazon ECS secrets.

A sample TaskDefinition for Piped as follows

# Enable remote-upgrade feature of Piped.
# https://pipecd.dev/docs/user-guide/managing-piped/remote-upgrade-remote-config/#remote-upgrade
# This allows upgrading Piped to a new version from the web console.

{
  "family": "piped",
  "executionRoleArn": "{PIPED_TASK_EXECUTION_ROLE_ARN}",
  "containerDefinitions": [
    {
      "name": "piped",
      "essential": true,
      "image": "ghcr.io/pipe-cd/launcher:v0.56.0",
      "entryPoint": [
        "sh",
        "-c"
      ],
      "command": [
        "/bin/sh -c \"launcher launcher --config-data=$(echo $CONFIG_DATA)\""
      ],
      "secrets": [
        {
          "valueFrom": "{PIPED_SECRET_MANAGER_ARN}",
          "name": "CONFIG_DATA"
        }
      ],
    }
  ],
  "requiresCompatibilities": [
    "FARGATE"
  ],
  "networkMode": "awsvpc",
  "memory": "512",
  "cpu": "256"
}

# This just installs a Piped with the specified version.
# Whenever you want to upgrade that Piped to a new version or update its config data you have to restart it.

{
  "family": "piped",
  "executionRoleArn": "{PIPED_TASK_EXECUTION_ROLE_ARN}",
  "containerDefinitions": [
    {
      "name": "piped",
      "essential": true,
      "image": "ghcr.io/pipe-cd/piped:v0.56.0",
      "entryPoint": [
        "sh",
        "-c"
      ],
      "command": [
        "/bin/sh -c \"piped piped --config-data=$(echo $CONFIG_DATA)\""
      ],
      "secrets": [
        {
          "valueFrom": "{PIPED_SECRET_MANAGER_ARN}",
          "name": "CONFIG_DATA"
        }
      ],
    }
  ],
  "requiresCompatibilities": [
    "FARGATE"
  ],
  "networkMode": "awsvpc",
  "memory": "512",
  "cpu": "256"
}

aws ecs register-task-definition --cli-input-json file://taskdef.json
aws ecs run-task --cluster {ECS_CLUSTER} --task-definition piped

Once the task is created, it will run continuously because of the piped execution. Since this task is run without startedBy setting, in case the piped is stopped, it will not automatically be restarted. To do so, you must define an ECS service to control piped task deployment.

A sample Service definition to control piped task deployment.

{
  "cluster": "{ECS_CLUSTER}",
  "serviceName": "piped",
  "desiredCount": 1, # This must be 1.
  "taskDefinition": "{PIPED_TASK_DEFINITION_ARN}",
  "deploymentConfiguration": {
    "minimumHealthyPercent": 0,
    "maximumPercent": 100
  },
  "schedulingStrategy": "REPLICA",
  "launchType": "FARGATE",
  "networkConfiguration": {
    "awsvpcConfiguration": {
      "assignPublicIp": "ENABLED", # This is need to enable ECS deployment to pull piped container images.
      ...
    }
  }
}

Then start your piped task controller service.

aws ecs create-service \
  --cluster {ECS_CLUSTER} \
  --service-name piped \
  --cli-input-json file://service.json

Docs-V0.56.x: Installing on a single machine

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

Having piped’s ID and Key strings

Ensure that the piped has been registered and you are having its PIPED_ID and PIPED_KEY strings.
If you are not having them, this page guides you how to register a new one.

Preparing SSH key

If your Git repositories are private, piped requires a private SSH key to access those repositories.
Please checkout this documentation for how to generate a new SSH key pair. Then add the public key to your repositories. (If you are using GitHub, you can add it to Deploy Keys at the repository’s Settings page.)

Installation

Downloading the latest piped binary for your machine

https://github.com/pipe-cd/pipecd/releases

Preparing a piped configuration file as the following:

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
  projectID: {PROJECT_ID}
  pipedID: {PIPED_ID}
  pipedKeyFile: {PATH_TO_PIPED_KEY_FILE}
  # Write in a format like "host:443" because the communication is done via gRPC.
  apiAddress: {CONTROL_PLANE_API_ADDRESS}
  git:
    sshKeyFile: {PATH_TO_SSH_KEY_FILE}
  repositories:
    - repoId: {REPO_ID_OR_NAME}
      remote: git@github.com:{GIT_ORG}/{GIT_REPO}.git
      branch: {GIT_BRANCH}
  syncInterval: 1m

See ConfigurationReference for the full configuration.

Start running the piped

./piped piped --config-file={PATH_TO_PIPED_CONFIG_FILE}

Docs-V0.56.x: Required Permissions

Mon, 01 Jan 0001 00:00:00 +0000

A Piped requires some permissions to deploy applications, depending on the platform.

Note: If you run a piped as an ECS task, you need to attach the permissions on the piped task’s task role, not task execution role.

For ECSApp

You need IAM actions like the following example. You can restrict Resource.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecs:CreateService",
                "ecs:CreateTaskSet",
                "ecs:DeleteTaskSet",
                "ecs:DeregisterTaskDefinition",
                "ecs:DescribeServices",
                "ecs:DescribeTaskDefinition",
                "ecs:DescribeTaskSets",
                "ecs:DescribeTasks",
                "ecs:ListClusters",
                "ecs:ListServices",
                "ecs:ListTasks",
                "ecs:ListTagsForResource",
                "ecs:RegisterTaskDefinition",
                "ecs:RunTask",
                "ecs:TagResource",
                "ecs:UntagResource",
                "ecs:UpdateService",
                "ecs:UpdateServicePrimaryTaskSet",
                "elasticloadbalancing:DescribeListeners",
                "elasticloadbalancing:DescribeRules",
                "elasticloadbalancing:DescribeTargetGroups",
                "elasticloadbalancing:ModifyListener",
                "elasticloadbalancing:ModifyRule"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:PassRole"
            ],
            "Resource": [
                "arn:aws:iam::<account-id>:role/<task-execution-role>",
                "arn:aws:iam::<account-id>:role/<task-role>"
            ]
        }
    ]
}

For LambdaApp

You need IAM actions like the following example. You can restrict Resource.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "lambda:CreateAlias",
                "lambda:CreateFunction",
                "lambda:GetAlias",
                "lambda:GetFunction",
                "lambda:ListFunctions",
                "lambda:PublishVersion",
                "lambda:TagResource",
                "lambda:UntagResource",
                "lambda:UpdateAlias",
                "lambda:UpdateFunctionCode",
                "lambda:UpdateFunctionConfiguration"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:PassRole"
            ],
            "Resource": [
                "arn:aws:iam::<account-id>:role/<function-role>"
            ]
        }
    ]
}