Enable Secrets Management on an Existing BYOVPC Cluster on GCP

beta

BYOVPC is an add-on feature that may require an additional purchase. To unlock this feature for your account, contact your Redpanda account team or Redpanda Sales.

Storing secrets in your cluster allows you to keep your cloud infrastructure secure as you integrate your data across different systems, for example, REST catalogs with your Iceberg-enabled topics. If you do not have secrets management enabled on an existing BYOVPC cluster, you can do so by following the steps on this page to update your cluster configuration. You can also create a new BYOVPC cluster with secrets management already enabled.

Replace all <placeholders> with your own values.

  1. Create one new service account with the necessary permissions and roles.

    Show commands 
    # Account used to check for and read secrets

gcloud iam service-accounts create redpanda-operator \
  --display-name="Redpanda Operator Service Account"

cat << EOT > redpanda-operator.role
{
  "name": "redpanda_operator_role",
  "title": "Redpanda Operator Role",
  "description": "Redpanda Operator Role",
  "includedPermissions": [
    "resourcemanager.projects.get",
    "secretmanager.secrets.get",
    "secretmanager.versions.access"
  ]
}
EOT

gcloud iam roles create redpanda_operator_role --project=<service-project-id> --file redpanda-operator.role

gcloud projects add-iam-policy-binding <service-project-id> \
  --member="serviceAccount:redpanda-operator@<service-project-id>.iam.gserviceaccount.com" \
  --role="projects/<service-project-id>/roles/redpanda_operator_role"

  2. Update the existing Redpanda cluster service account with the necessary permissions to read secrets.

    Show commands 
    cat << EOT > redpanda-cluster.role
{
  "name": "redpanda_cluster_role",
  "title": "Redpanda Cluster Role",
  "description": "Redpanda Cluster Role",
  "includedPermissions": [
    "resourcemanager.projects.get",
    "secretmanager.secrets.get",
    "secretmanager.versions.access"
  ]
}
EOT

gcloud iam roles create redpanda_cluster_role --project=<service-project-id> --file redpanda-cluster.role

gcloud projects add-iam-policy-binding <service-project-id> \
  --member="serviceAccount:redpanda-cluster@<service-project-id>.iam.gserviceaccount.com" \
  --role="projects/<service-project-id>/roles/redpanda_cluster_role"

  3. Bind the new service account.

    The account ID of the GCP service account is used to configure service account bindings. This account ID is the local part of the email address for the GCP service account. For example, if the GCP service account is my-gcp-sa@my-project.iam.gserviceaccount.com, then the account ID is my-gcp-sa.

    Show commands 
    gcloud iam service-accounts add-iam-policy-binding <redpanda_operator-gcp-sa-account-id>@<service-project-id>.iam.gserviceaccount.com \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:<service-project-id>.svc.id.goog[redpanda-system/<redpanda_operator-gcp-sa-account-id>]"

  4. Make a PATCH /v1/clusters/{cluster-id} request to update the cluster configuration.

    Show request 
    export CLUSTER_PATCH_BODY=`cat << EOF
{
    "customer_managed_resources": {
        "gcp": {
            "redpanda_operator_service_account": {
              "email": "<redpanda_operator-gcp-sa-account-id>@<service-project-id>.iam.gserviceaccount.com"
            }
        }
    }
}
EOF`
curl -v -X PATCH \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $AUTH_TOKEN" \
-d "$CLUSTER_PATCH_BODY" $PUBLIC_API_ENDPOINT/v1/clusters/<cluster-id>

  5. Check secrets management is available in the Cloud UI.

    1. Log in to Redpanda Cloud.

    2. Go to the Secrets Store page of your cluster. You should be able to create a new secret.

Next steps