Operator Fails to Inject secretKeyRef.name When Using auth.sasl.bootstrapUser

[calinah]

When configuring a Redpanda cluster with SASL (SCRAM) authentication via auth.sasl.bootstrapUser – and referencing a pre-existing Secret (rather than storing the password in plaintext) – the Operator fails to populate the secretKeyRef.name field for environment variables in the main container and initContainers. As a result, those containers end up with:

valueFrom:
  secretKeyRef:
    key: password
    # name: missing or ""

This empty name triggers Kubernetes validation errors (Invalid value: "": a lowercase RFC 1123 subdomain must consist of...), leaving the StatefulSet stuck because new Pods cannot be created.
Below is an example snippet from an initContainer where the issue appears:

env:
  - name: RPK_PASS
    valueFrom:
      secretKeyRef:
        key: password
  - name: RPK_USER
    value: alice

and the related CR definition:

auth:
  sasl:
    enabled: true
    mechanism: SCRAM-SHA-512
    secretRef: "gateway-arbitrum-sepolia-superusers"
    bootstrapUser:
      name: alice
      secretKeyRef:
        name: gateway-arbitrum-sepolia-bootstrap
        key: password

Workaround
The only way to avoid the invalid PodSpec is to omit bootstrapUser and rely solely on a top-level secretRef:. However, this prevents using a stable, custom-named bootstrap user from a pre-existing secret. It also reverts to the Operator managing the default kubernetes-controller credentials, which may be rotated regularly (see Issue redpanda-data/helm-charts#1596 and #1536).

The other potential suggestion—using auth.sasl.bootstrapUser.password in plaintext—is also undesirable for those needing secrets to remain outside the CR manifest. Also in api spec v1alpha2, auth.sasl.bootstrapUser.password is not a supported field - based on the spec taken from kubectl get crd redpandas.cluster.redpanda.com -o yaml and actual tests.

Environment
Redpanda Operator Versions: v2.3.8-24.3.6 and v2.3.7-24.3.6
Helm Chart Versions: 5.9.18 and 5.9.20
Redpanda Image: v24.3.6
Kubernetes: v1.29.14

Expected Behavior: Specifying auth.sasl.bootstrapUser.secretKeyRef should properly inject both key: and name: into every container needing RPK_PASS, allowing Pods to be created successfully without storing passwords in plaintext.

JIRA Link: K8S-558

[chrisseto]

For posterity, there's been a bit of discussion in the community slack about this issue.

It may be possible to work around this issue with the podTemplate field:

statefulset:
  podTemplate:
    spec:
      containers:
        - name: redpanda
          env:
          - name: RPK_PASS
            valueFrom:
              secretKeyRef:
                key: password
                name: gateway-arbitrum-sepolia-bootstrap