Release 1.21
cert-manager v1.21 includes:
- Removal of the default
tokenrequestRBAC from the Helm chart (breaking change) - Removal of Challenge and Order write permissions from the
cert-manager-editaggregate ClusterRole (breaking change) - Removal of configurable metrics path and port name Helm values (breaking change)
Major Themes
Default tokenrequest RBAC removed from Helm chart
⚠️ Breaking change
The Helm chart no longer creates a default Role and RoleBinding granting
the cert-manager controller permission to create tokens for its own
ServiceAccount (serviceaccounts/token: create).
This RBAC was added in v1.16 (cert-manager/cert-manager#7213) to support a "Using the cert-manager ServiceAccount" section in the Route53 documentation. That docs section was subsequently removed (cert-manager/website#1555) when the Route53 page was restructured, and no documented workflow — Route53 IRSA ambient, Vault Kubernetes auth, or any other issuer — requires the controller to mint tokens for its own ServiceAccount.
If you use serviceAccountRef.name pointing at the controller ServiceAccount,
you must now either:
- create your own
RoleandRoleBindinggrantingserviceaccounts/token: createon that ServiceAccount, or - migrate to a dedicated ServiceAccount with its own RBAC (recommended — see the Vault or Route53 documentation).
Credit to @everping and @kodareef5 for independently identifying (via privately reported security advisories) that this default RBAC widened the trust boundary beyond what cert-manager's published threat model documents.
Restrict Challenge and Order RBAC in cert-manager-edit ClusterRole
⚠️ Potentially breaking change
The cert-manager-edit aggregate ClusterRole no longer grants create for
challenges.acme.cert-manager.io or create, patch, update for
orders.acme.cert-manager.io. This fixes a security issue
(GHSA-8rvj-mm4h-c258)
where these permissions allowed namespace users to bypass Issuer solver
selectors and abuse ClusterIssuer credentials.
These resources are internal to cert-manager's ACME workflow and are not
intended to be created or modified directly by users. Challenge patch and
update are retained because the Challenge spec is immutable after creation
and users may need these verbs to remove stuck finalizers
(cert-manager/cert-manager#3851, cert-manager/cert-manager#3870).
This change was already shipped in patch releases v1.20.3 and v1.19.6, so if you are already running one of those versions this will not be a breaking change.
If you have tooling or workflows that create Challenge or Order resources directly (outside of the normal Certificate → CertificateRequest → Order → Challenge flow), you will need to grant those permissions explicitly.
Metrics port name and path Helm values removed
⚠️ Breaking change
The Helm values prometheus.servicemonitor.targetPort,
prometheus.servicemonitor.path, and prometheus.podmonitor.path have been
removed. The metrics path is always /metrics and the ServiceMonitor
targetPort is always the port named http-metrics — these were the defaults
and there was no supported reason to override them.
The controller Service metrics port has also been renamed from
tcp-prometheus-servicemonitor to http-metrics, aligning it with the webhook
and cainjector services.
Because the Helm values schema uses additionalProperties: false, users who
still have any of the removed keys in their values overrides will see a schema
validation error on upgrade. Remove these keys from your values file before
upgrading.
(cert-manager/cert-manager#8952, @erikgb)
Configurable CertificateRequest retry backoff duration
cert-manager 1.21 adds the --certificate-request-maximum-backoff-duration
controller flag (default: 32 hours), making the exponential backoff cap
configurable alongside the existing --certificate-request-minimum-backoff-duration
flag.
When a CertificateRequest fails, cert-manager backs off exponentially — by
default from 1 hour up to 32 hours. In environments with scheduled CA
maintenance windows, the backoff can grow so large that cert-manager does not
retry until hours after the CA comes back online. With this flag, cluster
operators can lower the ceiling to match their CA's maintenance schedule. For
example, --certificate-request-maximum-backoff-duration=1h ensures that
cert-manager retries at most every hour regardless of how many consecutive
failures have occurred.
The default of 32 hours preserves the behavior of previous releases. The flag
may also be set via the
certificateRequestMaximumBackoffDuration
field in the controller ControllerConfiguration API, or via the Helm chart:
config:certificateRequestMaximumBackoffDuration: 1h
See the controller CLI reference for the full list of flags, and What happens if issuance fails? Will it be retried? for background on how cert-manager retries failed issuances.
(cert-manager/cert-manager#8893, @lunarwhite)
Skip the self-check with waitInsteadOfSelfCheck
cert-manager 1.21 adds the waitInsteadOfSelfCheck solver option for ACME
HTTP01 and DNS01 challenges. When set, cert-manager skips its own self-check
and instead waits the configured duration after presentation before asking the
ACME server to validate. This is an escape hatch for environments where
cert-manager cannot reliably observe the same validation path as the ACME
server, such as split-horizon DNS or NAT loopback (or hairpinning).
See Skip the self-check with
waitInsteadOfSelfCheck
for configuration details.
Webhook Serving Certificate Renewal After System Suspend
The cert-manager webhook generates and automatically renews its own self-signed
serving certificate. Prior to 1.21, the renewal timer relied solely on Go's
monotonic clock (CLOCK_MONOTONIC). During system suspend (S3/S4) or VM live
migration, CLOCK_MONOTONIC stops advancing while wall-clock time
(CLOCK_REALTIME) continues. When the system resumes, the renewal timer has not
yet reached its deadline, so the webhook's serving certificate is never renewed
— even though it has already expired. This causes x509: certificate has expired or is not yet valid errors for all admission and conversion webhook calls.
cert-manager 1.21 adds a periodic ticker that polls wall-clock time against the
renewal deadline, detecting missed renewals regardless of whether
CLOCK_MONOTONIC advanced. The webhook now recovers within one minute of system
resume.
More details are available in the PR: https://github.com/cert-manager/cert-manager/pull/8464.
Community
As always, we'd like to thank all of the community members who helped in this release cycle, including all below who merged a PR and anyone that helped by commenting on issues, testing, or getting involved in cert-manager meetings. We're lucky to have you involved.
A special thanks to:
@Copilot@FelixPhipps@Peac36@SebTardif@apkatsikas@bitloi@dap0am@figaw@immanuwell@jabbrwcky@jnohlgard@jsoref@ltwongaa@lunarwhite@mateenali66@onurmicoogullari@putongyong@seanorama@texasich
for their contributions, comments and support!
Also, thanks to the cert-manager maintainer team for their help in this release:
And finally, thanks to the cert-manager steering committee for their feedback in this release cycle:
v1.21.0
Changes since v1.20.0:
Feature
- Add Venafi OAuth token request observability and a new
AuthFailedIssuer condition reason to distinguish bad credentials from transient infrastructure errors. (#8808,@FelixPhipps) - Add
certificateRequestMaximumBackoffDurationcontroller configuration option to cap retry backoff time for failed CertificateRequests. Configurable via config file,--certificate-request-maximum-backoff-durationCLI flag, or Helm valueconfig.certificateRequestMaximumBackoffDuration. Defaults to 32 hours for backward compatibility. (#8893,@lunarwhite) - Add an optional
waitInsteadOfSelfCheckfield to ACME HTTP01 and DNS01 solvers so cert-manager can skip its own self-check and ask the ACME server to validate after a configured wait. (#8858,@wallrj) - Add configurable
runtimeClassNamesupport for cert-manager components and ACME HTTP01 solver pods. (#8791,@jsoref) - Add new controller flag
--acme-http01-solver-extra-labels, allowing Helm'sglobal.commonLabelsto propagate to all dynamically-created ACME HTTP01 solver resources (Pods, Services, Ingresses, or Gateway API HTTPRoutes). (#8761,@lunarwhite) - Add opt-in
startupapicheck.ttlSecondsAfterFinishedHelm value to enable automatic cleanup of the startupapicheck Job via the Kubernetes TTL-after-finished controller. (#8523,@dap0am) - Added ARI support through the ACMEUseARI feature gate. (
#8798,@hjoshi123) - Added AWS IAM authentication support for Vault issuer, including IRSA (IAM Roles for Service Accounts) and ambient credentials (EC2/ECS). (
#8422,@bitloi) - Added
cert-manager.io/ignore-tls-listenersannotation for ignoring gwapi listeners. (#8727,@hjoshi123) - Added option to specify additional listener protocols the GatewayAPI integration will consider when creating certificates. (
#8683,@ThatsMrTalbot) - Adds support for the Modern2026 go-pkcs12 profile and FIPS 140-3 (
#8841,@seanorama) - Cainjector:
- Disabled client side rate-limiting if AP&F is enabled. (
#8757,@hjoshi123) - Extend the Venafi/CyberArk integration to also support PANW NGTS. (
#8779,@FelixPhipps) - Feat(certificate): adding certificate renewal policies (
#8258,@hjoshi123) - Make cainjector use SSA unconditionally and deprecate the ServerSideApply feature gate (
#8692,@erikgb) - Processed annotations
cert-manager.io/alt-names,cert-manager.io/ip-sansto Certificates generated from ingress like objects in cert-shim controllers. (#8927,@jabbrwcky) - Promote the CAInjectorMerging feature gate to GA (
#8583,@Copilot) - When using ACME HTTP-01 with a ListenerSet, setting the annotation
acme.cert-manager.io/http01-parentreffallback: "true"causes cert-manager to use the parent Gateway as the solver HTTPRoute parentRef instead of the ListenerSet. This enables TLS-only ListenerSets to rely on a shared Gateway HTTP listener for ACME challenges. (#8749,@apkatsikas)
Bug or Regression
- BREAKING: The Helm chart no longer ships a default
RoleandRoleBindinggranting the cert-manager controller ServiceAccount permission to create tokens for itself (serviceaccounts/token: create). This RBAC was added in v1.16 (#7213) but no documented workflow requires it, and the motivating Route53 docs section was removed in Oct 2024. If you rely onserviceAccountRef.namepointing at the controller ServiceAccount (an undocumented pattern), you must now create your ownRoleandRoleBindinggrantingserviceaccounts/token: createon that ServiceAccount, or migrate to one of the documented patterns (IRSA ambient, or a dedicated ServiceAccount with its own RBAC). (#8931,@wallrj-cyberark) - ACME challenges no longer terminally fail on transient network errors (TLS handshake timeouts, DNS failures, context cancellation) during nonce fetches and authorization waits. The challenge controller returns the error and lets the workqueue retry with backoff. (
#8760,@texasich) - Add dns issuer secrets validation before marking it as ready (
#8255,@Peac36) - Add missing issuer finalizer RBAC to the order controller to support owner references (
#8654,@erikgb) - ClusterIssuer metrics collector now correctly respects the enabled-controllers configuration, avoiding a redundant startup when only operating within a namespace. (
#8822,@lunarwhite) - Fix Venafi TPP issuer setup and signing regression on master: restore authentication of the vcert connector in the client constructor, which was removed in #8808. (
#8843,@wallrj-cyberark) - Fix a performance issue in the certificateRequestApproval webhook where CertificateRequests referencing a GroupKind whose CRD is not yet installed would trigger repeated API server discovery queries on every admission request. Negative results are now cached for 30 seconds. (
#8651,@mateenali66) - Fix webhook serving certificate not being renewed after system suspend. (
#8464,@Peac36) - Fixed a rare panic in the trigger controller when a Certificate is deleted from the informer cache while a reconcile is in progress (e.g. during namespace teardown). (
#8962,@hjoshi123) - Fixed an integer overflow in
renewBeforePercentagecalculations that caused Certificates with durations longer than approximately 3 years to be incorrectly rejected by validation or assigned incorrect renewal times. (#8947,@ThatsMrTalbot) - Fixed duplicate
parentRefbug when both issuer config and annotations are present. (#8619,@hjoshi123) - Fixed infinite re-issuance loop when issuer returns an already expired certificate (
#8610,@onurmicoogullari) - Fixed local
e2e-setup-samplewebhookinstallation to use the samplewebhook image repository and tag from the saved image tarball manifest. (#8821,@wallrj) - Fixed potential OOM in DNS-over-HTTPS client by bounding response body read with io.LimitReader (128 KB cap). (
#8803,@SebTardif) - Fixed validation of timezone-prefixed renewal window cron specs without a schedule. (
#8813,@immanuwell) - Harden ACME Challenge and Order resources: reject user-created Challenges
without Order ownership, enforce Order spec immutability, and detect
pre-placed same-name Challenges with mismatched specs. (
#8948,@wallrj-cyberark) - Helm chart bugfix: rename image helper to avoid umbrella chart conflicts (
#8753,@FelixPhipps) - Helm: Fix invalid YAML generated when both
webhook.configandwebhook.volumesare defined. (#8664,@jnohlgard) - Remove ACME Challenge
createand Ordercreate/patch/updatefrom the cert-manager-edit aggregate ClusterRole to prevent direct manipulation of these internal resources (GHSA-8rvj-mm4h-c258). (#8958,@wallrj-cyberark) - Remove issuer owner reference from challenges blocking challenge garbage collection (
#8743,@erikgb) - Update logic to identify and preserve the secret matching nextPrivateKeySecretName (
#8577,@putongyong) - Vault Issuer webhook validation now rejects
..path segments inspec.vault.pathand auth mount path fields, preventingpath.Joinfrom silently resolving relative segments before constructing the Vault API request. (#8930,@wallrj-cyberark)
Other (Cleanup or Flake)
- API cleanup: removed deprecated ObjectReference (
#8625,@inteon) - Remove Helm values
prometheus.servicemonitor.targetPort,prometheus.servicemonitor.path, andprometheus.podmonitor.path. The metrics path is always/metricsand the target port is alwayshttp-metrics. Rename the controller service metrics port fromtcp-prometheus-servicemonitortohttp-metricsfor consistency with other workloads. Users must remove these keys from their value overrides before upgrading. (#8952,@erikgb) - The
enableGatewayAPIandenableGatewayAPIListenerSetfields onControllerConfigurationare deprecated and moved into thegatewayAPIsub-struct asgatewayAPI.enabledandgatewayAPI.enableListenerSet. The old fields continue to work. (#8732,@ThatsMrTalbot) - Update base images to Debian 13 (
#8849,@ltwongaa)