--- title: "Set up release validation manually" description: "Configure release validation by editing your CircleCI configuration file, for organizations that cannot use the Smart Deployments guided setup." doc_version: "unversioned" last_updated: "2026-08-06" --- > For the complete documentation index, see [llms.txt](https://circleci.com/docs/llms.txt) # Set up release validation manually Release validation evaluates webhook signals from your monitoring tool during an evaluation window after each deploy. This guide shows how to configure it by editing your CircleCI configuration file, rather than through the Smart Deployments guided setup. ## Introduction The [Set up Smart Deployments](https://circleci.com/docs/guides/deploy/set-up-smart-deployments/) guided setup generates this configuration for you, but it is unavailable in two cases: * Your code is not stored in a GitHub repository. * Your organization does not have AI features enabled. Follow this guide in either case. The result is the same release validation feature. The difference is that you write the configuration and connect your monitoring tool yourself. ### What you can set up manually Release validation and deploy markers are configuration, so they work on any version control system. Rollback and deploy pipelines are a web app feature that currently requires GitHub. | Capability | Needs GitHub | Notes | | --- | --- | --- | | Deploy markers | No | Add the `circleci run release` commands to any project. See the [Set up Deploy Markers Manually](https://circleci.com/docs/guides/deploy/configure-deploy-markers/#set-up-deploy-markers-manually) section. | | Release validation | No | Add a `validation` block to your release job, as described on this page. | | Rollback and deploy pipelines | Yes | See the [Set up Rollbacks](https://circleci.com/docs/guides/deploy/set-up-rollbacks/) and [Set up Deploys](https://circleci.com/docs/guides/deploy/set-up-deploys/) guides. | | Automatic rollback | Yes | Requires a rollback pipeline, so it inherits the GitHub requirement. | Without a rollback pipeline, release validation detects failures but cannot revert them. A failing check still marks the release as failed, records the events that caused it, and surfaces the failure in the deploys UI. CircleCI then reports the rollback as skipped, because no rollback pipeline is configured. Treat validation as your alerting and audit trail in this case, and roll back through your own tooling. ## Prerequisites * A CircleCI account connected to your code. You can [sign up for free](https://circleci.com/signup/). * A CircleCI project with a workflow configured to deploy your code. * Deploy markers configured in that workflow. See the [Set up Deploy Markers Manually](https://circleci.com/docs/guides/deploy/configure-deploy-markers/#set-up-deploy-markers-manually) section. * A monitoring tool that can send outbound webhooks. Datadog, Prometheus, and Alertmanager are supported directly, and you can connect any other tool as a custom provider. * Organization administrator permissions, to create a webhook secret. ## 1\. Add a release job to your configuration Release validation attaches to a _release job_, which is a job with `type: release` that names the release plan created by your `circleci run release plan` step. If your configuration already has one, skip to the next step. The `plan_name` value must match the deploy name you passed to `circleci run release plan`. In the following example, both are `my-service-release`: Deployment job and release job ```yaml version: 2.1 jobs: deploy-my-service: executor: some-executor steps: - checkout - run: name: Plan deployment command: | circleci run release plan my-service-release \ --environment-name=production \ --component-name=my-service \ --target-version= - run: name: Perform deployment command: - run: name: Update planned deployment to running command: circleci run release update my-service-release --status=running release-my-service: type: release plan_name: my-service-release workflows: deploy-service: jobs: - deploy-my-service - release-my-service: requires: - deploy-my-service ``` For details of the `circleci run release` commands, see the [Configure Deploy Markers](https://circleci.com/docs/guides/deploy/configure-deploy-markers/) guide. For the release job keys, see the [Configuration Reference](https://circleci.com/docs/reference/configuration-reference/#job-type). ## 2\. Turn on release validation Add a `validation` block to your release job. Release validation needs only one field to turn it on: Release job with validation enabled ```yaml jobs: release-my-service: type: release plan_name: my-service-release validation: enabled: true ``` With `enabled: true` and nothing else, CircleCI creates a system catch-all check. The catch-all accepts any signal from your monitoring tool, whatever its name, and fails when the criteria matches `firing`, `triggered`, or `alert`. Start here. A catch-all is enough when one monitor guards one component, and it saves you from matching signal names before you have confirmed the webhook arrives at all. ## 3\. Add named validation checks Move to named checks when one catch-all is too blunt, for example when you want a latency monitor and an error rate monitor to fail the release independently. Each entry under `validation.webhooks` defines one check. As soon as you add any entry, CircleCI stops using the catch-all, and every monitor must send a `signal_name` matching the `name` of the entry that handles it. Release job with two named checks ```yaml jobs: release-my-service: type: release plan_name: my-service-release validation: enabled: true webhooks: - name: error_rate provider: datadog max_failures: 2 - name: latency_smoke provider: datadog ``` This example relies on the Datadog defaults for `data_points` and `fail_when`. Add explicit mappings when your monitoring tool sends payloads in a shape those defaults do not understand. For every field, the `data_points` keys, and the provider defaults, see the [Release Validation Configuration Reference](https://circleci.com/docs/guides/deploy/release-validation-reference/) page. ## 4\. Set the evaluation window The engine waits 20 minutes for signals by default. Set `evaluation_time` to change how long it waits after a deploy completes: Release job with an explicit evaluation window ```yaml jobs: release-my-service: type: release plan_name: my-service-release validation: enabled: true evaluation_time: 30m webhooks: - name: error_rate provider: datadog ``` The default window is 20 minutes. Match it to how quickly your monitors fire: a window shorter than your monitor evaluation interval closes before any signal can arrive, so the release passes without ever being checked. If you have a rollback pipeline configured, add `auto_rollback_on_failure: true` to revert automatically when a check fails. On a version control system other than GitHub, leave it out: there is no rollback pipeline for it to trigger. ## 5\. Connect your monitoring tool This step is required. Without it, the engine receives no signals, so every release passes without being checked. ### 5.1. Copy the webhook ingest URL All validation webhooks POST to the following URL: ```text https://circleci.com/api/v3/deploy/hooks//validate ``` Replace `` with your organization ID. The URL is the same for every webhook secret in your organization. You can also copy the full URL, with your organization ID already filled in, from your organization deploy settings. To find your organization ID and organization slug, select **Org** in the CircleCI web app sidebar to open your organization settings page. The organization ID is available to copy from the org settings overview page. > **Image:** screenshot showing where to find your organization ID Figure 1. Organization ID available in organization settings ### 5.2. Create a webhook secret CircleCI authenticates validation webhooks with a bearer token. To create one, follow these steps: 1. In the [CircleCI web app](https://app.circleci.com), select your org from the org cards on your user homepage. 2. Select **Org** from the sidebar to open your organization settings page. 3. Select **Deploys**. 4. Find the Webhook secrets card. 5. Select **Create webhook secret**. 6. Enter a descriptive name, for example `Production Datadog`. 7. Optionally set an expiration date. 8. Copy the plaintext token before you close the modal. The plaintext token is shown once. Store it in your monitoring tool or your secrets manager straight away. If you lose it, revoke the secret and create a new one. ### 5.3. Authenticate webhook requests Configure your monitoring tool to send the following custom header on every validation webhook: ```json { "Authorization": "Bearer " } ``` Replace `` with the token you copied. Do not include the angle brackets. ### 5.4. Configure your monitoring provider Follow the instructions for your monitoring tool. Each provider needs the ingest URL, the bearer auth header, and a set of tags that let CircleCI match the alert to the right release. #### Configure Datadog 1. In Datadog, create or edit a webhook integration for alerts. 2. Set the webhook URL to the CircleCI ingest URL. 3. Add the bearer auth custom header. 4. Set the payload to include the provider, the alert transition, and your tags: ```json { "provider": "datadog", "alert_transition": "$ALERT_TRANSITION", "tags": "$TAGS" } ``` 5. Ensure your alert notifications include the tags listed below on every payload. Use the format `tag_name:value` in the `tags` array. | Tag | Required | Purpose | | --- | --- | --- | | `component_name` | Yes | The component for the release under validation. | | `env` | Yes | The environment, for example `production`. | | `signal_name` | Yes, when using named webhooks | Must match a `validation.webhooks[].name` value in your configuration. This routes the alert to the correct check when a release has more than one check. | | `version` | No | The deployed version. Omit this tag only if you accept fan-out to all matching running releases. | | `project_id` | No | The CircleCI project UUID. Omit this tag to match any project that shares the same component and environment. | | `namespace` | No | The component instance namespace. Omit this tag to match any namespace for that component and environment. | Map your Datadog template variables so that `component_name`, `version`, `env`, and `signal_name` align with the values your deploy markers use. For Datadog, the engine also accepts the monitor tags `app:` and `service:` as aliases for `component_name`, and `environment:` as an alias for `env`. Prefer the canonical names. #### Configure Prometheus or Alertmanager 1. In Prometheus or Alertmanager, create or edit a webhook receiver. 2. Set the URL to the CircleCI ingest URL. 3. Add the bearer auth custom header. 4. Include the keys listed below under `groupLabels` in the webhook payload. | Label | Purpose | | --- | --- | | `alertname` | The validation signal name. This must match a `validation.webhooks[].name` value in your configuration. | | `component_name` | The component for the release under validation. | | `env` | The environment. | | `version` | The deployed version. | Check that your payload shape matches the following example: ```json { "provider": "prometheus", "status": "firing", "groupLabels": { "alertname": "error_rate", "component_name": "my-service", "env": "production", "version": "1.0.0" } } ``` #### Configure a custom monitoring tool 1. Configure an alert webhook in your monitoring tool. 2. POST to the CircleCI ingest URL with the bearer auth custom header. 3. Send a JSON body that includes the fields listed below. | Field | Purpose | | --- | --- | | `provider` | Set this to `custom`. | | `component_name` | The application or component name. | | `env` | The environment. | | `signal_name` | The validation signal. This must match a `validation.webhooks[].name` value in your configuration. | | `version` | The deployed version. | | `alert_transition` | The condition signal, for example `triggered`. | Check that your payload shape matches the following example: ```json { "provider": "custom", "component_name": "my-service", "version": "1.0.0", "env": "production", "signal_name": "error_rate", "alert_transition": "triggered" } ``` If your monitoring tool sends payloads in a different shape, map the fields explicitly in your configuration. See the [Data Point Keys](https://circleci.com/docs/guides/deploy/release-validation-reference/#data-point-keys) section. ## 6\. Verify your setup Confirm the two halves work in order. Check that validation starts, then check that your webhook arrives. 1. Trigger your deploy workflow and let it complete. 2. Open the deployment from the deploys timeline and find the Release Validation panel. It shows an evaluation deadline and each check in a **Listening** state. If the panel is absent, validation is not running: check that your release job has `validation.enabled: true` and that `plan_name` matches your `circleci run release plan` deploy name. 3. Trigger a test alert in your monitoring tool while the window is open. 4. Refresh the deployment page. The event appears under the matching check, with a note saying whether it matched the fail criteria. An event that arrives but attaches to no check means the payload reached CircleCI and the matching failed. See the next section. ## Troubleshooting If your webhook produces no event on the deployment, work through the following causes. | Cause | How to check | | --- | --- | | The request is not authenticated | Your monitoring tool reports a delivery failure. Confirm the `Authorization` header is `Bearer ` with a token that is neither revoked nor expired, in the Webhook secrets card of your organization deploy settings. | | The component name does not match | The extracted `component_name` must match the component on the release exactly. Confirm your monitor sends `component_name`, or a Datadog alias such as `app`, with the same value as `--component-name` in your deploy marker. | | The environment does not match | `env` is not optional for matching. Confirm your monitor sends the same value as `--environment-name` in your deploy marker. | | The signal name does not match | Once you add entries under `validation.webhooks`, each monitor must send a `signal_name` matching an entry name. A missing or unmatched name is ignored. Remove your `webhooks` entries to fall back to the catch-all and confirm delivery, then add them back. | | The criteria expression does not match | CircleCI normalizes criteria values to lowercase before evaluation. Write lowercase literals in `fail_when`, so `"triggered"` rather than `"Triggered"`. | | The window closed before the alert | Compare the alert time with the evaluation deadline on the deployment. If the monitor fires more slowly than the window, raise `evaluation_time`. | For what each field means during matching, see the [Data Point Keys](https://circleci.com/docs/guides/deploy/release-validation-reference/#data-point-keys) section. ## Next steps * Review every field you can set on the `validation` block. See the [Release Validation Configuration Reference](https://circleci.com/docs/guides/deploy/release-validation-reference/) page. * Understand what happens when validation passes or fails. See the [How Release Validation Works](https://circleci.com/docs/guides/deploy/smart-deployments-overview/#how-release-validation-works) section. * If your code is in a GitHub repository, set up a rollback pipeline to enable automatic rollback. See the [Set up Rollbacks](https://circleci.com/docs/guides/deploy/set-up-rollbacks/) guide.