Optimize your CI/CD pipeline with CircleCI Chunk AI agent

A slow CI/CD pipeline costs more than just time. Developers context-switch while waiting for builds, feedback loops stretch longer, and compute costs add up with every inefficient run. Most teams know their pipelines could be faster, but optimizing configurations requires deep knowledge of caching strategies, parallelism, and resource allocation.

The challenge compounds with AI-assisted development. As AI coding assistants help teams ship code faster, pipelines run more frequently. A 10-minute build that runs twice a day is annoying. That same build running ten times a day becomes a major bottleneck. The velocity gains from AI-generated code can be erased by slow feedback loops.

Chunk, CircleCI’s autonomous CI/CD agent, addresses this by bringing AI-powered optimization directly into your continuous integration pipeline. Unlike generic optimization tools, Chunk has access to your actual pipeline execution history, timing data, and resource usage patterns. It can analyze what’s actually slow in your specific workflow and generate targeted optimizations. In this guide, you’ll learn how to use Chunk to optimize your build configs automatically.

Setting up Chunk

To get started with Chunk, you’ll need a CircleCI account with a project already running pipelines. If you’re new to CircleCI, check out the getting started guide to connect your repository.

You can follow along with your own project or use our demo project on GitHub if you’d like a ready-made example.

Once your project is running in CircleCI:

1. Go to Chunk in the CircleCI web app sidebar.
2. Click Set up Chunk and follow the prompts to authorize GitHub access. Chunk requires read access to analyze your repository and pipeline configuration, and write access to create pull requests with its optimizations.
3. Connect your Anthropic or OpenAI API key when prompted. Chunk uses your own API credentials, so your code is never sent to external providers for model training and data stays under your control.

Chunk integrates directly with your existing CircleCI pipelines without requiring any reconfiguration. Once authorized, it can analyze both your repository code and your build history to understand how your project works. For complete setup instructions, see the Chunk documentation.

Asking Chunk to Optimize Your Pipeline

With Chunk enabled, navigate to Chunk in the sidebar. You’ll see a prompt interface where you can describe what you want Chunk to do.

Start with a simple prompt:

Optimize build config

Click Submit and Chunk begins working. It analyzes your .circleci/config.yml, examines your pipeline execution history, and identifies opportunities to reduce build time and resource usage.

Reviewing the Results

When Chunk completes its analysis, you’ll see a summary of the optimizations it identified and the changes it made. From this page, you can review the details and decide whether to open a pull request. If you’re not satisfied with the changes, you can skip the PR and refine your prompt instead.

Chunk explains its changes:

Summary
Optimizes the CircleCI pipeline by adding dependency caching,
parallel test execution, and improved Jest configuration.

Changes Made

CircleCI Configuration
- Added dependency caching using restore_cache and save_cache
  with checksum-based keys
- Changed npm install to npm ci for faster, more reliable installs
- Enabled parallel test execution with parallelism: 2
- Implemented test splitting by timings using CircleCI's test
  splitting feature
- Updated test job dependency from lint to build for better
  workflow ordering

Jest Configuration
- Set maxWorkers: '50%' to optimize resource usage
- Enabled Jest cache with custom directory .jest-cache
- Set testTimeout: 5000 for consistent test timing
- Configured mock behavior with clearMocks: true

Benefits
- Faster CI builds through dependency caching
- Reduced test execution time via parallelization
- More reliable installs with npm ci
- Optimized Jest performance with caching and worker configuration

If the changes look good, click Open Pull Request to create a PR in your repository.

Before merging, review the configuration changes to ensure they work with your project’s requirements. Chunk validates changes by running the pipeline, but you should verify the optimizations don’t affect your specific build needs.

More targeted prompts

The simple “Optimize build config” prompt works well for general improvements. For more control over what Chunk optimizes, try targeted prompts.

To focus on caching, run:

Add caching to reduce dependency installation time

To optimize for speed, run:

Reduce total pipeline execution time by parallelizing jobs

To reduce costs, run:

Optimize resource class usage to reduce credit consumption

To add specific features, run:

Add test splitting to distribute tests across multiple containers

To modernize configuration, run:

Update config.yml to use CircleCI orbs for common tasks

The more specific you make your prompt, the more focused Chunk’s optimizations will be.

Advanced configuration

Once you’re comfortable with the basics, you can fine-tune how Chunk operates with your project.

Environment setup

For projects with complex build processes, Chunk may need to understand your environment to generate accurate optimizations. You can provide a .circleci/cci-agent-setup.yml file that tells Chunk how to prepare your environment.

This is useful when optimizing pipelines that involve:

• Multi-stage builds: If your build has interdependent stages that affect timing
• Custom build tools: Compilers, bundlers, or tools that affect parallelization options
• Database dependencies: If integration tests need specific database setup that impacts test splitting
• Generated code: Build steps that generate code before tests can run

Here’s an example for a Node.js project:

version: 2.1

workflows:
  main:
    jobs:
      - cci-agent-setup

jobs:
  cci-agent-setup:
    docker:
      - image: cimg/node:20.11
    steps:
      - checkout
      - run:
          name: Install Dependencies
          command: npm ci
      - run:
          name: Build Project
          command: npm run build

Focus on environment preparation only. Chunk will analyze your main config.yml to understand the full pipeline structure.

Chunk will attempt to automatically create this file if needed. You can also manually generate one by going to Organization Settings -> Chunk Tasks and clicking the menu (…) for your task. Select Chunk Environment, then click Create File in GitHub. For more details, review the Chunk environment documentation.

Custom instructions

To guide how Chunk optimizes your pipelines, create a claude.md or agents.md file in your repository root. Document your preferences for:

• Resource class constraints (budget limits, required minimums)
• Caching strategies (what to cache, cache key patterns)
• Parallelization limits (maximum concurrent jobs)
• Required job ordering or dependencies
• Orbs you prefer to use or avoid

Chunk reads these files and applies your preferences when generating optimizations.

Historical analysis

Chunk’s optimization recommendations improve with more pipeline history. It analyzes execution times, resource utilization, and failure patterns across multiple runs to identify the most impactful optimizations. Pipelines with more history receive more targeted recommendations.

This historical analysis is one of the key advantages of having an AI agent embedded in your CI/CD pipeline. IDE-based tools can suggest generic best practices, but they can’t see how long your actual jobs take or which steps are the real bottlenecks. Chunk uses real execution data to prioritize optimizations that will have the biggest impact on your specific pipeline.

Chunk in the AI development ecosystem

While AI coding assistants help you write code faster in your IDE, Chunk operates at the CI/CD layer where it can optimize the infrastructure that validates and deploys that code.

This positioning creates a natural synergy with AI-assisted development. As AI coding assistants increase development velocity, pipeline optimization becomes more critical. Faster code generation means more frequent builds, making inefficient pipelines a bigger bottleneck. Chunk helps ensure your CI/CD infrastructure can keep pace with accelerated development workflows.

For teams using AI coding assistants, CircleCI also offers the CircleCI MCP Server, which connects AI assistants directly to your pipelines using the Model Context Protocol. This gives your IDE-based AI tools access pipeline status, build times, and failure patterns without leaving your editor. Together, Chunk and the MCP server create an integrated AI development workflow where both code generation and infrastructure optimization benefit from intelligent automation.

Conclusion

Slow pipelines don’t have to be a fact of life. Chunk analyzes your CircleCI configuration, identifies optimization opportunities, and generates faster, more efficient workflows that reduce both wait times and costs.

Ready to try it yourself? Sign up for a free CircleCI account and enable Chunk to start optimizing your pipelines automatically.