> For the complete CircleCI developer hub index, see [llms.txt](https://circleci.com/developer/llms.txt)

# circleci/evals

Orb for running assertions on LLM evaluation results. Email ai-feedback@circleci.com for feedback.


## Commands

### test

This command runs assertions on evaluation metrics, saves results in JUnit XML format, and makes them available in CircleCI's Tests tab.

| Parameter | Type | Default | Description |
|---|---|---|---|
| `assertions` | string |  | path to the JSON assertions file |
| `metrics` | string |  | path to the JSON evaluation metrics file |
| `results` | string | test_results.xml | path to store the JUnit XML results file |

## Examples

### run_evals_orb_test_command

Run assertions using the evals orb


```yaml
version: '2.1'
orbs:
  evals: circleci/evals@2.0
jobs:
  evals-test-assertions-job:
    docker:
      - image: cimg/base:current-22.04
    steps:
      - checkout
      - evals/test:
          assertions: assertions.json
          metrics: eval_results.json
          results: test_results.xml
workflows:
  test-eval-workflow:
    jobs:
      - evals-test-assertions-job
```