> ## Documentation Index
> Fetch the complete documentation index at: https://docs.rulebase.co/llms.txt
> Use this file to discover all available pages before exploring further.

# How AI QA works

> Follow a ticket from close to score: the eligibility gates, the evaluation delay, scorecard matching, scoring, and how human feedback shapes what comes next.

Automatic QA looks like one step from outside: a ticket closes, and later it
carries a score. Inside, it is a short pipeline with a decision at every stage.

**Nothing is being evaluated** is a gate question: eligibility settings and
scorecard scope. **The score is wrong** is a rubric question: your criteria and
their checks.

```mermaid theme={null}
flowchart TD
    A[Ticket closes in your help desk] --> B{Eligibility gates}
    B -->|Blocked| S[Evaluation skipped, with a reason]
    B -->|Passed| C[Evaluation scheduled for close time plus your evaluation delay]
    C --> D{Eligibility instructions read the conversation}
    D -->|Nothing worth scoring| S
    D -->|Worth scoring| E[AI scores each eligible agent against each matching scorecard]
    E --> F[QA score lands on the ticket and the agent]
    F --> G[Criterion feedback from your reviewers]
    G -.->|shapes later evaluations| E
```

## It starts when the ticket resolves

Resolution triggers automatic QA, because an open ticket can still turn around.
Rulebase sweeps recently resolved tickets on a schedule, so a few minutes between
close and any visible QA state is normal.

Two things must be true before a ticket is a candidate: it reached Rulebase
through your help desk connection, and an agent sent at least one customer-facing
message on it. A ticket where nobody replied has no agent work to judge.

## The gates run in a fixed order

Rulebase applies a series of gates and stops at the first that says no. The order
matters: the reason shown on a skipped ticket is the *first* thing that blocked
it, and others may also apply:

1. **Evaluation credits.** Your organization has evaluations left to spend.
2. **Eligibility window.** The ticket started inside the age limit you set.
3. **No evaluation already exists or is in flight.** A completed AI evaluation,
   or another request already queued, stops a second one.
4. **Linked tickets.** If you have asked Rulebase to wait, every linked ticket
   in the thread has to be resolved too.
5. **Ticket scope.** The customer replied, or a published scorecard has opted in
   to internal tickets.
6. **Scorecard match.** At least one published scorecard's scope covers this
   ticket.
7. **Eligible agents.** At least one agent on the ticket is on the QA roster,
   is a human agent (bots do not qualify), wrote enough to judge, and falls inside
   a matching scorecard's people scope.
8. **Coverage rules.** Your sampling rules selected this ticket, and selected
   these agents.

Scorecard match and eligible agents catch new accounts for opposite reasons: a
scorecard scoped too narrowly covers no tickets; a roster missing people leaves a
well-scoped scorecard with nobody to grade. A blocked ticket records its reason on
the ticket. [Why wasn't my ticket
evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) is the
gate-by-gate lookup for a specific ticket.

## The delay controls timing only

Once a ticket clears the gates, Rulebase schedules the evaluation for its close
time plus your **Evaluation delay**. The gates it passed are not re-run while it
waits.

The delay exists because tickets reopen. If a customer replies "actually, one
more thing" twenty minutes after close, an evaluation that already ran judged
half a conversation. So Rulebase rechecks resolution when the delay expires, and
a ticket that reopened in the gap is set aside. Aim for long enough that reopens
settle, short enough that reviewers
get their scores within the day.

## Eligibility instructions read the conversation

The gates so far read metadata; the last one reads the conversation. When the
delay expires, Rulebase prepares the conversation, transcribing call recordings
and reading image attachments as scorable evidence. Then it applies your
**Eligibility instructions** to the transcript itself.

This filter keeps QA off tickets where nobody would score anything. The default
asks for two things: the customer expressed a real request, and a human agent had
a two-way exchange about it. It catches spam, a "hello?" that went nowhere, and a
bot handoff with no human reply.

<Note>
  This gate and your coverage rules apply to **automatic** QA only. A manual
  request bypasses both, which is how you get a score on a specific ticket while
  you are still tuning the settings that filtered it out.
</Note>

## The AI scores against the matching scorecard

Rulebase produces one evaluation per eligible agent
per matching scorecard, so a ticket handled by three people can carry three
evaluations, each with its own score.

Within an evaluation, the AI works bottom-up through the structure in
[Create a scorecard](/guides/quality-assurance/create-a-scorecard). It reads each
check against the conversation, applies the check's conditions to score the
observation a pass, partial, or fail, and takes any **Additional instructions** on
the check into account. Criteria carry the points, sections roll up from their
criteria, and the total lands against the scorecard's 100. An auto-fail is the
exception: selecting one fails the evaluation outright.

Every result includes its reasoning, so you can see which check produced each
score.

## The score lands on the ticket and on the agent

When the evaluation completes, the score attaches in two places at once: the
ticket carries its QA result with the per-criterion breakdown, and the agent's
performance views and Insights dashboards pick up the same numbers. Coaching
draws on them too: attaching a ticket as evidence in a session brings its QA
score along.

Because the same rubric applies across a sampled slice of your volume, trends
and team differences are comparable.

## Feedback shapes the next evaluation

When a reviewer thumbs down a
criterion and explains what the AI got wrong, Rulebase turns that note into a
written correction in your organization's feedback library, scoped to the
scorecard and criterion it came from. Later evaluations retrieve it when they hit
a comparable situation.

Two tools apply to a score you disagree with:

* **Criterion feedback** corrects one result and teaches by example. Reach for it
  when the AI missed context you can describe in a sentence or two. See [Give
  feedback on AI QA](/guides/quality-assurance/give-feedback-on-ai-qa).
* **Additional instructions** change the rule everywhere, for every future
  evaluation of that check. Reach for them when the same exception keeps coming
  up. See [Add scorecard additional
  instructions](/guides/quality-assurance/add-scorecard-additional-instructions).

Neither reaches back across your history. Criterion feedback re-scores only the
one criterion you flagged on the evaluation you flagged it on; additional
instructions apply from the next evaluation onward. Every evaluation that already
ran keeps its score, so to judge old tickets under new guidance, re-evaluate them.

## Which setting controls which stage

| Stage                                     | What decides it                                                    | Where to change it                                                                                       |
| ----------------------------------------- | ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------- |
| Whether the ticket is a candidate         | Ticket resolved, agent replied, ticket reached Rulebase            | Your help desk connection under [Settings > Connections](https://app.rulebase.co/settings/connections)   |
| How far back Rulebase looks               | **Eligibility window**                                             | [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility)           |
| Whether one-sided tickets count           | **Ticket scope** on a scorecard                                    | [Create a scorecard](/guides/quality-assurance/create-a-scorecard)                                       |
| Which rubric applies                      | Scorecard **Add scoping**, and publishing the scorecard            | [Create a scorecard](/guides/quality-assurance/create-a-scorecard)                                       |
| Who gets evaluated                        | The QA agent roster, plus a scorecard's people scope               | [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster)                              |
| How much volume is covered                | **Coverage rules**                                                 | [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility)           |
| When evaluation starts                    | **Evaluation delay**, and **Wait for all linked tickets to close** | [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility)           |
| Whether the conversation is worth scoring | **Eligibility instructions**                                       | [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility)           |
| What gets judged, and how it scores       | Sections, criteria, checks, and conditions                         | [Create a scorecard](/guides/quality-assurance/create-a-scorecard)                                       |
| Exceptions to a single check              | **Additional instructions**                                        | [Add scorecard additional instructions](/guides/quality-assurance/add-scorecard-additional-instructions) |
| How a specific wrong result is corrected  | Criterion feedback                                                 | [Give feedback on AI QA](/guides/quality-assurance/give-feedback-on-ai-qa)                               |

## Reading the pipeline backwards

Start from a symptom:

* **Nothing at all is being evaluated.** Start at the scorecard. An unpublished
  scorecard, or one scoped past the tickets you care about, blocks everything
  downstream. It is the top cause on new accounts.
* **Some tickets are scored and similar ones are not.** That is a sampling or
  roster issue, and the rubric is fine. Coverage rules skip tickets on purpose,
  and an agent who never made the roster produces the same silence.
* **Scores exist but they are wrong.** Nothing in eligibility will help. The
  answer is in the criterion that scored wrong, its checks and conditions, and
  whether your exception is written down as an additional instruction.

For a specific ticket, don't reason it out from here. The eligibility tester
shows every gate at once for a real ticket, and [Why wasn't my ticket
evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) walks through
reading its result.

## Related

* [Rulebase glossary](/guides/concepts/glossary)
* [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility)
* [Create a scorecard](/guides/quality-assurance/create-a-scorecard)
* [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated)
* [Give feedback on AI QA](/guides/quality-assurance/give-feedback-on-ai-qa)