It starts when the ticket resolves
Resolution triggers automatic QA, because an open ticket can still turn around. Rulebase sweeps recently resolved tickets on a schedule, so a few minutes between close and any visible QA state is normal. Two things must be true before a ticket is a candidate: it reached Rulebase through your help desk connection, and an agent sent at least one customer-facing message on it. A ticket where nobody replied has no agent work to judge.The gates run in a fixed order
Rulebase applies a series of gates and stops at the first that says no. The order matters: the reason shown on a skipped ticket is the first thing that blocked it, and others may also apply:- Evaluation credits. Your organization has evaluations left to spend.
- Eligibility window. The ticket started inside the age limit you set.
- No evaluation already exists or is in flight. A completed AI evaluation, or another request already queued, stops a second one.
- Linked tickets. If you have asked Rulebase to wait, every linked ticket in the thread has to be resolved too.
- Ticket scope. The customer replied, or a published scorecard has opted in to internal tickets.
- Scorecard match. At least one published scorecard’s scope covers this ticket.
- Eligible agents. At least one agent on the ticket is on the QA roster, is a human agent (bots do not qualify), wrote enough to judge, and falls inside a matching scorecard’s people scope.
- Coverage rules. Your sampling rules selected this ticket, and selected these agents.
The delay controls timing only
Once a ticket clears the gates, Rulebase schedules the evaluation for its close time plus your Evaluation delay. The gates it passed are not re-run while it waits. The delay exists because tickets reopen. If a customer replies “actually, one more thing” twenty minutes after close, an evaluation that already ran judged half a conversation. So Rulebase rechecks resolution when the delay expires, and a ticket that reopened in the gap is set aside. Aim for long enough that reopens settle, short enough that reviewers get their scores within the day.Eligibility instructions read the conversation
The gates so far read metadata; the last one reads the conversation. When the delay expires, Rulebase prepares the conversation, transcribing call recordings and reading image attachments as scorable evidence. Then it applies your Eligibility instructions to the transcript itself. This filter keeps QA off tickets where nobody would score anything. The default asks for two things: the customer expressed a real request, and a human agent had a two-way exchange about it. It catches spam, a “hello?” that went nowhere, and a bot handoff with no human reply.This gate and your coverage rules apply to automatic QA only. A manual
request bypasses both, which is how you get a score on a specific ticket while
you are still tuning the settings that filtered it out.
The AI scores against the matching scorecard
Rulebase produces one evaluation per eligible agent per matching scorecard, so a ticket handled by three people can carry three evaluations, each with its own score. Within an evaluation, the AI works bottom-up through the structure in Create a scorecard. It reads each check against the conversation, applies the check’s conditions to score the observation a pass, partial, or fail, and takes any Additional instructions on the check into account. Criteria carry the points, sections roll up from their criteria, and the total lands against the scorecard’s 100. An auto-fail is the exception: selecting one fails the evaluation outright. Every result includes its reasoning, so you can see which check produced each score.The score lands on the ticket and on the agent
When the evaluation completes, the score attaches in two places at once: the ticket carries its QA result with the per-criterion breakdown, and the agent’s performance views and Insights dashboards pick up the same numbers. Coaching draws on them too: attaching a ticket as evidence in a session brings its QA score along. Because the same rubric applies across a sampled slice of your volume, trends and team differences are comparable.Feedback shapes the next evaluation
When a reviewer thumbs down a criterion and explains what the AI got wrong, Rulebase turns that note into a written correction in your organization’s feedback library, scoped to the scorecard and criterion it came from. Later evaluations retrieve it when they hit a comparable situation. Two tools apply to a score you disagree with:- Criterion feedback corrects one result and teaches by example. Reach for it when the AI missed context you can describe in a sentence or two. See Give feedback on AI QA.
- Additional instructions change the rule everywhere, for every future evaluation of that check. Reach for them when the same exception keeps coming up. See Add scorecard additional instructions.
Which setting controls which stage
Reading the pipeline backwards
Start from a symptom:- Nothing at all is being evaluated. Start at the scorecard. An unpublished scorecard, or one scoped past the tickets you care about, blocks everything downstream. It is the top cause on new accounts.
- Some tickets are scored and similar ones are not. That is a sampling or roster issue, and the rubric is fine. Coverage rules skip tickets on purpose, and an agent who never made the roster produces the same silence.
- Scores exist but they are wrong. Nothing in eligibility will help. The answer is in the criterion that scored wrong, its checks and conditions, and whether your exception is written down as an additional instruction.