Skip to main content
This page defines Rulebase’s core terms, grouped by the part of the product they belong to, with a link to the guide that goes deeper on each.

​
Scorecards, sections, criteria, and checks

A scorecard is a nested structure: it contains sections, sections contain criteria, and criteria contain checks. Points are set on criteria and roll up into section totals.
  • Scorecard — the whole rubric Rulebase scores against, with a name, a description, and an optional scope. Only a published scorecard evaluates anything; a draft is invisible to AI QA and to reviewers.
  • Section — a themed group of criteria such as Resolution or Communication. A section’s score is the sum of its criteria’s points; you do not set it directly.
  • Criteria — the individual item being judged, and the level where points are assigned. Each criterion has score options: Pass, Fail, and optionally Partial or Auto-fail.
  • Check — the instruction an evaluator follows inside a criterion. A criterion can hold several checks.
Inside a check:
  • Condition — the if/then rule that maps an observation onto one of the criterion’s non-passing scores, as in If <observation> … mark as fail. Every check needs at least one before the scorecard can be published.
  • Apply to — the pill on a check that limits which tickets the check runs on at all, by channel, by SLA breach, by call direction, and so on. It reads All tickets until you narrow it.
  • Additional instructions — extra guidance applied while the check is being scored, for exceptions your reviewers hold in their heads. If you catch yourself writing “ignore this check when…”, you want Apply to instead.
A scorecard’s top-level points must add up to exactly 100, so scores are comparable across tickets. Create a scorecard walks through the structure and the point budget, and Add scorecard additional instructions covers additional instructions.

​
Tickets, conversations, work items, and interactions

The records Rulebase scores go by different names depending on where you see them.
  • Ticket — a customer support conversation from your help desk. This is the term the QA screens use.
  • Conversation — the same object as a ticket, as named by the API and the upload endpoints. Uploading conversations and API responses say “conversation”.
  • Work item — back-office work an agent did in an internal tool with no customer on the line, such as a fraud review or a KYC check. Work items are scored with the same scorecards as tickets, and a scorecard’s Applies to scope decides which of the two it covers. See Uploading work items.
  • Interaction — the umbrella term for customer touchpoints, and the name of the list that holds them. Interaction type separates support tickets from sales calls.
  • Ticket scope — whether the customer replied. A ticket where they did is External; a one-sided ticket where they never did is Internal, and it is skipped unless a published scorecard opts in to internal tickets.

​
The kinds of evaluation

An evaluation is one agent’s score, on one ticket, against one scorecard. A ticket handled by three people can carry three evaluations, so a ticket’s score and an agent’s score can be different numbers in reports.
  • AI QA evaluation — the score Rulebase produces automatically once a ticket clears the eligibility gates. See How AI QA works.
  • Manual evaluation — a score a person enters by hand against the same scorecard, either instead of or alongside the AI result. Anyone with permission to perform evaluations can create one, and a manual request also works on tickets automatic QA passed over.
  • Evaluation validation — review of a trainer’s manual evaluation by someone who can provide model-training feedback. The trainer’s evaluation counts immediately. If the validator publishes corrections, that evaluation replaces the trainer’s result.
  • Calibration — assigning several reviewers to evaluate the same tickets so you can compare how they scored and align on standards. Each reviewer produces a real evaluation. Switch it on under Settings > Features.
  • Criterion feedback — a thumbs up or thumbs down on one criterion of an AI result, with a note. Feedback can re-score that single criterion; a manual evaluation replaces the whole judgement. Give feedback on AI QA covers it.
  • Auto-fail — a score option a criterion can carry. Selecting it fails the whole evaluation rather than costing that criterion’s points.

​
Eligibility, coverage rules, and scoping

Four mechanisms control which tickets QA covers. Each answers a different question and is set in a different place: Eligibility is several settings, grouped under Eligibility and Timing in evaluation settings:
  • Coverage rules — sampling. A rule keys on something about the ticket (channel, brand, team, or group) and sets a target, either a percentage of matching tickets or a fixed count per day, week, or month. No rules at all means no sampling limit, not zero coverage.
  • Eligibility instructions — plain-language rules Rulebase applies to the conversation itself, to filter out tickets nobody would score: spam, a customer who typed “hello?” and vanished, a bot handoff with no human reply. They gate automatic QA only.
  • Eligibility window — an age limit measured from when the ticket started, so connecting a help desk with years of history does not flood QA.
  • Wait for all linked tickets to close — holds evaluation until every linked ticket in the thread is resolved, for teams that split one issue across several tickets.
  • Evaluation delay — how long after close Rulebase waits before evaluating. The delay is measured from when the ticket closes and decides when the evaluation runs; the eligibility window is measured from when the ticket started and decides whether it runs at all.
Configure evaluation eligibility is the guide for all of these, and the eligibility tester it describes will tell you which mechanism is holding a specific ticket back.

​
AI agents

An AI agent is an automation you build and run inside Rulebase, under Home > Agents. The Agents permission group governs them: a permission checklist that says Agents means AI agents. Create a managed agent covers building one.

​
Agents, teams, groups, roles, and memberships

  • Agent — a support rep you run QA and performance reports on, listed under Settings > People > Agents. Viewing agents and their reports requires the View agents and performance reports permission, in the Insights group.
  • Agent roster — the list of agents Rulebase evaluates. Someone missing from it is not evaluated, and someone still on it after they left keeps appearing in team performance views. See Manage the QA agent roster.
  • Team — a named group of agents with an optional team lead, maintained by you in Rulebase. Teams drive grouping in QA reports and exports, and an agent belongs to one team at a time. See Create and manage teams.
  • Group — a grouping that mirrors your help desk’s own groups rather than one you build in Rulebase. Tickets and agents arrive already assigned to one, and groups are offered for coverage rules, scorecard scoping, and filtering in the same menus as teams.
  • Role — the bundle of permissions someone carries. System roles ship with Rulebase and cannot be edited; custom roles are yours. Roles and permissions is the dictionary of what each permission grants.
  • Membership — one person’s login in your organization, carrying their role and optionally a data scope that narrows what they can see. Membership is separate from the agent roster: an agent with Not invited in the roster’s Membership column is still evaluated normally, they just cannot sign in.
  • Queue — a group of members used for assignment routing, such as the reviewers who share evaluation validation work. Queues are about assignment; teams and groups are about reporting.
Teams and groups appear in the same scoping and filter menus. Teams are created and renamed in Rulebase. Groups follow your help desk, so a group that is missing or misnamed is usually fixed there rather than here.