# Record an agent release Source: https://docs.rulebase.co/api-reference/agent-releases/create-agent-release rulebase_api2_public_openapi.json POST /v1/agent_releases Records that a new version of an agent was deployed. Call it from CI after each production deploy. # Get API key identity Source: https://docs.rulebase.co/api-reference/authentication/get-api-key-identity rulebase_api2_public_openapi.json GET /v1/identity Returns the organization identity authenticated by the API key. # Get the current organization Source: https://docs.rulebase.co/api-reference/authentication/get-current-organization rulebase_api2_public_openapi.json GET /v1/organization Returns safe metadata for the organization authenticated by the API key. # Generate a presigned URL for conversation upload Source: https://docs.rulebase.co/api-reference/conversations/generate-a-presigned-url-for-conversation-upload rulebase_api2_public_openapi.json POST /v1/conversation_uploads/presign Generate a time-limited URL for uploading a conversation archive. > [!NOTE] > > For detailed instructions on how to create and use the presigned URL to upload conversations, see the [Uploading conversations guide](/guides/uploads/conversations). # Get a conversation upload Source: https://docs.rulebase.co/api-reference/conversations/get-a-conversation-upload rulebase_api2_public_openapi.json GET /v1/conversation_uploads/{id} Retrieves one conversation upload by its Rulebase upload ID. # List conversation uploads Source: https://docs.rulebase.co/api-reference/conversations/list-conversation-uploads rulebase_api2_public_openapi.json GET /v1/conversation_uploads Returns conversation uploads for the organization. # Upload a conversation Source: https://docs.rulebase.co/api-reference/conversations/upload-a-conversation rulebase_api2_public_openapi.json POST /v1/conversation_uploads Uploads one conversation as multipart form data. > [!NOTE] > > For payload requirements and examples, see [Uploading conversations](/guides/uploads/conversations). # List evaluations Source: https://docs.rulebase.co/api-reference/evaluations/list-evaluations rulebase_api2_public_openapi.json GET /v1/evaluations Returns agent evaluations. # Apply a batch of work items Source: https://docs.rulebase.co/api-reference/work-items/apply-work-items rulebase_api2_public_openapi.json POST /v1/work_items/batch Creates, updates, or restores up to 100 work items by external ID in one request. # Create a work item Source: https://docs.rulebase.co/api-reference/work-items/create-work-item rulebase_api2_public_openapi.json POST /v1/work_items Creates a work item. Fails with 409 when the external ID already exists. # Delete a work item Source: https://docs.rulebase.co/api-reference/work-items/delete-work-item rulebase_api2_public_openapi.json DELETE /v1/work_items/{id} Soft-deletes a work item. Idempotent, and re-applying the external ID restores it. # Get a work item Source: https://docs.rulebase.co/api-reference/work-items/get-work-item rulebase_api2_public_openapi.json GET /v1/work_items/{id} Retrieves one work item by its Rulebase work item ID. # Get a work item upload Source: https://docs.rulebase.co/api-reference/work-items/get-work-item-upload rulebase_api2_public_openapi.json GET /v1/work_items/upload/{id} Polls a bulk work item upload job by its upload ID. # List work items Source: https://docs.rulebase.co/api-reference/work-items/list-work-items rulebase_api2_public_openapi.json GET /v1/work_items Lists work items newest-completion-first with cursor pagination and filters. # Update a work item Source: https://docs.rulebase.co/api-reference/work-items/update-work-item rulebase_api2_public_openapi.json PATCH /v1/work_items/{id} Updates a work item while preserving omitted fields and merging custom attributes. # Upload work items in bulk Source: https://docs.rulebase.co/api-reference/work-items/upload-work-items rulebase_api2_public_openapi.json POST /v1/work_items/upload Uploads work items from a CSV file. # Create or update a work item event Source: https://docs.rulebase.co/api-reference/work-items/upsert-work-item-event rulebase_api2_public_openapi.json POST /v1/work_items/{id}/events Creates or updates a source-system event associated with a work item. # Changelog Source: https://docs.rulebase.co/changelog Product updates and announcements **November 3, 2026** ## Score by severity We've added a new way to score: by how serious a mistake is, not by points. Each mistake is marked **Critical**, **Major** or **Minor**, and the ticket gets the score of its worst mistake. To try it, create or duplicate a scorecard, open **Advanced settings** and choose **Scores**. ![A Scores scorecard with Critical, Major, Minor and Pass levels](https://mintcdn.com/rulebase/-yrf-G5AOs-TE9iN/images/changelog/2026-11-03/scores-editor.png?fit=max\&auto=format\&n=-yrf-G5AOs-TE9iN\&q=85\&s=febf6308bfeec41821ede92e50151cd7) We've also added a **Scores quality** dashboard. It shows where mistakes happen, by level, reason and agent. Click any number to see the tickets behind it. ## Your Inbox We've given everyone an **Inbox**: a short list of tickets you should look at, each with a reason why. Instruct can add tickets to it for you, for example every low score from yesterday. ![Inbox with two tickets and their reasons, the first one open beside the list](https://mintcdn.com/rulebase/-yrf-G5AOs-TE9iN/images/changelog/2026-11-03/inbox.png?fit=max\&auto=format\&n=-yrf-G5AOs-TE9iN\&q=85\&s=39e5c3e080bcdb57bb5ada39974b7029) ## Agents that work for you You can now create **agents** in Instruct to take care of recurring work. For example, a "QA Watch" agent can check low scores every morning and tell your team lead. Just ask Instruct to set one up. You'll find your agents in the Chats sidebar, and everything they run on a schedule under **Scheduled**. ![Agents in the Chats sidebar next to the list of scheduled tasks](https://mintcdn.com/rulebase/-yrf-G5AOs-TE9iN/images/changelog/2026-11-03/agents-scheduled.png?fit=max\&auto=format\&n=-yrf-G5AOs-TE9iN\&q=85\&s=46c7e0edd275e75eabea1d00b1c6f7d0) ## Switch teams in one click On a team page, click the team name to jump to another team, or use **‹ ›** at the top right to go through them one by one. We keep your tab and dates, so comparing teams is quick. ![The team menu open on a team page](https://mintcdn.com/rulebase/-yrf-G5AOs-TE9iN/images/changelog/2026-11-03/team-switcher.png?fit=max\&auto=format\&n=-yrf-G5AOs-TE9iN\&q=85\&s=07e5c269d2d9726579efbe1c353bb6ce) ## Keyboard shortcuts You can now get around Rulebase from the keyboard. Press **?** on any page to see what's available. For example, **J** and **K** move between tickets, **E** starts an evaluation and **R** marks a ticket Done. ![The keyboard shortcuts list](https://mintcdn.com/rulebase/-yrf-G5AOs-TE9iN/images/changelog/2026-11-03/keyboard-shortcuts.png?fit=max\&auto=format\&n=-yrf-G5AOs-TE9iN\&q=85\&s=2642c05f1b23e4fe177c178ffcf0e475) ## Rulebase in your language Rulebase is now available in French, Italian and Spanish. Pick yours under **Settings → Profile → Language**. ![The language menu with English, French, Italian and Spanish](https://mintcdn.com/rulebase/-yrf-G5AOs-TE9iN/images/changelog/2026-11-03/language.png?fit=max\&auto=format\&n=-yrf-G5AOs-TE9iN\&q=85\&s=a839d9a08fad4197bc400827b824cd92) ## Smaller improvements * **Coaching:** you can coach several agents in one session, add to-dos with due dates, and agents can confirm they've read your feedback. * **Team and agent pages** are redesigned. Managers can use **View as** to see Rulebase the way an agent does. * **Reports:** ask questions about a report in the **Chat** panel, share it in a few clicks, or duplicate it for another team. * **Dashboards:** we've added scatter charts, a default date range you can save for each board, and a **Ticket form** breakdown. * **Exports** now match the filters on your list and include written feedback. * **Tickets** and **Evaluations** now show everything by default, with no date filter. * When you accept a contested evaluation, you now choose **Pass**, **Fail** or **Partial**. * Rulebase now works on your phone, and Home loads faster. **October 21, 2026** ## Dark mode Rulebase now has a dark mode. Open the account menu from your organization name at the top of the sidebar and pick **Light**, **Dark** or **System**. Your choice is remembered in your browser. ![Home in light mode and dark mode, side by side](https://mintcdn.com/rulebase/Rej5JFEIeSmBiTq7/images/changelog/2026-10-21/light-and-dark-home.png?fit=max\&auto=format\&n=Rej5JFEIeSmBiTq7\&q=85\&s=22b6a774bd7a75fc4c0e0188a9d94806) The account menu itself has been redesigned. Your name and email sit at the top beside the theme control, then your organization (with a switcher if you belong to more than one), **Settings**, **Docs**, **Changelog** and **Sign out**. ![The account menu in dark mode, with the Light, Dark and System control](https://mintcdn.com/rulebase/Rej5JFEIeSmBiTq7/images/changelog/2026-10-21/dark-mode-account-menu.png?fit=max\&auto=format\&n=Rej5JFEIeSmBiTq7\&q=85\&s=756b3986bb8ebfd2a5c761239bebdae9) ## Knowledge sources open beside the review When a criterion mentions a knowledge base document, or a check cites one, clicking it now opens the document in a panel next to the review instead of a new tab. Citations scroll to the cited passage and highlight it. Press **Escape** or **✕** to close the panel. Cmd-click still opens a new tab. ![A failed check beside the knowledge panel, with the cited guideline highlighted](https://mintcdn.com/rulebase/Rej5JFEIeSmBiTq7/images/changelog/2026-10-21/knowledge-panel-citation.png?fit=max\&auto=format\&n=Rej5JFEIeSmBiTq7\&q=85\&s=2175b492eae0dbaa235a4c36072fe856) Criterion result cards are simpler too. The result and score sit in one chip (for example **Partial · 10 / 20**), checks open from a single **2 checks** toggle, and the reasoning reads as normal paragraphs. ## Reports you can share and export Reports are easier to find, share and work on. ![A report with Export and Share in the top bar and the Share popover open](https://mintcdn.com/rulebase/Rej5JFEIeSmBiTq7/images/changelog/2026-10-21/report-share-export.png?fit=max\&auto=format\&n=Rej5JFEIeSmBiTq7\&q=85\&s=824d700c99339426c079877a453b6a2f) * **Insights → Reports** has two tabs: **All** and **Created by you**. Report cards now show a live preview of what's inside. * Opening a report takes you to its latest run. On wide screens, a list on the left lets you jump between periods, teams and agents. * **Share** lets you add people or teams, choose who else in your organization can open the report, and copy a link. * **Export** downloads the report as a Word document, PDF or Markdown. * **Edit report** opens the instructions and schedule. Select any part of the instructions and choose **Ask for changes** to have Instruct edit just that passage. The same works on drafts Instruct writes in chat. * Anyone in your organization can now preview a report for any team or agent from its instructions. Previews are not saved or sent. * When you ask Instruct for a report, it drafts it in chat first so you can review it, then asks whether to schedule it. Agents can also see reports that are shared with them on their performance page, under **Reports**, grouped by month. ## Faster manual evaluations * If you submit with a failed or partial check that has no reason, Rulebase takes you straight to that check and highlights the reason list. Submit again to move to the next one. * **Fail** and **Partial** are only offered when the check has a matching reason. Hover a disabled option to see why. * The result filters (All, Fail, Partial, Pass, N/A) explain what they show on hover. * Error messages now name the criterion instead of showing an internal field path. ## Request evaluations for whole teams When requesting evaluations, search now returns **Teams** as well as agents. Pick a team and every agent in it is included. The footer shows the total, for example **25 agents · 75 tickets**. Switch between **Select agents** and **Upload CSV** at the top of the dialog. ## Coaching * **Coaching → Sessions** now opens with three cards for the last 30 days: sessions delivered, how many agents improved after coaching, and sessions that are overdue. Click a card to see the sessions behind it. * Each suggested session shows a short **Reason** explaining why it was suggested, with **Approve** and **Dismiss** on every row. **Manage rules** is in the **⋯** menu. * **Agent feedback** is now called **Coaching Notes/Action Plan**. You can attach images and files to it, and to your private session notes, by clicking attach, pasting or dragging them in. * Opening a coaching session from an agent's page shows it in a dialog, so you stay on the agent. * New sessions default to today's date, and a session can't be marked delivered before its meeting date. ## Settings * **Settings → Security** lets admins choose how long someone can be inactive before they are signed out, from 1 hour to 30 days. The default is 7 days. * **Settings → Notifications** has a new **Agent evaluation reminders** option that emails agents a daily or weekly summary of evaluations they haven't read yet. It is off by default. * **MCP servers** now live in **Settings → Connections**, next to Integrations and API keys. * **Settings → Routing** can now match work items by **Task type**, so you can send, for example, AML tasks from one team to a specific reviewer. ## Improvements * Breadcrumbs show where a page sits in the app, not the pages you opened to get there. * **Cmd+K** search is faster and now works while you're typing in a chat box, including on Home. * You can search the **Your teams** list in the sidebar when adding a team. * Hover an agent's name in a chat transcript or call transcript to see their details. * Zendesk follow-up tickets are now linked to the ticket they follow up on, even when the customer's reply doesn't mention the original ticket number. * Images in Notion knowledge base pages are kept when they sync. * Disputed evaluations can now be reassigned by your routing rules, even if someone else already had the ticket. * Fixed QA sampling now spreads evaluations across your channels, such as chat, email and calls. * The public API's `GET /v1/evaluations` now includes scorecard and criterion results. * Instruct chats use a slightly smaller, easier-to-scan text size. **October 15, 2026** ## Routines Some automations don't need a graph. A **routine** follows instructions you write, like a runbook, and runs on a schedule, a cron expression, an event, or when you click **Run now**. Create one from **New → New routine** on Home's Workflows panel. Describe what to look for and what to do about it, pick when it runs, and it goes live straight away. Graph workflows are still there under **New → New workflow**. A routine can use every Rulebase tool, the skills you choose (type **@** in the instructions to mention one), and the MCP servers you connect to it, so it can post to Slack or update another tool when it finds something. Each run shows on the **Runs** tab as **Acted** or **Quiet**, with the version it ran and how long it took. Edits save as a draft until you **Publish**. You can also ask Instruct to set up a routine for you. **September 27, 2026** ## A cleaner workspace Every list page now keeps its controls in one toolbar at the top of the card: your views, search, display, filters, then Clear, Save and Export. The page header only holds the page name and **New chat**. Active filters sit in a grey band under the toolbar, and **Clear** now removes every filter, even on a saved view. Once a page has more than three saved views, they move into a **Your views** menu. Hover a view to preview its filters. Complaints now has saved views too, just like Tickets and Evaluations. ![Tickets toolbar with the Your views menu open](https://mintcdn.com/rulebase/tG9xH-5gwiEe-HLM/images/changelog/2026-09-27/tickets-your-views-toolbar.png?fit=max\&auto=format\&n=tG9xH-5gwiEe-HLM\&q=85\&s=55c4222efc64be903beb853716a1f726) We also refreshed the colours, with a new blue for buttons and highlights. ## Home shows what to do next Home now opens with a short list of **Suggested tasks** based on what you have been working on recently, like re-reviewing tickets that failed a criterion or creating a weekly report. Accept one to add it to your tasks, or dismiss it. Below that, **Tasks** collects the reviews, calibrations, coaching sessions and disputes waiting on you, grouped by when they are due. When you mark a review or coaching task done, it is resolved at the source too. The side panel shows this week's low scores and how each team is performing. ![Home with suggested tasks, tasks, low scores and team performance](https://mintcdn.com/rulebase/tG9xH-5gwiEe-HLM/images/changelog/2026-09-27/home-suggested-tasks.png?fit=max\&auto=format\&n=tG9xH-5gwiEe-HLM\&q=85\&s=249a2333236c93f7e12c1e2fbfbdc369) ## Chats have their own page **Chats** is now in the main sidebar. Pin the chats you use most, and find the rest grouped by Today, Yesterday, Previous 7 days and so on. Rename, export or archive a chat from its menu. Saved chats also show up in **Cmd+K** search. If a chat is interrupted mid-answer, you can now pick it up again with **Retry** instead of losing the reply. ![Chats page with pinned and recent chats](https://mintcdn.com/rulebase/tG9xH-5gwiEe-HLM/images/changelog/2026-09-27/chats-page.png?fit=max\&auto=format\&n=tG9xH-5gwiEe-HLM\&q=85\&s=6b2f20fc38e4e2e9e55324826e65573a) ## Ticket reviews are easier to read Each criterion is now a single card with its result and reasoning. Checks are listed one per row, with the most serious first, and **Show criteria** opens the full check when you need it. **AI** and **Manual** sit side by side at the top of the review, and if there is no manual evaluation yet you can start one from there. ## Faster manual evaluations Filter checks by **All, Fail, Partial, Pass or N/A**, with a count on each. Failed and partial checks open by default. Score a check with a single **Fail / Partial / Pass / N/A** toggle, and pick feedback reasons from a full, wrapping list instead of a cut-off dropdown. Saved evaluations use the same layout, so they read the same way afterwards. ![Manual evaluation with check cards and feedback reasons](https://mintcdn.com/rulebase/tG9xH-5gwiEe-HLM/images/changelog/2026-09-27/manual-evaluation-check-cards.png?fit=max\&auto=format\&n=tG9xH-5gwiEe-HLM\&q=85\&s=6e5fedd7b9aca2c878dc183bdd77a843) ## Coach several agents at once The **New session** dialog has been redesigned. Pick several agents, or a whole team, and a session is created for each of them. The coach defaults to you, and you can add focus areas and tickets before you create the session. Coaching performance is now available to every workspace with coaching turned on. ![New coaching session dialog](https://mintcdn.com/rulebase/tG9xH-5gwiEe-HLM/images/changelog/2026-09-27/new-coaching-session-dialog.png?fit=max\&auto=format\&n=tG9xH-5gwiEe-HLM\&q=85\&s=21cb31e66ed22a7b05031be6d8c59c9c) ## Complaints * Paste a ticket ID into Complaints search to find the complaint it belongs to. * Opening a linked ticket from a complaint shows it alongside the complaint, with an expand button for the full conversation. * Linked tickets now sit above Activity on a complaint. * Tickets show their linked complaints in **Linked activity**. ## Improvements * Scheduled exports now offer three clear datasets: Evaluations, Interactions and Complaints. Past exports are still available. * Added a **Work item is imported** trigger in **Settings -> Routing**, so new back-office work can be assigned as soon as it arrives. * Drag across checkboxes to select several tickets or evaluations, or use **Select all**. * The manual evaluation banner disappears once the evaluation is complete. * Opening a canvas in full view now has **Minimise**, which takes you back to the chat it came from. * Instruct gives more thorough answers to QA questions, and can update a canvas made in another chat. **September 14, 2026** ## Complaints now live in Settings → Reviews We've moved **Complaints** into **Settings → Reviews**. Manage complaint types there alongside customer issues and products. Older Complaints settings links still take you to the same place. **September 8, 2026** ## Control which tickets get evaluated Eligibility rules moved out of a dialog and onto their own page under **Settings -> Evaluations -> Eligibility**. Exclude tickets by tag or by customer, keep your custom instructions alongside them, and test the draft rules against a real ticket in the side panel before saving. The customer picker now shows and searches by email. ![Eligibility editor with excluded tags, excluded customers, and the test panel](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-09-08/eligibility-editor-test-panel.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=73d3a21010a039ac83205169aa803c18) Skipped-evaluation reasons in the Review tab now name the agent instead of showing an internal ID. ## Restore earlier scorecard versions Every scorecard publish is now recorded with a short summary of what changed. Open the version history from the scorecard page to compare versions and restore an earlier one. ![Scorecard version history with publish summaries and restore action](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-09-08/scorecard-version-history.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=144171784b4e0322a4ad5544554bdfa9) Scorecard checks also gained an **Agent team** target. Checks scoped to teams the agent is not on are marked not applicable instead of being scored. ## Export complaints with the columns you need **Compliance -> Complaints** has an **Export** button with a column picker, live preview, and email delivery. The Complaints tab on Tickets uses the same export, so filtered complaint lists no longer download as QA spreadsheets. Complaint categories can be limited to call, email, or chat, and the category dialog uses filter pills for workspace, channel, and risk level scoping. ## See agent workload on SLA breaches SLA breach badges and the SLA outcomes drilldown now show who was assigned and how many emails, chats, and calls they were carrying at the time. Hover a breach for the full picture, or choose **Learn more** to open Instruct with the agent, ticket, and breach time already loaded. ## Back office work in reports and Performance When Work item QA is on, weekly agent performance reports include completed back-office work items, with a **Back office tickets completed** chart on the agent overview. **Insights -> Performance** agent tables count back-office evaluations too. A new root cause table on Scorecard performance slices findings by agent, task type, and evaluator, shown per 100 evaluations so agents with different review volume can be compared. ## Dashboards and Insights Charts can filter by conversation custom fields, resolution time, and handling time thresholds, and each chart grouped by week can choose its own **Start of week**. ![Chart editor showing a custom field filter and the Start of week control](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-09-08/chart-editor-custom-field-week-start.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=ac2dd6e594b251be92b57a4385163c29) Instruct canvases appear under **Insights -> Reports** in a new **Created by you** section, and each team in **Your teams** has an **Overview** page with its leads, brand, and agents. ![Team Overview page reached from the Your teams sidebar section](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-09-08/your-teams-overview.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=83819b3a83157798c17d71872d60e6cb) ## Improvements * Added Veriff and ComplyAdvantage to **Settings -> Connections**, with screening previews that show the search inputs and matching entities. * Added **Export now** to the recurring export dialog, and switched the Tickets export to the All interactions column picker. * Added agent start dates and an optional **New joiner period** in agent settings. * Let teams be deleted without losing their evaluation history. * Labeled unknown call speakers instead of forcing them into agent or customer. * Let QA evaluations read linked tickets in full, so follow-up work in a related ticket counts toward case continuity. * Changed redaction placeholders to `[REDACTED ]` so evaluators do not mistake them for agent errors. **August 25, 2026** ## Saved views moved onto Tickets, Back office, and Evaluations Saved views are now pill tabs on the page they filter. **All tickets** is the default tab on Tickets, with **All tasks** on Back office and an organization-wide **Evaluations** page under Quality assurance. Create and edit views inline with suggested names, hover a tab to preview its filters and count, and drag tabs to reorder them. The Records section and view sharing are gone; every saved view is visible to the whole organization. ![Tickets page with saved-view tabs and a hover preview of one view](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-08-25/tickets-saved-view-hover.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=102b7dc4795de43ca37fa147d6e8346f) Tickets now open to tickets started in the last 7 days, filters are grouped by quality, date, people, and ticket attributes, and the **Queue** filter is available on every ticket list. ## Route and review back-office work items Back office work items can be assigned to a reviewer and routed like tickets. A **Work item is evaluated** trigger in **Settings -> Routing** assigns items by QA score, including to assignment queues, and coverage rules gain a **Back office** tab. Agents can contest work-item evaluations, with routing, email, and Slack notifications when they do. ![Routing rule editor with the Work item is evaluated trigger](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-08-25/routing-work-item-evaluated.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=00b3f5db4342a0eb15fdd9999e760158) Filter work items by team, resolved date, or any custom attribute you have imported, and break work-item charts down by those attributes. Reviewers can give thumbs up or down on AI criterion results. ## Instruct draws charts you can edit Ask Instruct an analytics question and it renders the chart in the chat. Open it in a workspace tab to see the raw results, tweak it in the chart editor, and place it on a dashboard when you are ready. The chat panel is now a tabbed workspace for tickets, sheets, canvases, and background tasks. Instruct can also refresh a ticket and transcribe all of its calls in one step, search Jira and JSM requests, and look up work items. ## Rulebase in Slack: channels, contests, and call recaps Rulebase can listen in a Slack channel once someone turns proactivity on there. It answers when addressed by name, runs lookups before replying, and only offers writes it can carry out with your confirmation. When an agent contests an evaluation, the reviewer gets a Slack card with the finding and **Accept** or **Reject** actions. After a call, the handling agent can receive a short recap in Slack that links to the transcript in Rulebase. Recaps are off until each person turns them on. ## Performance targets and approvals Set targets for teams, agents, and brands from the Performance details panel, with an optional target date and a roadmap from current to goal. Organizations with evaluation approvals on get an **Approved reviews** metric and an approval status filter on dashboards. ![Performance target detail with baseline, current value, and roadmap to the target date](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-08-25/performance-target-details.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=3ab0d7cc22b77c668d30206d4d1b92b7) **Insights -> Performance** regains an **Agents** tab for organization-wide agent search. ## Improvements * Added a **Your teams** sidebar section that pins teams and opens their evaluations. * Let assignment queues use custom percentage shares as well as even round-robin. * Added a **Team** condition on the Ticket is evaluated and Evaluation is contested routing triggers. * Let SLA policies be deactivated without deleting them. * Listed recent exports on **Settings -> Exports** with download links. * Added a Gradient Labs connection in **Settings -> Connections**. * Let the command menu find tickets by Jira issue key. * Redesigned the Beam Zendesk app around the open ticket and your recent calls. * Removed Disputes and Marketing from the Compliance sidebar. * Added a progress bar at the top of the page while navigating. **August 11, 2026** ## Comment on criteria and see who has read feedback Each criterion on a QA evaluation has a comment thread. Mention a colleague to notify them by email with a link that opens the exact criterion; agents see only threads they were mentioned in. Evaluations show whether the agent has opened them, and the evaluations list has a **Read status** filter. ![Evaluations list filtered by read status](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-08-11/evaluation-read-status-filter.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=abb37b91b63e543961d53fa7bf702579) ## Manage integrations as connections **Settings -> Integrations** is now **Connections**, grouped by what each provider does for you, with a page per provider that consolidates multiple accounts. Slack can be disconnected from there, and you can connect one or more status page RSS feeds so QA evaluations know about incidents that overlapped a ticket. ![Settings Connections catalog grouped by capability](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-08-11/connections-catalog.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=0f60c06c83bdb7bcfb3c2d0a50031553) ## Move agents between teams without losing history Changing an agent's team opens a dedicated dialog that lets you carry their past evaluations to the new team, from a date you choose or all of it. Team CSV imports offer the same choice. Teams can have any number of leads. ![Move agent dialog with team history options](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-08-11/move-agent-team-history.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=04bd751d1df35257c31bf094b038d28c) ## Find anything from the command menu The command menu searches tickets, work items, agents, customers, and knowledge documents, remembers where you have been, and offers actions such as creating a complaint or asking Instruct. ![Command menu showing recent items and actions](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-08-11/command-menu-actions.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=0da372313ae9aa83b100eff7dbbbe68d) ## Coaching sessions with tickets, topics, and agent feedback Attach evaluated tickets and topics when creating a coaching session. Coaches can share feedback the agent sees while keeping session notes private, and coaching sessions can be started from an agent's timeline. ![New coaching session form with two attached tickets](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-08-11/coaching-create-attached-tickets.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=79e90d23229c93637e14cbf383ff934c) ## Dashboards and Performance **Average CSAT** is a dashboard metric, with Zendesk CSAT responses synced in. A **Closed conversations** source measures each close action and credits the agent who closed it, and scorecard criteria charts can be broken down by outcome with stacked bars. ![Chart editor with the Average CSAT metric selected](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-08-11/average-csat-chart-editor.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=4dfb88af72aca909a789ac9afc4ed652) Performance pages accept any date range instead of whole weeks, and the QA Tickets page shows all tickets, with **Not evaluated** as its own group. ## Improvements * Re-evaluate specific tickets by pasting their IDs, alongside condition-based selection. * Queued Tickets and Evaluations exports and emailed them, so large exports no longer time out. * Added name and address redaction as opt-in rules in **Settings -> Security**. * Gave back office work items a review status, display options, and a place on the agent dashboard. * Let reports be delivered as Slack direct messages. * Let new workspaces choose a data region at signup and follow a setup checklist on Home. * Created a QA evaluation per participating agent on multi-agent work items. **July 28, 2026** ## QA for back-office work Back office work items can be reviewed the way tickets are: open an item, submit a manual evaluation, or queue an AI review, then filter and group the list by QA score. Import work items and their activity logs from CSV, Excel, JSON, or text files with an AI-assisted preview, and include work items in calibration sessions. ## Team, brand, and team lead performance **Insights -> Performance** has **Teams** and **Brands** tabs with QA score trends, plus a **Team leads** view and a dedicated team lead page. Pick which columns each table shows, and set a team performance page as your homepage. ![Insights Performance Teams tab with QA score trends](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-07-28/performance-teams.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=d04e61fccb89c4da3864ac157acd4ee7) Manage Zendesk brands and team assignments from **Settings -> People**, where members and invitations share one screen and agents can be invited in bulk. ## A simpler sidebar The sidebar is reorganized around Quality assurance, Compliance, Records, Resources, Lists, and Chats, with fixed **Tickets**, **Back office**, **Sales calls**, and **Customers** destinations. The Customers sheet became a **Companies** directory with sorting, filtering, export, and a detail page per company. Boards and reports share one tabbed Insights page, and Docs and Changelog moved into the organization menu. ## Re-evaluate criteria across past evaluations From **Home -> Tasks**, pick a date range, the criteria to re-run, and whether to target failed or auto-failed criteria. Preview how many evaluations match, then track the run and its before-and-after scores in the same list as requested evaluations. ![Re-evaluation request dialog on Home Tasks](https://mintcdn.com/rulebase/JbkVGDVfTEu9i47P/images/changelog/2026-07-28/tasks-reevaluation-request.png?fit=max\&auto=format\&n=JbkVGDVfTEu9i47P\&q=85\&s=d779d782127b9a2736e04c60e2108d8b) ## Calibration sampling and progress Sample a percentage of eligible tickets when creating a calibration instead of assigning every match. A **Calibration progress** report under **Insights -> Analytics** tracks AI-vs-human and human-vs-human alignment across sessions, with a criterion drilldown. ## Instruct and MCP Create and edit scorecards with Instruct, which now proposes changes for your approval. Instruct can also search Rulebase product guides, and generated files can be viewed and downloaded from chat. The MCP server gains read-only performance and coaching tools, work item write tools, and an EU endpoint. ## Improvements * Broke AI credit spend down by use case and person on **Settings -> Usage**, including QA evaluation spend, with an activity feed. * Coaching: a step-by-step create wizard, one Scenario/Progress sheet for session detail, reminders before a simulation is due, and an **Add to Google Calendar** link. * Added Scribe as a knowledge base integration. * Let administrators remove sensitive attachments from conversations. * Added a **Tags** filter on every conversation list and preserved filters when you return from a ticket. * Showed scoped members a clear no-access page instead of a blank screen for tickets outside their teams. **July 14, 2026** ## Submit and validate manual evaluations [Manual evaluations](/guides/quality-assurance/perform-a-manual-evaluation) can now move through a validation loop. For team members who cannot provide model-training feedback, selecting **Submit for review** publishes their manual score immediately and creates a validation submission. Routing rules for **Evaluation approval is requested** can assign the conversation to a reviewer, queue, or role. ![Manual evaluation editor with Submit for review button](https://mintcdn.com/rulebase/9kTViqYGOL-jsOQu/images/draft-evaluation-submit-for-review.png?fit=max\&auto=format\&n=9kTViqYGOL-jsOQu\&q=85\&s=d4d372a9de275922d6e9003f904b0ab5) Reviewers can approve or reject individual checks, correct scores, leave check-level feedback, and **Publish** an overriding evaluation. After review, trainers can see who validated their evaluation, compare original and published results, and read feedback on corrected checks. ![Rejected check with corrected score and feedback fields](https://mintcdn.com/rulebase/u-WJnsfjVWFsL9Za/images/draft-evaluation-reject-check.png?fit=max\&auto=format\&n=u-WJnsfjVWFsL9Za\&q=85\&s=0e792f75a00266d3d6ddc7424398d2e0) ## Find reviewer feedback from Home **Home -> Tasks** now surfaces recent feedback on your manual evaluations. Each item previews the feedback with its conversation and review time, then opens the related evaluation. ![Home Tasks showing recent evaluation feedback](https://mintcdn.com/rulebase/LmDNkLxqrTRcrtJn/images/draft-evaluation-recent-feedback.png?fit=max\&auto=format\&n=LmDNkLxqrTRcrtJn\&q=85\&s=f9412c6c33e92623e757672980962938) ## Manage agent rosters and QA exports **Settings -> People -> Agents** now previews CSV changes before an import and lets you [download and manage the existing agent roster](/guides/quality-assurance/manage-agent-roster). Bulk actions can move selected agents to another team or remove them from the active evaluation roster without deleting their historical work. ![Import agents dialog with CSV upload, existing-agent export, and roster removal option](https://mintcdn.com/rulebase/LmDNkLxqrTRcrtJn/images/agent-import-csv.png?fit=max\&auto=format\&n=LmDNkLxqrTRcrtJn\&q=85\&s=f3a078fcb32e519bf56226ff84bc1b9f) [QA exports](/guides/quality-assurance/export-qa-evaluations) now provide separate **Criteria scores** and **Criteria feedback** options. Feedback appears in its own column for each criterion, with conversation and knowledge references converted into readable text. ![Export evaluations dialog with separate Criteria scores and Criteria feedback columns](https://mintcdn.com/rulebase/LmDNkLxqrTRcrtJn/images/evaluation-export-columns.png?fit=max\&auto=format\&n=LmDNkLxqrTRcrtJn\&q=85\&s=a0cbe293c6f3d607817b6e908f9f6314) ## Populate compliance tests from sampled records The [**Populate test** dialog](/guides/compliance/populate-tests) can now add random samples of conversations, disputes, compliance cases, or compliance alerts directly to the current test. For conversation samples, choose an **Interaction type** and describe relevant language in **Transcript search**. Repeated runs skip records already in the sheet. Processing and failure states show each generated cell's progress, and completed results include supporting policy or framework citations. ![Populate test dialog with interaction type, transcript search, and sample size fields](https://mintcdn.com/rulebase/LmDNkLxqrTRcrtJn/images/compliance-populate-test.png?fit=max\&auto=format\&n=LmDNkLxqrTRcrtJn\&q=85\&s=94aefd2b3b21ba4a79cdd49fb1939a72) ## Duplicate, move, and filter dashboard charts Custom dashboard [chart menus](/guides/insights/manage-dashboard-charts) now include **Duplicate**, **Move to**, and **Copy to**. The chart editor supports repeated **Brand**, **Partner**, and **Agent joined date** filters. Conversation drilldowns can export every matching row to CSV while respecting visible columns and local filters. ![Custom dashboard chart menu with Duplicate, Move to, and Copy to actions](https://mintcdn.com/rulebase/LmDNkLxqrTRcrtJn/images/dashboard-chart-actions.png?fit=max\&auto=format\&n=LmDNkLxqrTRcrtJn\&q=85\&s=59ea8d47a6b528a2dd206ae693923e74) ## Fixes * Fixed calendar-based dashboard comparisons so month-to-date, week-to-date, year-to-date, and today use matching previous periods. * Preserved dashboard team and partner scope when opening expanded charts. * Kept manual evaluation feedback and review state scoped to the correct agent on multi-agent conversations. * Kept completed QA evaluations visible after an agent is removed from the active roster. * Fixed feedback-driven re-evaluations so the correct evaluation reruns. * Prevented a failure that could stop call transcripts from processing. ## Improvements * Grouped **Settings -> Evaluations** controls under **Eligibility**, **Timing**, **Scoring**, and **Review**, with **Score bands** under Scoring. * Added Linq under **Settings -> Integrations** so Instruct and managed agents can send iMessage, RCS, and SMS messages. * Made the agent chat sidebar resizable. * Added the approver's identity to completed human approval workflow steps. * Added **Open in new tab** to the conversation right-click menu. * Added an **AI evaluation date** filter and restored score-level grouping in evaluation lists. * Added a **Team lead** column to the Teams table. * Let Front-connected agents download image attachments for analysis. **June 23, 2026** ## Coaching sessions track follow-up work Coaching sessions now live on agent profiles with notes, linked evidence tickets, ticket-specific feedback, and a history timeline. You can add evaluated tickets to a session from conversation review, open feedback links directly into the related session, see upcoming coaching on Home, and let Instruct create sessions that default to next Monday when no date is provided. ## Agent workflows have clearer version history Managed agents now support workflow versions, restore summaries, publish validation, loop and note nodes, left-to-right graph layouts, node search, and clearer activity that shows tool calls and who started each run. Restored versions are summarized in the version dialog, and deleted agents no longer appear in active run, schedule, webhook, email, or chat workflows. ## Connected tools handle more support work Agents can now search live Intercom conversations, tag Intercom conversations, pick Intercom assignees from searchable admin and team lists, send Slack DMs through directory tools, and match Zendesk tickets to Aircall call records. Slack and Intercom connections in Settings now use the newer connection flow where available. ## Fixes * Fixed sidebar credits so they keep displaying when the agent-credits query fails. * Fixed truncated status badges in the bulk review import table. * Fixed human approval page and email titles to say "Awaiting your approval" instead of workflow node names. * Guarded graph subflow connections to prevent invalid workflow links. * Made scheduled runs and Slack-connected sessions more reliable. * Rate-limited Aircall matching requests so call matching stays stable during Zendesk updates. ## Improvements * Added configurable Home task widgets and an upcoming coaching module. * Added canonical coaching session links, copy-link UI, and direct links from feedback to the related session. * Added restore summaries, hover-only version actions, and version graph previews to managed-agent publishing. * Added the run creator to agent activity so manual and delegated runs show who started them. * Added live Intercom search, native Intercom tagging, and searchable Intercom assignment tools for managed-agent workflows. * Added native Slack connection rows under Settings -> Integrations. **June 15, 2026** ## Dashboards and reports get sharper Insights can now create Average QA score and Evaluations boards from templates, with score-level trends, agent charts, AHT coverage, team or partner scope controls, and critical-error breakdowns by agent and failure reason. Report pages also have a cleaner reader, copy links, table-of-contents support, and better handoff into Instruct. ## Recurring exports move into Settings Settings -> Export now lets you configure recurring exports for QA, evaluations, and all interactions, preview rows before saving, send an export now, and receive generated spreadsheets by email. Dashboard chart display settings are shared with everyone viewing the board, and expanded charts keep the same percent-of-total preferences as dashboard cards. ## Customers become company-backed records The Customers area now uses company-backed records with list and detail views, saved tabs, grouping, company field filters, comments, clients, field-change activity, and agent tools for creating, updating, commenting on, and bulk-setting company attributes. ## Instruct and assistants carry more context Instruct now understands stored views, dashboard statistics, selected reports, and page filters, so chat starts from the context you are looking at. Rulebase assistants can also work from Slack, Linear, Jira, and case threads with persisted tool context, Jira attachment analysis, case tools, and inline managed-agent delegation. ## Fixes * Fixed expanded Instruct overlay layout and restored the shorter composer placeholder. * Fixed weekly SLA outcome reporting and marked open weekly resolution outcomes correctly. * Restored the QA Analysts analytics view on `/analytics`. * Fixed team, partner, and site filtering in dashboard drilldowns. * Prevented invalid generated React artifact sources from being saved. * Restored email rendering for EU worker-sent exports and reports. * Prevented Zendesk email comments and XCALLY dialer uploads from syncing with the wrong channel or customer direction. ## Improvements * Added recurring export schedules, preview rows, background delivery, and manual Send now actions. * Added copy links to dashboard sharing and shared chart display settings across dashboard cards and expanded views. * Added dashboard team and partner scopes, rate-within-group metrics, agent heatmaps, and critical-error drilldowns. * Added company-backed customer views with saved tabs, grouping, field filters, comments, people, field-change activity, and agent tools. * Added Instruct view context, dashboard statistics access, drag-and-drop attachments, and role permissions for Chats and Instruct. * Added Slack, Linear, Jira, and case assistant context persistence so follow-up turns keep tool results. * Added holiday calendars, conditional work schedules, and frozen weekly SLA outcomes for schedule-aware reporting. **June 8, 2026** ## Workbench brings agents and reviews together Workbench now brings agent work, recent chats, assigned reviews, calibrations, and requested evaluations into one place. You can search recent chats and skills from the composer, attach images or files to Workbench chats, and start review work from compact task sections that keep conversation navigation scoped to the list you opened. ![Workbench with agents, recent chats, assigned reviews, calibrations, and requested evaluations](https://email-assets.rulebase.co/CleanShot%202026-06-08%20at%2019.30.07%402x.png) ## Agent workflows are easier to build Agent instructions now include a graph workspace for editing workflow steps, tools, schedules, and branching behavior. Undo and redo controls, clearer validation errors, live run progress, stored step outputs, and lock controls make workflow editing easier to inspect and recover from. ![Agent workflow graph with editing controls and run progress](https://email-assets.rulebase.co/CleanShot%202026-06-08%20at%2019.32.20%402x.png) ## Custom fields work across more surfaces You can now create Rulebase-owned custom fields from Settings -> Objects, see conversation field values in the conversation details panel, use more field types as dashboard breakdowns, and let managed agents update owned conversation fields during a workflow. ## Dashboards and SLA drilldowns get sharper Dashboard charts can keep their own date ranges, use searchable View by controls, show heatmaps, sort heatmap columns, and break down QA and SLA metrics by custom conversation fields. SLA charts now open outcome drilldowns with ticket rows, pending-outcome filters, exports, and clearer links back to the conversations behind each result. ## Fixes * Fixed deleted-from-Intercom conversations so the badge appears first and broken external links are disabled. * Reconnected dropped Instruct chat streams in the background after transient failures. * Fixed QA Documentation scoring to use current linked-ticket context. * Fixed dashboard drilldowns to preserve matching custom field filters and card actions. * Fixed long links and wide markdown tables in shared chats and workflow output. * Fixed employee read results for connected MCP clients. * Prevented XCALLY re-uploads from leaving stale call-part authors and phone-agent metadata. ## Improvements * Added compact task sections for assigned tickets, calibrations, and requested evaluations, with filtered bulk review requests. * Added undo and redo controls, graph locks, stored step outputs, and clearer run progress to workflow graphs. * Let dashboard charts keep their own date ranges, use searchable View by controls, and start from an Average QA score template. * Added pending-outcome filters and exports to SLA outcome drilldowns. * Made Rulebase-owned custom fields available from Settings -> Objects, dashboard breakdowns, and agent workflows. * Preserved stored view display options and cleared the modified state when options return to defaults. * Moved report pages onto the newer reports route with cleaner cards and direct report deletion. **May 25, 2026** ## More flexible custom dashboards You can now make any custom dashboard your homepage, arrange and resize dashboard charts across screen sizes, and add rich text cards for notes or explanations alongside charts. Dashboard charts can also be reused across boards and edited once to update every board that uses them. Chart setup now starts from searchable metric cards, so you can pick a common metric quickly or still create a chart from scratch. You can also toggle chart data labels per board and drill from Evaluated agents charts into a per-agent sheet for the selected date. ![Custom dashboard with resizable charts and a rich text card](https://changelog-assets.rulebase.co/dashboards-layout.png) ![Add chart dialog showing searchable metric cards](https://changelog-assets.rulebase.co/CleanShot%202026-05-25%20at%2016.35.42%402x.png) ## Chat with your data Instruct now understands the current ticket, agent, dashboard, and chart with visible attachment chips, so follow-up questions start with the page you are viewing. Open it from a dashboard chart to carry the selected chart's data and filters into the conversation. Instruct can also create and edit dashboards, groups, and charts directly from chat. ![Instruct chat with a dashboard chart attached](https://changelog-assets.rulebase.co/CleanShot%202026-05-25%20at%2016.38.43%402x.png) ## Connect Rulebase with MCP Rulebase now has a remote MCP server for AI clients like Claude Code, Claude Desktop, and Cursor. MCP clients can read Rulebase data including employees, scorecards, and knowledge-base answers after you sign in with your Rulebase account. See the [MCP docs](/guides/mcp) for the endpoint and setup steps. **May 20, 2026** ## Performance now lives in Insights Agent Performance has moved into **Insights**, alongside **Radar** and **Analytics**. Use Insights as the home for QA performance, coaching opportunities, and team progress. ## Organize and personalize Insights boards Your custom dashboards now live on the **Insights** page alongside **Radar** and **Analytics**. You can pin important boards, browse boards by section, and keep the views your team uses most in one place. You will also notice filters for **All boards**, **Created by you**, and **Shared with you**, plus board menus for sharing, renaming, duplicating, deleting, and moving boards between sections. ![Insights page showing pinned boards and board sections](https://changelog-assets.rulebase.co/CleanShot%202026-05-20%20at%2000.24.11.png) Board cards can now have their own emoji, header color, and header pattern. Use this to make important boards easier to spot, especially when your Insights page has several sections. ![Customize card dialog with emoji, color, and pattern controls](https://changelog-assets.rulebase.co/CleanShot%202026-05-20%20at%2000.25.43.png) **May 19, 2026** ## Chat artifacts can be shared Chat-generated reports and artifacts now have public share links with richer preview cards, so teams can send the output of a chat without copying the full conversation. ## Dashboard chat keeps the handoff Dashboard chat now opens in an overlay and preserves the session handoff from the page you started on. That makes it easier to ask a follow-up, return to the dashboard, and keep exploring the same analysis. ## Streams can resume Long-running chat streams can now resume instead of leaving the page in a broken state when a connection is interrupted. **May 17, 2026** ## Tags land on conversations Conversation tags now appear in ticket views and filters, with operator-created tags loaded into the same tag list. Agents can also apply tags from automated workflows, keeping manual triage and automation aligned. ## Quality review gets its own view Insights now includes a Quality review page with live QA data, critical-failure summaries, and lowest-scoring signals. Drilldowns make it easier to move from a pattern to the conversations behind it. **May 12, 2026** ## Skills join Knowledge Knowledge now includes a Skills section with an AI editor, giving teams a more direct place to manage the reusable instructions agents depend on. ## What's new moves into the app Release notes now have an in-app **What's New** surface, so shipped product updates can be read from Rulebase instead of only through the docs site. **May 8, 2026** ## Faster QA administration Admins can now test QA eligibility rules before they run, backfill evaluations from the admin console, and choose which columns appear in QA exports. Reviewers also get a quicker path to re-evaluate from the feedback dialog when a result needs another pass. Scorecards can now opt into single-sided ticket evaluations, so teams that review outbound-only or agent-only work can measure those conversations without reshaping their programs. ## Agents are easier to launch The new agent setup flow makes it clearer which tools an agent can use and what will happen when it runs. Chat-triggered agent runs now queue reliably, show clearer statuses, and can alert a teammate when a conversation needs follow-up. ## QA navigation is cleaner The QA sidebar now separates the main workflows from saved views, so teams can get to evaluations, requests, exports, and workload review without scanning a long mixed list. **April 30, 2026** ## Security gains an audit trail Privacy settings are now grouped under **Security**, where admins can review audit logs for sensitive account and access-management changes. Audit events include richer target previews and filters, so it is easier to answer who changed what without digging through internal logs. ## QA eligibility gets more precise QA programs can now include custom eligibility instructions and organization timezone context, helping scheduled evaluations pick the right conversations for each team. Evaluation views also added group, brand, partner, and channel filters, plus clearer skip reasons in daily evaluation reports. ## Analytics can drill into risk and score levels Compliance and QA charts now support risk-level and score-level breakdowns with clearer labels, colors, and drilldowns. Critical-only views make it easier to focus on high-impact failures before expanding into the full distribution. **April 23, 2026** ## Agents can use more workspace context Rulebase agents can now work with workspace data, Slack context, and structured tools when they respond to a request. This makes agent runs more useful for operational work like finding the right data, preparing a response, or assigning a conversation to the right person. ## QA requests show their progress Scheduled and user-requested QA evaluations now expose clearer progress while they run. Requesters receive completion emails, admins can run scheduled requests on demand, and long-running jobs can be cancelled instead of left in an ambiguous state. ## Complaints are easier to slice Complaint views added workspace columns, filters, CSV export fields, partner filters, and drilldowns from analytics charts. Linked complaints can now resolve with their conversations, keeping complaint queues closer to the work your team already closed. **April 15, 2026** ## Coaching sessions feel more live Simulation setup can now be guided by an agent, while live practice sessions stream customer responses and suggested next steps more smoothly. Reports finish faster, show clearer evaluation status, and keep the customer persona and internal context close at hand during review. Teams can also add pacing controls, follow-up guidance, passing scores, and reattempt rules to assignments, making coaching useful for repeatable team training rather than one-off demos. ## Workload review gets better filters Workload mode now handles per-agent scorecards and score-level filters, so managers can review the right set of evaluations for each agent. Criterion-level progress also appears while evaluations are running, making large review batches easier to track. ## Sheets can be restored Rulebase sheets now keep version history. You can review previous versions and restore a sheet when an experiment, bulk edit, or automation changes more than expected. **April 12, 2026** ## Introducing agent coaching Your agents can now run realistic practice sessions before they handle real tickets. Coaching simulations let you design a scenario — complete with an AI-generated customer persona and a step-by-step plan — and then drop into a live chat where the AI plays the customer. You can ground the scenario in your own training documents and knowledge base, so the practice feels like your actual product and policies, not a generic exercise. When you create a simulation, you describe the situation and optionally upload a PDF or reference your KB. Rulebase generates a customer persona with traits you can refine and a scenario plan with ordered steps. Once you're ready, you start the session and respond as the agent while the AI improvises as the customer within the scenario's boundaries. When the session ends, Rulebase evaluates the transcript and generates a report: overall score, summary, knowledge gaps, key issues, and scorecard-aligned criterion results. ![Coaching simulation creation with AI-generated persona and scenario plan](https://mintcdn.com/rulebase/r5Wr8jebJAzAsiac/images/coaching-simulation-create.png?fit=max\&auto=format\&n=r5Wr8jebJAzAsiac\&q=85\&s=aaedc3db9ac1eb1da5cb01fab5db36b9) If you attach a scorecard to the simulation, the AI customer will probe the areas your rubric cares about — without explicitly lecturing on the criteria — and the post-session report maps directly to your scorecard structure. This means your agents are evaluated during practice with the same criteria you care about in real conversations. ![Live simulation chat with customer persona sidebar and guided response options](https://mintcdn.com/rulebase/Bm2BhWSW1pIM7u6i/images/coaching-simulation-chat.png?fit=max\&auto=format\&n=Bm2BhWSW1pIM7u6i\&q=85\&s=4d65d56d70bc6a2d2620b701bec485ad) ## Richer analytics dashboards Custom dashboards now give you substantially more control over how you visualize and explore your data. You can organize charts into named, collapsible sections — so a single dashboard can cover QA performance, complaint trends, and SLA metrics without becoming a wall of charts. ![Custom analytics dashboard with collapsible sections, comparison periods, and multiple chart types](https://mintcdn.com/rulebase/Bm2BhWSW1pIM7u6i/images/analytics-dashboard.png?fit=max\&auto=format\&n=Bm2BhWSW1pIM7u6i\&q=85\&s=e8812794760a068dfcacfd651a92af68) Charts support new display options. Time-series charts can group by day, week, or month, and you can enable a comparison period that overlays the prior window as a dashed line or faded bar. Stacked bar charts and pie charts are available alongside the existing line and bar types, and you can set a display unit — duration, percent, score — so axes and tooltips read in business terms instead of raw numbers. You can now break down charts by agent, team, manager, and partner — and filter the entire dashboard by partner when relevant. QA evaluation charts support breakdowns by evaluation method (AI vs. human) and by reviewer. New chart types cover complaints, conversations, average handling time, and QA score distributions, each with their own dimension options. ## Marketing compliance reviews You can now submit marketing materials for automated compliance review directly in Rulebase. Upload a file — or paste a website URL — and Rulebase runs an AI review against your selected regulatory frameworks. The system captures a screenshot and readable page content for websites, so the review covers the live page as your customers see it. When a review completes, you see a summary alongside prioritized issues ranked by risk level: low, medium, or high. Each issue includes inline citations tied to specific framework documents, so you can trace every finding back to the regulation that supports it. The review list is searchable by material name with a dedicated Rejected section for flagged reviews, and status updates stream in real time so you never need to refresh. ## Smarter routing and workflow automation Routing rules gained several new capabilities this month. Assignment queues now distribute work in round-robin order instead of picking at random, so load is shared more evenly across your team. You can route conversations directly to a named member, reorder rules with drag-and-drop, and trigger routing when a complaint is created — with conditions on workspace and complaint risk level. You can also use an agent's QA score as a routing condition, separate from the conversation-level score. For contested evaluations, rules can branch on contest count and reassign ownership on subsequent contests. ## Agent performance and knowledge gaps The agent performance table now shows average handle time, a trend indicator versus the prior period, and plain-language performance bands — Met expectations, On track, or Needs improvement — so managers can scan speed, direction, and standing at a glance. Summary metrics at the top of the page show the org-wide average QA score and how many agents need attention. Rulebase also generates weekly knowledge gap reports that cluster weak spots tied to KB-linked criteria across your team. You can view gaps, supporting evidence, and which agents are affected from the agent performance dashboard. ## Partners and Intercom improvements You can now create and manage third-party partners as first-class records — with types (BPO, fintech, sponsor bank), associated domains, and linked people. Partner data flows into analytics filters and routing conditions across the platform. Intercom side conversations now appear in the conversation timeline, and Rulebase automatically detects and links related conversations when message text contains Intercom inbox URLs. **April 6, 2026** ## Coaching moves into team workflows Coaching assignments now use real workspace data, a dedicated learning tab, and a standalone agent practice experience. Drafts, autosave, persistent threads, and async document-based scenario generation make it easier to build a simulation without losing work. During a simulation, agents can use macro-style response options or write and send manually. Afterward, report generation streams progress and shows evaluation status instead of leaving the session feeling finished but unscored. ## Marketing reviews can run in Rulebase Marketing teams can submit files or URLs for automated compliance review against selected regulatory frameworks. Rulebase captures page context, ranks issues by risk level, and links findings back to the supporting framework language. ## Knowledge gaps show better evidence Knowledge-gap dialogs now organize evidence more clearly, helping managers see which missed criteria, conversations, and supporting details led to a gap. **March 31, 2026** ## Introducing agent coaching Your agents can now run realistic practice sessions before they handle real tickets. Build a scenario, ground it in documents or knowledge-base content, and let an AI customer play through the situation while the agent responds. When the session ends, Rulebase evaluates the transcript and generates a report with an overall score, summary, knowledge gaps, key issues, and scorecard-aligned criterion results. ## Performance views highlight who needs attention Agent Performance now shows average handle time, trend direction, and plain-language performance bands like **Met expectations**, **On track**, and **Needs improvement**. Summary metrics at the top of the page show the org-wide average QA score and how many agents need attention. Weekly knowledge-gap reports also cluster weak spots tied to KB-linked criteria, with supporting evidence and affected agents available from the performance dashboard. ## Dashboards get comparison views Custom dashboards can now organize charts into sections, group time-series charts by day, week, or month, and show comparison periods against the prior window. Stacked bars, pie charts, duration units, percent units, and score units make charts easier to read without translating raw values. **March 25, 2026** ## Dashboards can break down more work Dashboard charts now support more dimensions, including agent, team, manager, partner, reviewer, and evaluation method. New complaint, conversation, handling-time, and QA score chart types help teams explore the operational work behind their review programs. ## Routing gets more operational Routing rules can now use complaint creation, complaint risk, contest count, workspace, and agent QA score as conditions. Rules can be reordered, route directly to a named teammate, and distribute assignments in round-robin order so ownership is easier to control. ## QA lists show better context Evaluation lists now default to a more useful recent window, and conversation views show more partner, brand, and group detail alongside QA controls. Employee scorecard scoping also appears more clearly, making it easier to understand why a scorecard applies before you review. **March 3, 2026** ## Introducing compliance testing Compliance teams have been stuck between spreadsheets that don't scale and rigid GRC tools that take months to configure. Rulebase compliance testing is a new way to build, run, and report on regulatory testing programs — designed to move at the speed your team actually works. Describe what you want to test in plain language, optionally upload your policies, and Rulebase generates a complete program for you: controls, test scripts, sampling rules, and a ready-to-run test sheet. What used to take days of setup now takes minutes. Six frameworks ship today — **BSA / AML**, **UDAAP**, **ECOA**, **Reg E**, **TILA / Reg Z**, and **FCRA / Reg V** — each with built-in controls and population-aware defaults. Select one or combine several to cover broader mandates. You can test sales calls for UDAAP disclosure violations, review CX interactions for Reg E error-resolution compliance, audit dispute handling under TILA / Reg Z and FCRA, or evaluate SAR filings and KYC onboarding cases under BSA / AML. You pick the frameworks and Rulebase gives you the right controls and test population out of the box. ![Compliance Testing Demo](https://mintcdn.com/rulebase/9RXuKMd86-TWo2Mj/images/compliance-testing-demo.gif?s=0e051a6cb25a61f91cab6a8ab40240db) Your testing runs in a live Sheets workspace purpose-built for compliance analysts. Rulebase samples conversations or cases from your connected data sources and populates the sheet. When you hit Run, each cell spins up its own AI agent that pulls real-time regulatory context from your framework documents and executes your test scripts against the underlying record. Results stream in cell by cell with clear status indicators, so you always know what's done and what's still running. The workspace is fully interactive. You can work across multiple sheets per test, test and re-test specific test questions and test cases, filter and sort the data, and export to CSV. Our built-in chat agent can populate sheets with the right random or risk-based samples, add rows, and restructure columns on your behalf — so you spend less time wrangling data. When testing is complete, you generate an analysis report directly from the data. Reports summarize findings and surface targeted recommendations — and you can export them to PDF for your stakeholders. ![Compliance Testing Analysis](https://mintcdn.com/rulebase/9RXuKMd86-TWo2Mj/images/compliance-testing-analysis.png?fit=max\&auto=format\&n=9RXuKMd86-TWo2Mj\&q=85\&s=04c92de6ca06ea8bb27dff6ccf661ce4) ## Improved QA workflows Manual evaluations now support rich text comments with @mentions, image attachments, and auto-save — so your reviewers can write detailed feedback, tag teammates, and attach visual evidence without worrying about losing a draft. Evaluators can also now choose which specific agents they want to evaluate. Instead of being assigned a random sample, your team can focus reviews on the agents or queues that matter most to them. ![Manual QA Evaluation](https://mintcdn.com/rulebase/9RXuKMd86-TWo2Mj/images/qa-image-upload.png?fit=max\&auto=format\&n=9RXuKMd86-TWo2Mj\&q=85\&s=f975f1f8184c8285a8e20794abb6326f) You can now configure scorecard criteria that depend on each other. When one criterion fails, you can set dependent criteria to automatically fail as well — so your scorecards can enforce rules like "if disclosure was missing, then resolution fails". This works with both standard fail and auto-fail outcomes, giving you more precise control over how scores cascade across your scorecards. Rulebase's QA system now also dynamically searches your knowledge base during evaluations. When a criterion requires checking an agent's answer against your different sections of a knowledge base, the AI pulls the relevant articles — filtered by brand and language — instead of relying on static context. ## New integrations You now have Alloy and Google Drive as integration options in your [settings](https://app.rulebase.co/settings). Connect Alloy to pull structured compliance case data — like onboarding records, transaction monitoring alerts, and KYC reviews — directly into your testing sheets. Connect Google Drive to reference policy documents and internal procedures stored in Sheets or Docs without downloading and re-uploading them. # Create and manage roles Source: https://docs.rulebase.co/guides/access-management/create-and-manage-roles Build a custom role with exactly the permissions you intend, then edit, clone, or delete it as responsibilities change. A role is the bundle of permissions someone carries around Rulebase. The system roles that ship with every organization cover the usual shapes (an administrator who owns everything, an agent who only sees their own work), but real teams have people in between: a validator who reviews manual evaluations without touching member settings, or a knowledge owner who maintains scorecards. Build a custom role for those. ![Settings People Roles page with the Create role button, a search bar, and a table of system and custom roles with Type and Members columns](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/roles-page.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=b87e3883380cfc49f7cdfa2ae508f2ad) ## Before you start * Roles live on the **Roles** tab of [Settings > People](https://app.rulebase.co/settings/people/roles), alongside Members, Invitations, Agents, Teams, and Partners. * You need the **Manage members, assign roles, and create or edit roles** permission to create, edit, clone, or delete roles. Without it, the management controls are hidden. * A custom role needs a name and at least one permission. Rulebase blocks the save otherwise. * For what each permission grants, keep [Roles and permissions](/guides/roles-and-permissions) open in a second tab. ## System roles and custom roles The **Type** column tells you which kind of role you have and what you can do to it. System roles are Rulebase's defaults, shared by every organization and read-only. Open one and the dialog shows `System roles cannot be modified` with every field greyed out. Custom roles belong to your organization alone. | | System | Custom | | - | - | - | | Who creates it | Ships with Rulebase | You do | | Name, description, permissions | Read-only | Editable | | Can be deleted | No | Yes | | Can be cloned as a starting point | Yes, except **Administrator** and **Agent** | Yes | Most custom roles are a system role plus or minus one thing, so cloning **Quality Analyst** and unchecking a box beats building a permission set from scratch. ## Create a custom role 1. Go to [Settings > People > Roles](https://app.rulebase.co/settings/people/roles). 2. Click **Create role**. 3. Enter a **Name**, like `Evaluation validator` or `Scorecard editor`. This is what people pick from when assigning roles later. 4. Add a **Description**. Optional, but it becomes the Description column and the only hint colleagues get about why the role exists. 5. Under **Permissions**, check what the role should be able to do. 6. Click **Create role**. ![Create role dialog with an evaluation reviewer name, a description, and the grouped Permissions checklist below](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/roles-create-role-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=edae571d00213878a667fa95e3ecc09a) The role appears immediately with a **Custom** badge and zero members. It affects access only once you assign it to someone. ## Choose permissions deliberately The **Permissions** checklist is grouped by area: * **People**: inviting members, managing members and roles, managing teams, and managing the agent roster. Grant only to people who should be able to change who has access or who gets evaluated. * **Evaluations & calibrations**: evaluating conversations, marking conversations Done, submitting manual evaluations for approval, approving submissions and training the AI with feedback, deleting evaluations, assigning conversations for review, and viewing calibrations. * **Scorecards & knowledge**: maintaining scorecards, compliance rules, complaint categories, and knowledge base content. * **Insights**: organization-wide reports and analytics, viewing the agent roster, viewing AI evaluations on every conversation, and setting performance targets. The AI evaluations permission is the only one separating the `Quality Analyst` and `Compliance Analyst` system roles. * **Settings & integrations**: organization settings, connections to systems like Zendesk and Intercom, and API keys. * **Workflows**: viewing, running, and building AI workflows. * **Instruct**: reading and using Instruct chat. Build the role around the narrowest job it has to do. A reviewer who approves submitted evaluations wants **Evaluations & calibrations**, and probably **Insights** to see trends. Do not add **People**; that group lets someone change roles, including their own. > [!NOTE] > > The **Workflows** permission group refers to the AI workflows you build under > [Home > Agents](https://app.rulebase.co/home/agents). For the support agents on > your roster, visibility comes from **View the agent roster** in the **Insights** > group and editing comes from **Manage the agent roster** in **People**. ## Assign the role to people Assignment happens on a different tab, [Settings > People > Members](https://app.rulebase.co/settings/people/members). Two ways to apply it: * **One person:** click their row to open **Edit membership**, choose the role from the **Role** dropdown, and click **Save changes**. * **A group:** select their row checkboxes, click **Update role** in the floating action bar, then pick the role once and apply it to everyone selected. New hires can skip this. The [invite dialog](/guides/access-management/invite-members) assigns a role in the invitation. The **Members** count on the roles table confirms it landed; filtering the Members tab by **Role** shows exactly who carries a role before you change or delete it. > [!TIP] > > Every role change is recorded. To see who widened access and when, > [Settings > Audit logs](https://app.rulebase.co/settings/audit-logs) lists > **Created Role**, **Updated Role**, and **Deleted Role** events alongside member > changes. ## Edit, clone, and delete roles Management happens from the roles table. Clicking a row opens the role for editing; the overflow menu at the end of each row holds Clone and Delete. ![Roles table with the overflow menu open on a custom role row, showing Clone and Delete options next to System and Custom type badges](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/roles-row-actions-menu.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=d1c86abffa3cadec58bb9b6095591e80) ### Edit a role 1. Click the role's row to open **Edit role**. 2. Change the **Name**, **Description**, or any permission checkbox. 3. Click **Save changes**. **Save changes** stays disabled until you change something. Removing a permission takes it from every member at once, so check the **Members** count first if the role is widely used. ![Edit role dialog for a custom role with a newly checked permission and the Save changes button enabled](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/roles-edit-role-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=4dd724c62b863fc805d4af459397a260) ### Clone a role Open the overflow menu on the role to copy and choose **Clone**. The **Create role** dialog opens prefilled with the original's permissions and a name ending in `(Copy)`. Rename it, adjust the checkboxes, and save; the original is untouched. This is the fastest way to build a variant of a system role, which cannot be edited directly. ### Delete a role 1. Open the overflow menu on a custom role. 2. Click **Delete**. 3. Read the confirmation, then click **Delete**. ![Delete role confirmation dialog warning that the role will be permanently deleted and assigned members will lose their permissions](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/roles-delete-role-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=0823f9b86221f23a81c87e83a62bab03) Deleting cannot be undone, and Rulebase will not stop you deleting a role that still has members. Move those people first: check the **Members** count and reassign anyone on the Members tab. The **Delete** option does not appear for system roles. ## Confirm the role does what you intended Before handing a new role to a group: * The **Type** column shows **Custom**. * The description reads like something someone else could act on. * Reopening the role shows exactly the permissions you meant to grant. * The **Members** count matches the number you assigned. * A holder can reach the screens they need. If something is missing, the permission that covers it is named in [Roles and permissions](/guides/roles-and-permissions). ## Related * [Roles and permissions](/guides/roles-and-permissions) * [Invite members](/guides/access-management/invite-members) * [Create and manage teams](/guides/access-management/create-and-manage-teams) * [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster) * [Identity provider role assignment](/guides/identity-provider-role-assignment) * [Rulebase glossary](/guides/concepts/glossary) # Create and manage teams Source: https://docs.rulebase.co/guides/access-management/create-and-manage-teams Group agents into teams with a lead and use them for QA reporting, roster management, and coaching. Teams are how you organize agents for QA reporting and coaching. Each team has a name, an optional team lead, and a list of agents. When you filter Insights dashboards or run QA reports by team, the grouping comes from here. ![Settings People Teams page with a Create team button, search bar, and a table listing three teams with their team leads and agent counts](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/teams-page.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=ffab4944e8ed26a0e4bbad5dc93f6a37) ## Before you start * You need the **Manage teams and their members** permission, which `Administrator`, `Manager`, and `Team Leader` hold. It covers creating, renaming, and deleting a team, assigning a lead, and importing agents from CSV. See [Roles and permissions](/guides/roles-and-permissions). * Teams live at [Settings > People > Teams](https://app.rulebase.co/settings/people/teams). * Agents must already exist in Rulebase. If they haven't been added yet, [invite them](/guides/access-management/invite-members) or import them through the [Agents](https://app.rulebase.co/settings/people/employees) page first. ## Create a team 1. Go to [Settings > People > Teams](https://app.rulebase.co/settings/people/teams). 2. Click **Create team**. 3. Enter a **Name** your organization will recognize in reports and filters, like `Billing Team` or `EMEA Support`. 4. Click **Create team**. ![Create team dialog with the Name field filled in as Billing Team](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/create-team-dialog.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=538044f472942755ba92eb1ae96a34f8) The new team appears in the teams table. It starts empty; open it to assign a lead and add agents. ## Set up the team detail page Click a team name to open its detail page. This is where you do the day-to-day management: assigning a lead, adding agents, and reviewing who's on the roster. ![Team detail page for the Billing Team showing the team name with a pencil icon, a five agent count badge, the Add agent button, an assigned team lead, and the Agents table listing five agents with their emails](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/team-detail-manage.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=efeb48760ab911f2b3c68912f61a6ec3) ### Assign a team lead On a team with no lead yet, the **Team lead** section is an "Add team lead / Assign a member to lead this team" prompt. Click it, search for a member by name, and select them. The lead then shows with their name and email, as in the image above, and the overflow menu on their row offers **Change team lead** and **Remove team lead**. The lead must be a member of the same organization. Once set, they travel with the team into QA evaluation exports, so the export can show who was responsible for each evaluated agent. ### Add agents Click **Add agent** at the top of the detail page. A search dropdown appears listing agents who aren't already on this team. Select one and they join immediately. For larger moves, use **Import from CSV** in the overflow menu (the three-dot icon next to **Add agent**). The CSV needs a single `email` column, and **Download template** in the dialog gives you a correctly formatted starting file. Rulebase matches each row to an existing agent; any email it cannot match comes back as a row-level error such as `No agent found with email …`, and the rest of the file still imports. > [!NOTE] > > Assigning an agent to a team also marks them evaluable, which puts them into > future QA evaluations and performance reports. If you are adding someone who > should not yet be scored, adjust them on the > [Agents](https://app.rulebase.co/settings/people/employees) page after the import. An agent belongs to one team at a time, so adding them here moves them off whichever team they were on before. To reassign many agents at once, the [agent roster](/guides/quality-assurance/manage-agent-roster) has a **Change team** bulk action that is faster than opening each team in turn. ### Remove agents and rename the team Each agent row in the Agents table has an overflow menu with a **Remove** option. Removing an agent takes them off the team but does not deactivate them; they remain in Rulebase and can be added to another team. To rename the team, click the pencil icon next to the team name in the header. To delete the team entirely, open the overflow menu next to **Add agent** and choose **Delete team**. Deleting a team does not delete its agents. They are left without a team until you assign them to another one. ## How teams appear elsewhere Once a team has agents, the grouping shows up across the product: | Where | What the team does there | | - | - | | [Insights > Performance](https://app.rulebase.co/insights/performance?view=teams) | Browse performance by team, then drill into an individual agent | | [Agents roster](https://app.rulebase.co/settings/people/employees) | Filter the roster by team and move agents between teams in bulk | | QA evaluation exports | Each evaluated agent's team and team lead are included in the export | An agent left on the wrong team lands in the wrong export rows and the wrong performance view, so re-check membership whenever your org structure shifts. ## Related * [Invite members](/guides/access-management/invite-members) * [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster) * [Export QA evaluation data](/guides/quality-assurance/export-qa-evaluations) * [Roles and permissions](/guides/roles-and-permissions) * [Rulebase glossary](/guides/concepts/glossary) # Invite members Source: https://docs.rulebase.co/guides/access-management/invite-members Add people to your Rulebase organization with the right role so they can start working immediately. Everyone who works in Rulebase needs their own account. Inviting someone sends them an email with a link to join your organization, and the role you choose controls what they can see and do once they accept. ![Settings People Members page showing the Invite button, search bar, role filter, and a member table listing seven people across different roles with Active and Pending statuses](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/invite-members-page.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=116811cf82d4c5bd1d97b077edb2c953) ## Before you start * You need an admin role or permission to manage invitations. * Have the email addresses ready. You can invite several people at once. * If the role you want to assign doesn't exist yet, create it first under [Settings > People > Roles](https://app.rulebase.co/settings/people/roles). See [Roles and permissions](/guides/roles-and-permissions). ## Send invitations 1. Go to [Settings > People > Members](https://app.rulebase.co/settings/people/members). 2. Click **Invite**. 3. In the **Email addresses** field, type one or more addresses separated by commas. 4. Choose a **Role** from the dropdown. 5. Click **Send invitations**. ![Invite team members dialog with email addresses field and role dropdown](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/invite-members-dialog.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=19c50c0d589cfe2a62c06df2ee0eaf50) Rulebase sends each address an invitation email. Until the recipient accepts, they appear as **Pending** on the [Invitations](https://app.rulebase.co/settings/people/invitations) tab. ## Scoping access for non-admin roles If your organization has partners configured, the invite dialog shows an additional **Scope** field. Scoping narrows what data the invited person can reach: * **Entire organization** — access to data across the whole organization. This is the default. * **Specific groups** — access limited to the groups you select. * **Specific teams** — access limited to the teams you select. * **Specific partners** — access limited to the partners you select. Two roles ignore scope entirely. Administrators always have access across the entire organization, and agents can only ever access their own tickets and activity. Selecting either role disables the scope picker. > [!TIP] > > Everyone in a single invite batch receives the same role and the same scope. > When onboarding a mixed group, send one batch per role rather than one batch for > everyone. ## Track and revoke pending invitations The [Invitations](https://app.rulebase.co/settings/people/invitations) tab lists every invitation with its **Email**, **Role**, **Status**, and the date it was **Invited**. Organizations with partners configured also see a **Scope** column showing where that person's access will land. While an invitation is still **Pending**, its row has an overflow menu with a **Revoke** option. Revoking marks the invitation as **Revoked** and the link stops working. Once someone has accepted, the menu disappears; manage them from the [Members](https://app.rulebase.co/settings/people/members) tab instead, where you can change their role or remove them from the organization. ## What happens after someone accepts The recipient clicks the link in their invitation email and sets up their account. They then appear on the **Members** tab, and they can log in and work according to their role's permissions right away. There's no separate approval step on your side. ## Related * [Roles and permissions](/guides/roles-and-permissions) * [Create and manage teams](/guides/access-management/create-and-manage-teams) * [SSO setup](/guides/sso-setup) * [Identity provider role assignment](/guides/identity-provider-role-assignment) # Set up partners and scope access to them Source: https://docs.rulebase.co/guides/access-management/set-up-partners Create the partners your outsourced work belongs to, keep agent assignment automatic, and limit a partner's own reviewers to their own data. A partner is an organization whose people do work in your help desk: a BPO, an outsourcer, a sponsor bank. In Rulebase the partner rides on the agent, not the ticket: each agent belongs to at most one partner, and a ticket counts as that partner's when one of its agents does. Use partners to compare an outsourcer against your in-house team. Partners also scope access. A member's access can be limited to a partner, so a team lead at your BPO signs into your Rulebase and sees their own people and tickets rather than everybody's. > [!NOTE] > > The **Scope** field on invitations and memberships only appears once you have set > a **Partner custom field** on this page. Creating partners by hand does not > unlock it, so if you are here to restrict what an outsourced reviewer can see, > do the **Configure** step below even if you intend to assign people manually. ## Before you start * Partners live on the **Partners** tab of [Settings > People](https://app.rulebase.co/settings/people/partners), beside Members, Invitations, Agents, Teams, and Roles. * You need an admin role. Without it the tab is read-only: no **Create partner**, no **Configure**, no way to add or remove people. * For automatic assignment, you need a dropdown custom field on your people whose options are the partner names. Only dropdown fields work; a text, number, date, or yes/no field is offered in the picker, saves without complaint, and then never files anyone. ## Create a partner 1. Click **Create partner**. 2. Enter the **Name**. This is what appears in filters, routing rule conditions, and dashboard breakdowns, so use the name your team says out loud. 3. Optionally add **Domains**: type a domain and press Enter for each one. They are recorded against the partner and shown in the list; they are a label for your own reference rather than an access rule. 4. Choose the **Type**. The options depend on your organization's own type, so a fintech is offered Sponsor bank, BPO, and Other, while other organizations see the full set. 5. Click **Create partner**. ![Create partner dialog with a name, one domain entered as a tag, and BPO chosen as the type](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/partners-create-partner-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=5284736693d119a0853a82d003973da8) The list then shows each partner with its type, domains, and a **People** count. Click a row to open it. ![Partners tab listing three BPO partners with their domains and people counts, beside Create partner and Configure buttons](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/partners-settings-list.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=e20ee444a5fd6d6bcc8b9fdc8ae3b4fc) ## Let the roster maintain itself To keep assignments current as an outsourcer's headcount changes, point Rulebase at a field instead of assigning agents by hand. 1. Click **Configure**. 2. Under **Partner custom field**, search for the dropdown field on your people that names their partner and select it. Fields synced from a help desk are listed with their source in brackets, so a Zendesk dropdown works here as readily as one you created yourself. 3. Click **Save**. ![Set up partners dialog with a Partner dropdown field selected as the partner custom field](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/partners-set-up-partners-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=2bead1449f7ff99d61f53c0f79cb1cc8) From then on, setting that field on an agent files them under the matching partner, and Rulebase creates the partner itself the first time it sees an option it has no partner for, named after the option's label. Clearing the field unassigns them. > [!TIP] > > The sync happens when an agent's field value is saved, not retroactively. If your > people already carry values in that field, re-save them, or assign the first > batch by hand; existing values do not file themselves. ## Add and remove people by hand Manual assignment works alongside the field, and it is the quicker path for a handful of people. 1. Open the partner. 2. Under **People**, click **Add**. 3. Search for the person and select them. They are added immediately. 4. To take someone out, click the **×** at the end of their row. ![Bluewave Support partner page with a BPO badge and a People table listing three agents with their email addresses](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/partners-partner-people.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=a8b03fc94953243152d7470ec68f4b89) Removing a person unassigns them from the partner. It does not remove them from your agent roster, and their historic evaluations keep the partner that was recorded at the time. ## Scope a member to their partner Once a partner custom field is configured, **Specific partners** becomes one of the scope choices for new invitations and for people who already have access. [Invite members](/guides/access-management/invite-members) covers the invite dialog. For someone who is already a member: 1. Open the **Members** tab of [Settings > People](https://app.rulebase.co/settings/people/members). 2. Click their row to open **Edit membership**. 3. Set **Scope** to **Specific partners**, then pick the partners they should reach. **Entire organization**, **Specific groups**, and **Specific teams** are the alternatives. 4. Click **Save changes**. If **Scope** is greyed out, look at the **Role** above it: the control is disabled while the role is Administrator or Agent, because neither can be narrowed. ![Edit membership dialog for a member with the Manager role, Scope set to Specific partners, and one partner selected](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/partners-member-scope-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=d9b42372f1cb1455be1aa1866a008571) The **Members** and **Invitations** tabs both grow a **Scope** column once partners are configured; use it to audit who can see everything. A member scoped to one or more partners can reach only: * **Agents** belonging to those partners * **Teams** those partners' agents sit in * **Tickets** where one of those partners worked * **QA evaluations** whose scored agent belongs to those partners The restriction is enforced on the data, not just hidden in the interface, and it fails closed: a membership scoped to partners with nothing selected sees nothing rather than everything. Two roles ignore scope: administrators always have full access, and agents are always limited to themselves. See [Roles and permissions](/guides/roles-and-permissions). > [!NOTE] > > Deleting a partner unassigns its people and drops it from any membership scoped > to it. A member scoped only to that partner is left scoped to nothing, which means > an empty Rulebase rather than an error message. Re-scope those memberships in the > same sitting. ## Where partners show up afterwards **Filtering.** Partner is a filter dimension on interaction and evaluation lists. [Filter by team, partner, brand, and channel](/guides/quality-assurance/filter-interactions) explains why it sits under a **Ticket** heading despite describing the agent. **Routing.** **Partner** is available as a routing rule condition, so you can send one outsourcer's low scores to a different reviewer than your own team's. See [Create routing rules](/guides/quality-assurance/create-routing-rules). **Reporting.** Dashboard charts accept a Partner filter where their data source supports it, which is how a per-partner quality view gets built. See [Manage dashboard charts](/guides/insights/manage-dashboard-charts). ## Related * [Invite members](/guides/access-management/invite-members) * [Create and manage teams](/guides/access-management/create-and-manage-teams) * [Roles and permissions](/guides/roles-and-permissions) * [Filter by team, partner, brand, and channel](/guides/quality-assurance/filter-interactions) # Contest an evaluation Source: https://docs.rulebase.co/guides/agent-portal/contest-an-evaluation Dispute a criterion you think was scored wrongly, attach your evidence, and follow the review through to a decision. If a criterion was marked as failed and you think that is wrong, you can say so on the ticket itself. Contesting sends the specific criteria you name to a reviewer, along with your explanation, and they either change the result or tell you why they are keeping it. Disputes are a normal part of a QA program, and a clear, specific dispute is useful to your team even when the original result stands. ## When you can contest * The evaluation has to have **at least one failed criterion**. There is nothing to dispute on a clean result. * It must be **your current evaluation** for that ticket. If a reviewer has already replaced it, the newer one is the one that counts. * You have to be inside your organization's **dispute window**, counted in days from the date of the evaluation. The length is a setting, and some organizations turn it off so there is no deadline at all. * You can have **one open dispute per ticket** at a time. Once the reviewer finishes, you can raise another on the same ticket if the new result is still wrong. The **Contest score** button on the evaluation is disabled (not hidden) when one of these does not hold, and hovering it tells you which one. "The dispute window has closed" means the deadline passed; "Tickets cannot be contested while an active contest is open" means your earlier dispute is still with a reviewer. ## Start the dispute 1. Open the ticket from your [Evaluations](https://app.rulebase.co/agent/evaluations) list and read the criterion you disagree with, along with its reasoning. 2. Click **Contest score** above the breakdown. 3. Rulebase opens the contest form. The transcript sits on the left, and the failed criteria are listed in the middle with a checkbox each. 4. Tick the criteria you want reviewed. **Select All Evaluations** takes every failed criterion on that scorecard, and **Check results** expands the individual checks behind a criterion, which is where the specific judgement usually sits. ![The contest form with the ticket transcript on the left and failed criteria available to select on the right](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-portal-contest-form.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=5ac5862157a6e93ac1b2beafdd6f1054) Only failed criteria appear. If the scorecard you are looking at has none, switch scorecards with the picker at the top when the ticket was scored against more than one. ## Make your case Each criterion you tick opens its own block on the right, with two optional checkboxes and one thing you have to write: * **I disagree with the score**: tick this when the pass or fail is wrong. * **I disagree with the evaluation feedback**: tick this when the result may be fair but the reasoning misrepresents what happened. * **What did we get wrong with this evaluation?**: your explanation, and the only required field. **Submit evaluation contest** stays disabled until every criterion you selected has one, so a dispute cannot go out with a blank criterion attached to it. You can attach images as evidence under each criterion. This is the place for a screenshot of the policy you followed or the internal note you were working from. Attachments are optional and images only. ![A contest form filled in, with the criterion selected, I disagree with the score chosen, an explanation written, and Submit evaluation contest ready](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-portal-contest-form-filled.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=a245c3a16fd853c1d293f037cc0b6c4a) > [!TIP] > > Point at something checkable. "The refund timeframe I quoted is the one in the help > centre article on that date, screenshot attached" gives a reviewer something they can > confirm in a minute. "This is unfair" leaves them nothing to act on, even when you > are right. When you are ready, click **Submit evaluation contest**. ## What happens next Submitting marks the ticket as under review and puts it in front of a reviewer. On the ticket you will see an orange banner reading **This evaluation is currently being reviewed**. The score stays as-is while the dispute is open, and there is no way to edit or withdraw it once submitted, so reread your explanation before you send it. The reviewer works through your dispute criterion by criterion. They can accept your point and re-score that criterion, or keep the original result and explain their reasoning. Where they change the score, Rulebase produces an updated evaluation, and that one becomes the score that counts for you. The original stays readable as the history of the ticket. When the review is finished, the ticket shows a green **Your dispute has been reviewed** banner with a link to the updated evaluation. ## Follow it in Contested Evaluations Once you have raised your first dispute, a **Contested Evaluations** tab appears in your portal navigation. It lists your disputes in two groups. ![The Contested Evaluations tab with one unresolved dispute and two resolved ones, marked accepted and rejected](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-portal-contested-evaluations.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=7a5ba4c4ae7dd16408961e8473eaa527) * **Unresolved**: still with a reviewer. The status column reads **Unresolved** and there is nothing for you to do. * **Resolved**: decided. The status column shows **Accepted** when every point you raised was accepted, or **Rejected** when at least one was kept. **Rejected** does not always mean the whole dispute failed. It means at least one criterion was kept; the others may have been accepted. Click the row to open the ticket, then use **View results** next to **Evaluation contested** on the evaluation. Each criterion you raised is laid out three ways: the original result, your **Contest** with the comment and any images you attached, and **Results** with the outcome, the new score where your point was accepted, and the reviewer's written feedback. ![A resolved dispute showing the original failed criterion, the agent's comment, and a Rejected outcome with the reviewer's written feedback](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-portal-contest-results.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=79c07e03a4f75cff8b98c07276c41514) Read the feedback before deciding whether to raise a second dispute. Where a point was kept, the reviewer is usually naming the specific thing the transcript does not show; address that point if you dispute again. ## Related * [Read your evaluations and feedback](/guides/agent-portal/read-your-evaluations) * [Use the agent portal](/guides/agent-portal/overview) * [Rulebase glossary](/guides/concepts/glossary) # Use the agent portal Source: https://docs.rulebase.co/guides/agent-portal/overview What you can see and do in Rulebase as an agent: your scores, your weekly reports, your evaluations, and your coaching sessions. Your company uses Rulebase to review support tickets against a scorecard. The agent portal is your view of that: the scores on your own tickets, the reasoning behind each one, and any coaching logged for you. When you sign in with an agent account, Rulebase takes you straight to your performance view, and the pages you can open are the ones about you. ![The agent portal Overview tab showing the agent's profile card and average QA score](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-portal-overview.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=f4e47871feb50d4b3d3c1ce90636840f) ## Where you land Signing in opens [your performance overview](https://app.rulebase.co/agent/summary). Four tabs sit across the top: * **Overview**: your scores and trends for a date range. * **Reports**: written weekly summaries of how your tickets went. * **Evaluations**: every scored ticket of yours, newest first. * **Contested Evaluations**: only appears once you have disputed at least one evaluation. This is where you follow what happened to it. The card on the left is your profile: your name, your team, and the date you joined. Below it is **Coaching**, covered further down. ## What the overview shows The overview covers whatever date range is selected in the top right. It opens on the last 30 days, and **Download PDF** saves the whole page if you want a copy before a one-to-one. * **Average QA score**: the average across your evaluations in the range, with the change against the previous period of the same length. * **QA score trend**: the same score plotted over time, so a bad week reads as a dip and not your whole record. * **Tickets resolved** and **average handling time**: volume and speed for the range. Both are clickable; pick a point and Rulebase lists the tickets behind the number. * **SLA**: how your first responses and resolutions compared with the targets your organization set. * **Adherence by criteria**: each scorecard criterion with the share of your evaluations that passed, partially passed, or failed it. Adherence is usually the most useful section. It shows the pattern across tickets, and a criterion sitting at 80% pass tells you which habit to change first. ![Adherence by criteria showing pass and fail percentages for each scorecard criterion](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-portal-scorecard-adherence.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=4a1e8e1d17034840407bdfbc604b7d7b) ## Your weekly report **Reports** holds a short written summary for each week: how the week went, what held up well, and what needs attention. Pick a week from the list on the left. The current week is labelled, and earlier months stay collapsed until you open them. ![A weekly agent report with a written summary of the week's tickets](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-portal-weekly-report.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=e73ab8566a447db20e93077170fb70ab) The report draws from that week's evaluations, so the findings are the same ones you can open ticket by ticket under **Evaluations**, pulled into one place. Read the report first, then go to individual tickets when you want the full detail. ## Your evaluations and your coaching sessions **Evaluations** lists your scored tickets grouped by score, and opening one shows the criterion-by-criterion breakdown with the reasoning attached. That is the part most people come here for, and it has its own guide: [Read your evaluations and feedback](/guides/agent-portal/read-your-evaluations). If you disagree with a result, [Contest an evaluation](/guides/agent-portal/contest-an-evaluation) covers how to say so and what happens next. The **Coaching** tab on the profile card lists the one-to-one sessions your coach has logged for you, with the tickets they attached as examples. These are read-only; the coach writes them. ## What you will not find here Your portal is scoped to you. There is no team or agent filter, and no view of anyone else's scores or reports. > [!NOTE] > > Reviewers can mark an AI evaluation as right or wrong on a single criterion, > which corrects the model over time. Those controls are not shown in > the agent portal, so a criterion may have been discussed internally without > anything visibly changing on your copy. If a result looks wrong to you, > contesting it is the route that gets a person to look. ## Related * [Read your evaluations and feedback](/guides/agent-portal/read-your-evaluations) * [Contest an evaluation](/guides/agent-portal/contest-an-evaluation) * [Rulebase glossary](/guides/concepts/glossary) # Read your evaluations and feedback Source: https://docs.rulebase.co/guides/agent-portal/read-your-evaluations Open a scored ticket, read the criterion breakdown and the reasoning behind each rating, and understand how the score was reached. An evaluation is one score, on one of your tickets, against one scorecard. Everything that produced the number is on the page, criterion by criterion, with the reasoning written out and the transcript sitting beside it. ## Find the ticket 1. Open the **Evaluations** tab in the [agent portal](https://app.rulebase.co/agent/evaluations). 2. The list opens on the last 30 days. Adjust the date range in the top right if the ticket you want is older. 3. Scan the groups, or search by ticket subject using the magnifying glass. 4. Click a row to open it. ![The Evaluations tab listing scored tickets grouped into low, medium, and high scores](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-portal-evaluations-list.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=6588fede8053840c48e2aceb2a3b9d83) Each row carries the score, the customer, the ticket subject, and a small avatar for who evaluated it. Hover the avatar and you see **Evaluated by AI**, or the reviewer's name when a person scored it by hand. The **Low scores**, **Medium scores**, and **High scores** groups use thresholds your organization chooses, so the cut-offs may differ from another company's. Use the sliders icon to change grouping. **Type** groups by AI versus manual, and **No grouping** gives you a flat list, newest first. ## What the evaluation view shows Opening a row shows the score on the left, the ticket details and full transcript on the right, and the criterion breakdown down the middle. ![An open evaluation showing the quality score, criterion results, and the ticket transcript](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-portal-evaluation-detail.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=0c81d4184ca7a8b1b3e4507c97566b58) The tab under the score has your name on it. When a reviewer also scored the same ticket by hand, a **Manual evaluation** tab appears next to **AI review** and you can compare the two. A ticket handled by more than one person carries a separate evaluation for each of you, so the score at the top can differ from what a colleague sees on the same ticket. ## Read one criterion Each criterion in the breakdown is a question about the interaction, with a result and a short explanation: * **The question**: the criterion as it is written on your team's scorecard. * **The result badge**: usually **Pass**, **Partial**, or **Fail**, in whatever wording your scorecard uses, followed by the points as `25 / 25` or `0 / 25`. * **The reasoning**: a plain-language explanation of why that result was chosen, usually quoting or paraphrasing what happened in the ticket. * **View checks** or **Reason for rating**: expands the individual checks behind the criterion, where your scorecard uses them. * **View explanation**: expands the fuller justification, and the help-centre answer with links when the criterion was judged against your knowledge base. ![A failed criterion expanded to show its reasoning and the explanation behind the rating](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-portal-criterion-breakdown.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=fb3bd559faaa1362b5015ca2d64d0208) The reasoning is specific, so it usually points at one line in the transcript, and you can scroll the conversation on the right to that exact moment. If the reasoning quotes something you did not say, or applies a policy that has since changed, that is worth contesting. ## How the score is put together The score at the top comes from the points each criterion earned against the points it had available. Criteria are not always worth the same, so one fail on a heavily weighted question moves the number more than two fails on light ones, and adding up the badges in your head can leave you short of an explanation. Three scoring rules affect the total: * **Auto-Zero.** Some scorecards mark a criterion as critical. Failing it zeroes the whole scorecard, and the score shows an **Auto-Zero** label. The rest of the breakdown still shows what you passed. * **A capped score.** Your scorecard can cap the total when a particular criterion fails. When that happened, hovering the score shows the reason for the cap. * **Dependency rules.** One check can hang on another criterion's result. Where it does, the explanation opens with **Dependency rule** and spells out the connection. > [!TIP] > > Read **Adherence by criteria** on your Overview alongside individual evaluations. > One fail is a single ticket; the same criterion failing across several weeks is > a pattern a coach will raise. ## AI scores and manual evaluations Most tickets are scored by AI. When a reviewer scores the same ticket by hand and publishes that manual evaluation, the manual result **replaces** the AI score. Your averages, reports, and performance trends use the manual evaluation from that point on. The AI score stays visible under the **AI review** tab on the ticket so you can still see what the model thought. When a trainer submits a manual evaluation for validation, that evaluation changes your score immediately. If the validator corrects and publishes it, the validator's evaluation becomes the result that counts. ## When a score changes Scores are not always final. If you dispute a result and a reviewer agrees, the reviewer's corrected evaluation becomes the one that counts, and your Evaluations list shows that one. On the ticket you will see a green banner reading **Your dispute has been reviewed** with a link to the updated evaluation. While a dispute is still open, the ticket shows an orange **This evaluation is currently being reviewed** banner. The score stays as-is until the reviewer finishes. [Contest an evaluation](/guides/agent-portal/contest-an-evaluation) covers the whole path. ## Related * [Contest an evaluation](/guides/agent-portal/contest-an-evaluation) * [Use the agent portal](/guides/agent-portal/overview) * [How AI QA works](/guides/concepts/how-ai-qa-works) * [Rulebase glossary](/guides/concepts/glossary) # Create a managed agent Source: https://docs.rulebase.co/guides/agents/create-a-managed-agent Describe a job in plain language, let the builder turn it into an agent with triggers and tools, run it once, then publish it so it runs on its own. A managed agent is a piece of work you describe once and stop doing yourself: summarize yesterday's billing disputes, open a Jira issue for tickets that mention a chargeback, email a Monday digest to the leads. Write what you want in a sentence or two; Rulebase builds the steps. > [!NOTE] > > Rulebase uses the word "agent" for two unrelated things. This guide is about the > **AI agents you build**, which live under **Home > Agents**. The people on your > QA roster are also called agents and live under **Settings > People > Agents**. > See [Manage the agent roster](/guides/quality-assurance/manage-agent-roster) for > those, and the [glossary](/guides/concepts/glossary) for the distinction. ## What you need Access comes in three levels: | Permission | What it allows | | - | - | | View | Open the agent list, read an agent's setup, see its run history | | Run | The above, plus run an agent now and use the builder chat | | Manage | The above, plus create, edit, publish, duplicate, and delete agents | Administrators have all three; no other role has any until granted in the **Agents** permission group under [Settings > People > Roles](https://app.rulebase.co/settings/people/roles). If there is no **New** button on **Home > Agents**, you do not have Manage: you can read agents and their runs but not build one. ## Describe the job Open **Home > Agents** and click **New**. ![Home Agents tab with the New button, a featured Churn risk analyst template offering Deploy, and the Your agents, Customer service, Fraud, and Risk category tabs](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/managed-agent-templates.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=5646ee079f45135b7a013603eaa77d4c) The dialog asks what you want the agent to do, and your answer decides most of what follows. ![Create a new agent dialog asking what you want the agent to do, with a description of a weekday billing dispute summary and a Create agent button](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/managed-agent-create-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=d16d2eec9608b6dfa0050140982e8bf0) Brief it like a new colleague: what to look at, when, what to do with what it finds, and where the output should go. A vague brief like "check disputes" gives the builder nothing to work with. Then click **Create agent**. > [!TIP] > > For a common job, check the category tabs (**Customer service**, **Fraud**, > **Risk**) first. Templates and Rulebase-suggested agents there come pre-briefed; > **Deploy** creates one you can then adjust, usually faster than starting from a > blank box. ## Let the builder do the first draft Creating an agent drops you on its **Instructions** view with your description already sent to the builder chat. The builder is an agent itself: it writes the instructions, then lays out the steps the agent will take: what triggers it, which tools it calls, and where a human needs to approve something. Correct it as it works, through the chat box at the bottom (*Ask or edit this agent...*). "Only weekdays," "use the support queue, not sales," "email finance too": each message revises the agent in place. Instructions save themselves; there is no save button. Two tabs sit at the top of an agent: * **Instructions** is the build view: the name, the description, what the agent is told to do, its triggers and tools, and the steps it will run. * **Workbench** is where you talk to the agent and see what it has been doing. Name the agent while you are here. Every new agent is called **New agent**. Click the title to rename it, and add a description underneath so the next person knows what it is for. ## Check the triggers **Triggers** decide when the agent wakes up. Most agents want one of two: **On a schedule**: hourly, daily, weekly, or monthly, with a time and a timezone, plus specific weekdays for a weekly schedule or a date for a monthly one. The timezone matters: a "9am" digest built in one timezone lands mid-afternoon for a team in another. **On something happening**: a ticket created, updated, closed, or solved; a sales call ending; a Jira issue created or its status changing; a Slack message arriving; an email arriving; or a webhook from a system of your own. An agent can hold more than one trigger: "every weekday morning, and also whenever a high-priority dispute lands." How you change a trigger depends on the agent. When the builder has laid out steps (as with a new agent), triggers come from those steps; change them in the chat ("make it Tuesdays," "also run when a ticket is reopened"). Agents that run from instructions alone show a `+` next to **Triggers** for adding one by hand. ## Check the tools **Tools** are what the agent is allowed to touch; nothing outside the list is available to it. They are grouped by what they do: * **Rulebase**: **Web search**, and **Workspace data**, where you choose whether the agent may only read your Rulebase data or also write to it. * **Communication**: **Slack**, **Microsoft Teams**, and **Email**. Email sends only to recipients you approve. * **Data**: **Snowflake**, for querying your warehouse. * **Developer tools**: **Open API**, for an API of your own with an OpenAPI schema and key, and **MCP Server**. * **Integrations**: the help desks and platforms already connected to Rulebase. Review the list: an agent that can write to your data and email anyone should have that access on purpose. As with triggers, a step-based agent's tools follow from its steps, so "stop emailing finance, post it in `#billing` instead" is a chat message with no checkbox to toggle. Where a `+` is available, the tool panel can also connect a missing source on the spot instead of sending you to [Settings > Connections](https://app.rulebase.co/settings/connections). This creates a real, organization-wide connection that other agents and people can use too. For anything that reaches a customer, ask the builder for an approval step before that action: the agent pauses, the people you name get an email with a link, and the run continues only when someone approves. Ask for it by name. The builder will not always add one. ## Run it once before you trust it Click **Run**. If the agent needs input to start, a **Run agent** dialog asks for it; otherwise it starts immediately. Then read what it did in the activity panel below the instructions. It lists recent runs; expanding one shows each step in order, which tools it called, what came back, and where it stopped. Hand anything unclear to the chat with **Ask about this run**. A first run that is nearly right is normal. Tell the builder what was off and run it again. ## Publish it Until you publish, the agent is a draft: it runs when you click **Run** and at no other time. **Publish** turns it on, and the button afterwards shows its state: | Button | Meaning | | - | - | | **Publish** | Never published. Triggers are not firing. | | **Live** (green) | Published, and what you see is what is running. | | **Update** (orange) | Published, but you have edited it since. The old version is still the one running. | Editing a live agent does not change its behaviour until you publish again, so a running agent keeps working while you rework it. Clicking **Live** or **Update** opens **Publish version**, which lists every version with who published it and when. From a version's `...` menu, **Restore** brings that version's setup back, undoing a change that made things worse. **Unpublish** stops the agent from running without deleting anything. Back on **Home > Agents**, the filter next to the search box splits the list into **Published** and **Draft**. > [!TIP] > > Before a risky change to an agent people rely on, **Duplicate** it from the > agent's `...` menu and experiment on the copy. Deleting an agent is permanent and > takes its run history with it. ## Related * [Glossary](/guides/concepts/glossary) * [Manage the agent roster](/guides/quality-assurance/manage-agent-roster) * [Use Instruct](/guides/instruct/use-instruct) * [Manage integrations](/guides/integrations/manage-integrations) # Record agent releases from CI Source: https://docs.rulebase.co/guides/agents/record-agent-releases Send a release to Rulebase on every production deploy so Agent monitoring can mark capabilities shipped and compare before and after. A release marks when a new version of your AI agent went live. Agent monitoring uses releases to mark a capability as shipped when you link one to it, and to compare how often the capability fails before and after that version. You can add releases by hand on the agent's page under **Settings > Agent monitoring > Agents**. To record every deploy automatically, call the releases API from your deploy pipeline. ## What you need * An organization API key. See [API access](/guides/api-access). * The agent ID. It's shown in the **Releases** section of the agent's page under **Settings > Agent monitoring > Agents**. ## Add a deploy step After your production deploy succeeds, send the agent ID, a version name, and the commit: ```bash theme={null} curl --fail-with-body -X POST https://api.rulebase.co/v1/agent_releases \ -H "Authorization: Bearer $RULEBASE_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "agent_id": "'"$RULEBASE_AGENT_ID"'", "name": "v1.4.2", "commit_sha": "'"$GITHUB_SHA"'", "released_at": "'"$(date -u +%Y-%m-%dT%H:%M:%SZ)"'", "notes": "Adds the transfer status tool" }' ``` Organizations in the EU region use `https://eu.api.rulebase.co`. `released_at` defaults to the time of the request. `notes` is optional. ### GitHub Actions ```yaml theme={null} - name: Record release in Rulebase if: success() env: RULEBASE_API_KEY: ${{ secrets.RULEBASE_API_KEY }} RULEBASE_AGENT_ID: ${{ vars.RULEBASE_AGENT_ID }} run: | curl --fail-with-body --retry 3 -X POST https://api.rulebase.co/v1/agent_releases \ -H "Authorization: Bearer $RULEBASE_API_KEY" \ -H "Content-Type: application/json" \ -d "{\"agent_id\": \"$RULEBASE_AGENT_ID\", \"name\": \"${GITHUB_SHA::7}\", \"commit_sha\": \"$GITHUB_SHA\"}" ``` ## Retries Sending the same `commit_sha` again returns the release that already exists with status `200` instead of creating a second one. Without a `commit_sha`, the release `name` is used instead. Retrying a failed deploy step is safe. ## Reference [Record an agent release](/api-reference/agent-releases/create-agent-release) # Set up API access Source: https://docs.rulebase.co/guides/api-access Create a Rulebase API key, store it safely, and send requests to the correct regional base URL for your organization. Anything you send to Rulebase programmatically travels over an API key: pushing conversations from a helpdesk we don't have a native integration with, or filing back-office work items from your internal admin tool. One key is usually enough to get started. The base URL is region-specific. Rulebase runs separate US and EU deployments, and a key issued in one region does not work in the other. A valid key sent to the wrong host returns the same `401 Unauthorized` as a mistyped one, so the error message won't tell you which mistake you made. ![API keys section of Rulebase Connections settings, listing two active keys with their truncated key prefix and created date](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/api-keys-settings-list.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=01b2b10c53dbe74e91ba30526cf188b7) ## Before you start * You need permission to manage integrations. Admin roles have it; if the **Create API key** button is greyed out, that's what's missing. * Know which region your organization is on; the next section explains how to tell. * Have somewhere to put the key before you create it. A secrets manager or your deployment platform's environment variables are both fine. A chat message is not, and you only get one chance to copy it. ## Create a key API keys live on the **API keys** tab in Connections settings. 1. Go to [Settings > Connections](https://app.rulebase.co/settings/connections). 2. Open the **API keys** tab. 3. Click **Create API key**. 4. Copy the key from the **API key created** dialog, using the copy button next to it. 5. Paste it into your secret store, then click **Done**. ![API key created dialog showing the full API key with a copy button and a warning that the key cannot be viewed again](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/api-keys-created-dialog.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=92a6372b6b280cff8235912042fff732) There are no options to fill in. The key is generated the moment you click, works straight away, and never expires on its own. > [!NOTE] > > The dialog is the only place the full key is ever shown. Once you close it, > Rulebase can't display the key again, and there is no "reveal" action on the > list. If you lose a key, create a new one and revoke the old one. Production keys start with `rk_live_`. Back on the list, each row shows only the first 16 characters followed by an ellipsis, which is enough to tell your keys apart when you come back to revoke one, and not enough to authenticate with. ## Find your region's base URL Your region is the one you sign in to. If you log in at `app.rulebase.co` you're on US; if you log in at `eu.app.rulebase.co` you're on EU. There's no region setting to look up, and no way to change it yourself. Rulebase exposes one public API host per region: | Surface | US | EU | | - | - | - | | App | `https://app.rulebase.co` | `https://eu.app.rulebase.co` | | API | `https://api.rulebase.co` | `https://eu.api.rulebase.co` | Every public endpoint is versioned under `/v1`. Existing integrations that use deprecated aliases continue to work during their sunset period, and those responses include `Deprecation`, `Sunset`, and successor `Link` headers. Use the canonical `/v1` path for new integrations and whenever you update an existing one. > [!TIP] > > If you're on EU, put the base URL in one configuration value that every call site > reads, rather than writing the host into each request. Otherwise a single unprefixed URL can survive into production and show up > later as a `401` that looks like a credentials problem. ## Send your first request Authenticate by putting the key in an `Authorization: Bearer` header. Listing conversation uploads is a good first call because it only reads, so you can run it against a live organization without changing anything: ```bash theme={null} curl "https://api.rulebase.co/v1/conversation_uploads?limit=1" \ -H "Authorization: Bearer YOUR_API_KEY" ``` A working key returns `200` with an envelope, even when your organization hasn't uploaded anything yet: ```json theme={null} { "data": [], "meta": { "page": { "next": null, "limit": 1 } } } ``` A key that Rulebase doesn't recognize returns `401` with this body in either region: ```json theme={null} { "error": "Unauthorized" } ``` If you get the `401`, check these three things: * **The host.** If you sign in at `eu.app.rulebase.co`, confirm you're calling `eu.api.rulebase.co` and not `api.rulebase.co`. Rule this out first, because nothing in the response hints at it. * **The header.** The value is the word `Bearer`, a space, then the key. A trailing newline from a shell variable is enough to break it. * **The key itself.** Check the first 16 characters against the list in [Settings > Connections](https://app.rulebase.co/settings/connections). If the row isn't there, the key was revoked. > [!TIP] > > `rulebase doctor` (from `npm install -g @rulebase/cli`) tries your key against > both regions and names the one that accepts it, which settles the first bullet > in one command. See the > [Rulebase CLI](/guides/cli) guide. ## What a key carries Every key has the same properties: * **Organization-wide access.** A key authenticates as your organization, not as the person who created it. Requests made with it aren't attributed to you and aren't limited by your role. * **No per-key scopes.** There are no read-only keys, no per-endpoint permissions, and no IP restrictions. Every key can reach every endpoint that accepts API-key authentication. * **Tied to one region.** Each region keeps its own key store, so a key created in the EU app authenticates only against EU hosts. * **No expiry.** A key stays valid until someone revokes it. Because a key is org-wide and open-ended, treat it as a shared production credential. Creating one key per integration costs nothing and means you can retire a single system later without breaking the others. ## Revoke a key Revoking is a delete. There's no disable-and-re-enable, so only do this once you're sure the key is out of service or you know it's leaked. 1. Go to [Settings > Connections](https://app.rulebase.co/settings/connections) and find the key in the **API keys** list, matching on its prefix. 2. Open the overflow menu at the end of its row. 3. Click **Delete**. 4. Confirm with **Delete** in the dialog. ![Confirmation dialog warning that deleting an API key is permanent and that applications using the key will lose access](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/api-keys-delete-confirmation.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=42aa16230b09ac41ee66adfbf7357a70) The key stops working straight away, and anything still sending it starts getting `401 Unauthorized`. Rulebase keeps the data that was uploaded with the key; only the credential goes away. To rotate rather than retire an integration, create the replacement key and deploy it first, then come back and delete the old one. ## Related * [API reference](/api-reference/authentication/get-api-key-identity) * [Upload a conversation](/api-reference/conversations/upload-a-conversation) * [List conversation uploads](/api-reference/conversations/list-conversation-uploads) * [Uploading conversations](/guides/uploads/conversations) * [Confirm received uploads](/guides/uploads/confirm-received-uploads) * [Uploading work items](/guides/uploads/work-items) * [Rulebase CLI](/guides/cli) * [Roles and permissions](/guides/roles-and-permissions) # Zendesk app Source: https://docs.rulebase.co/guides/apps/zendesk Install the Rulebase app from the Zendesk Marketplace to give agents a Rulebase panel in the ticket sidebar. The Rulebase app puts a Rulebase panel in the Zendesk ticket sidebar, so an agent working a ticket can see what Rulebase knows about it and act on it without leaving Zendesk. This is the opposite direction from the [Zendesk integration](/guides/integrations/zendesk), which connects Zendesk to Rulebase so tickets sync in for review. The app is how Rulebase shows up inside Zendesk, and it depends on that integration: the panel finds your organization through your connected Zendesk account, so connect Zendesk in Rulebase first. ## Prerequisites * You must be a Zendesk admin to install apps. * Zendesk must already be connected in Rulebase. See [Zendesk](/guides/integrations/zendesk). * You need a Rulebase API key from [Settings > Connections](https://app.rulebase.co/settings/connections). ## Installation ### Step 1: install the app 1. Visit the [Rulebase Zendesk App](https://www.zendesk.co.uk/marketplace/apps/support/1159969/rulebase/) in the Zendesk Marketplace 2. Click **Install** to add Rulebase to your Zendesk account ### Step 2: configure the integration 1. In Zendesk Admin Center, go to **Apps and Integrations** > **My Apps** 2. Select **Rulebase** and open the settings panel 3. Enter your Rulebase API key ### Step 3: have each agent sign in The first time an agent opens the panel it asks them to log in to Rulebase, with a **Log in** button that opens a Rulebase login window. Each agent signs in with their own Rulebase account, once per browser. ## What agents see in the ticket sidebar The panel reads the open ticket and shows the matching Rulebase conversation: * **The customer and the ticket reference**, with the ticket number, a copy button, and when the conversation started * **Ticket summary**, Rulebase's own summary of what the ticket is about, rather than a copy of the first message * **Badges** for what Rulebase flagged on the conversation, such as customer sentiment, a detected complaint, an SLA breach, or compliance risk * **Linked activity**, the related tickets, Jira issues, and Slack messages Rulebase has tied to this conversation, so an agent can see the earlier ticket on the same issue * **Actions**, where the agent can create a Jira ticket from the conversation. When Rulebase has already drafted one, it appears as a **Suggested** action with the request type filled in. Creating the ticket also posts a private internal note on the Zendesk ticket with a link to the new Jira issue. * **Ask a question**, a prompt box at the bottom that opens a Rulebase chat in the panel, so an agent can look something up mid-ticket instead of switching tabs If the ticket has not synced to Rulebase yet, the panel imports it on the spot, so a brand new ticket can take a moment to fill in. A ticket Rulebase cannot match shows **No ticket found** instead, and the usual cause is that the Zendesk account holding the ticket is not the one connected to Rulebase. # Rulebase CLI Source: https://docs.rulebase.co/guides/cli Push, read, and reconcile work items from a terminal, an agent, or a CI job with @rulebase/cli: JSON in, JSON out, no interactive prompts. `@rulebase/cli` is the Rulebase command line. Beyond the credential helpers (`doctor`, `login`, `whoami`), it exposes the whole work-item API as non-interactive commands that read JSON and print JSON. That makes it the shortest path from a script, a coding agent, or a CI job to the [`/v1/work_items`](/api-reference/work-items/create-work-item) endpoints, without writing an HTTP client or handling regional hosts yourself. ## Install and run The CLI is a zero-dependency Node package. Node 20 or newer is required. ```bash theme={null} # Install once, then call it by name npm install -g @rulebase/cli rulebase work-items list --limit 5 ``` The package installs a single `rulebase` binary. Prefer the global install on a workstation or an agent image: the command is short, and there is no npm registry round trip on every invocation. In CI, add it as a dev dependency instead, so the version is recorded in your lockfile rather than resolved at job time: ```bash theme={null} npm install --save-dev @rulebase/cli npx rulebase work-items list ``` Every example below assumes `rulebase` is on your `PATH`. If you would rather not install anything, prefixing any command with `npx @rulebase/cli` works the same way. `rulebase --help` lists every command and the exit codes in one screen. ## Authenticate Work-item commands authenticate with a **Rulebase API key**, the same key you would put in an `Authorization: Bearer` header. Create one in [Settings > Connections](https://app.rulebase.co/settings/connections); see [Set up API access](/guides/api-access) for the details. > [!NOTE] > > The OAuth login (`rulebase login --region us|eu`) exists for MCP tool calls only. > REST work-item commands never use it. If `whoami` shows you are signed in with > OAuth but no API key is configured, the work-item commands will still fail with > a credentials error. Two ways to supply the key: ```bash theme={null} # 1. Environment variable — best for CI export RULEBASE_API_KEY=rk_live_... rulebase work-items list # 2. Stored credential — best for a workstation export RULEBASE_API_KEY=rk_live_... rulebase login --auth api-key # verifies the key, then stores it unset RULEBASE_API_KEY ``` Without `RULEBASE_API_KEY`, `login --auth api-key` prompts for the key with the input hidden. **The CLI never accepts an API key as a command-line argument**, because argv shows up in shell history, `ps` output, and CI transcripts. ### Where credentials live, and which one wins Stored credentials go in `~/.rulebase/credentials.json`, in a directory created with mode `0700` and a file written with mode `0600`. Override the location with `RULEBASE_CREDENTIALS_PATH`. Precedence, highest first: 1. `RULEBASE_API_KEY` from the environment. 2. The stored key for the region you passed with `--region`. 3. The stored key, when only one region has one. If keys are stored for both regions and you do not pass `--region`, the CLI stops rather than guessing, and tells you to pick one. ### US and EU Rulebase runs separate US and EU deployments with separate key stores, and a key sent to the wrong region returns exactly the same `401 Unauthorized` as a revoked key. The CLI resolves the region for you: it uses `--region` (or `RULEBASE_REGION`) when given, the region the key is stored under otherwise, and probes both regions when it has neither. ```bash theme={null} rulebase work-items list --region eu RULEBASE_REGION=eu rulebase work-items list ``` ## Output contract Every work-item command writes exactly one JSON object to stdout and nothing else. Progress and warnings go to stderr, never secrets. ```json theme={null} { "ok": true, "data": { "id": "0197ac58-70c9-7bc0-9ba8-1bb7ce8f78cc" } } ``` ```json theme={null} { "ok": false, "error": { "code": "unauthorized", "message": "The API key was rejected (401).", "details": { "region": "us" } } } ``` Exit codes let a script branch without parsing the body: | Code | Meaning | | - | - | | `0` | Success | | `1` | A batch apply completed with at least one failed item | | `2` | Usage, malformed JSON, or schema validation error | | `3` | Missing, ambiguous, or wrong-region credentials | | `4` | Network failure or an API error response | ## Commands All commands take `--region us\|eu` and read input from a file path or `-` for stdin. ### `work-items create` Creates a work item from a JSON object. Create-only: a duplicate `external_id` returns exit code `4` with `error.code` `conflict`. ```bash theme={null} cat > item.json <<'JSON' { "external_id": "TASK-9182", "agent_email": "alex@example.com", "type": "fraud_review", "status": "completed", "completed_at": "2026-05-12T14:32:00Z", "custom_attributes": { "risk_score": 78 } } JSON rulebase work-items create --file item.json ``` ### `work-items get` By Rulebase ID or by your own identifier: ```bash theme={null} rulebase work-items get 0197ac58-70c9-7bc0-9ba8-1bb7ce8f78cc rulebase work-items get --external-id TASK-9182 ``` ### `work-items list` Newest completion first, cursor paginated: ```bash theme={null} rulebase work-items list --limit 50 --status completed rulebase work-items list --cursor "$CURSOR" rulebase work-items list --status pending --status in_progress rulebase work-items list --completed-after 2026-05-01T00:00:00Z rulebase work-items list --query TASK-91 rulebase work-items list --all # follows every page ``` `meta.page.next` carries the cursor for the next page and is `null` on the last one. With `--all` the CLI walks the pages itself and returns one combined array. ### `work-items update` Patches a stored work item. Omitted fields keep their values, `custom_attributes` merge with what is stored, and `external_id` cannot change. ```bash theme={null} echo '{"status":"completed","completed_at":"2026-05-12T14:32:00Z"}' \ | rulebase work-items update 0197ac58-70c9-7bc0-9ba8-1bb7ce8f78cc --file - ``` ### `work-items delete` Soft-deletes a work item and requires `--yes`, so no script deletes anything by accident: ```bash theme={null} rulebase work-items delete 0197ac58-70c9-7bc0-9ba8-1bb7ce8f78cc --yes ``` The call is idempotent: deleting an already-deleted work item still succeeds and reports `"deleted": false`. ### `work-items apply` The batch workhorse. Creates, updates, or restores work items by `external_id`. ```bash theme={null} rulebase work-items apply --file work-items.json cat work-items.json | rulebase work-items apply --file - rulebase work-items apply --file work-items.json --dry-run ``` The canonical input is the same envelope the API takes: ```json theme={null} { "work_items": [ { "external_id": "TASK-9182", "agent_email": "alex@example.com", "type": "fraud_review" }, { "external_id": "TASK-9183", "type": "kyc_verification", "custom_attributes": { "risk_score": 12 } } ] } ``` A bare array is normalized to that envelope, so `[{...},{...}]` works too. Input is capped at 25 MB. The API accepts 100 items per request; the CLI splits anything larger into chunks of 100 automatically and stitches the responses back together, so `index` values in the output are positions in **your** input, in your order: ```json theme={null} { "ok": true, "data": { "dry_run": false, "results": [ { "index": 0, "external_id": "TASK-9182", "status": "updated", "data": { "id": "0197..." } }, { "index": 1, "external_id": "TASK-9183", "status": "created", "data": { "id": "0198..." } } ], "summary": { "total": 2, "created": 1, "updated": 1, "unchanged": 0, "restored": 0, "error": 0 } }, "meta": { "chunks": 1, "chunk_size": 100 } } ``` Item statuses are `created`, `updated`, `unchanged`, `restored`, or `error`. If any item fails, the successful items are still applied, the envelope reports `"ok": false` with `error.code` `partial_failure`, the full `results` array is still present, and the exit code is `1`. `--dry-run` returns the same shape with `"dry_run": true` and writes nothing. ### `work-items schema` Prints the bundled JSON Schema for the three input shapes, so an agent can validate before sending: ```bash theme={null} rulebase work-items schema | jq '.data.work_items_batch' ``` The bundle ships with the package and is checked against the published OpenAPI document on release, so it cannot silently drift from the API. ## Agent and CI recipes Reconcile a day of records, failing the job if any item is rejected: ```bash theme={null} #!/usr/bin/env bash set -euo pipefail export RULEBASE_API_KEY="$RULEBASE_API_KEY" # from your secret store ./export-work-items.sh > /tmp/work-items.json rulebase work-items apply --file /tmp/work-items.json --dry-run > /tmp/plan.json jq '.data.summary' /tmp/plan.json rulebase work-items apply --file /tmp/work-items.json > /tmp/result.json ``` `set -e` turns the exit code into a failed job: `1` means some items were rejected (inspect `/tmp/result.json` for the `error` entries), `3` means the credential is missing or pointed at the wrong region, `4` means the API or the network failed. Pipe a single record straight out of another tool: ```bash theme={null} jq -n '{external_id:"TASK-9182",type:"fraud_review"}' \ | rulebase work-items create --file - ``` Collect every completed work item from a date: ```bash theme={null} rulebase work-items list --all --status completed \ --completed-after 2026-05-12T00:00:00Z \ | jq '.data | length' ``` ## JSON apply vs CSV upload Both write work items; they suit different sources. | | `work-items apply` | CSV upload | | - | - | - | | Input | JSON objects you already hold | A CSV export | | Endpoint | `POST /v1/work_items/batch` | `POST /v1/work_items/upload` | | Timing | Synchronous | Asynchronous job you poll | | Feedback | Per item: created, updated, unchanged, restored, error | Per file: row counts | | Preview | `--dry-run` | None | | Events | Inline per work item | Not supported | Use apply from agents and services. Use [CSV upload](/guides/uploads/work-items) when a human or a legacy system produces a spreadsheet. ## Behaviour notes **Retries and idempotency.** Apply is keyed on `external_id`, so re-running the same input is safe: items that did not change come back as `unchanged`. `create` is not idempotent: the second attempt returns `conflict` so you find out that you sent the same record twice. The CLI retries `429` and transient network failures a few times with backoff, honouring `Retry-After`; when it gives up, `error.details.retry_after` carries the seconds the API asked for. **Rate limits.** The API allows roughly 5,000 requests per minute per organization. Batching 100 items per request keeps a large backfill comfortably below it. **Deletion and restoration.** `delete` is a soft delete: the work item leaves reads and lists and stops being summarized, while evaluations that reference it keep resolving. Applying the same `external_id` again restores it and reports `restored`. There is no hard delete and no bulk delete, and items you leave out of an apply are never deleted. **Credential safety.** Keys are only ever read from the environment or the `0600` credential file, never from argv. Output is redacted: the CLI prints a key family and length (`rk_live_…32 chars`), never the key. ## Troubleshooting **`unauthorized` (exit 3 or 4) that looks like a bad key.** Nine times out of ten this is the region. A key issued in EU returns the same `401` against US as a revoked key does. Run: ```bash theme={null} rulebase doctor ``` `doctor` tries both regions and names the one that accepts your key; then pass `--region eu` or set `RULEBASE_REGION=eu`. **`credentials_ambiguous` (exit 3).** Keys are stored for both regions. Pass `--region`. **`invalid_json` or `invalid_input` (exit 2).** The file or stdin did not parse, or an item is missing a required field. Compare against `work-items schema`; note that `completed_at` is required whenever `status` is `completed` or `cancelled`. **Exit 1 with `partial_failure`.** The request reached the API and most items landed. Look at the entries in `data.results` whose `status` is `error`; their `error` message names the field at fault. Fix and re-apply; items that already landed come back `unchanged`. ## Related * [Set up API access](/guides/api-access) * [Uploading work items](/guides/uploads/work-items) * [Work item API reference](/api-reference/work-items/apply-work-items) # Coaching overview Source: https://docs.rulebase.co/guides/coaching/overview Turn QA results into logged 1:1 coaching sessions grounded in real tickets and their scores. Coaching connects your QA program to agent development. A **coaching session** is a logged 1:1 record you keep for one agent, built from real tickets and their QA scores. Use it to document a coaching conversation and give the agent something to refer back to. If you don't see **Coaching** in the sidebar, ask an admin to enable it under [Settings > Features](https://app.rulebase.co/settings/features). ## What a coaching session contains | | Coaching sessions | | - | - | | **Purpose** | Document real coaching, grounded in real tickets | | **Where you create it** | The agent's performance page | | **What it contains** | A date, markdown notes, and linked tickets + QA scores ("evidence") | | **Who does the work** | The coach writes it; the agent reads it | | **Scoring** | None — it's a record, not a test | > [!NOTE] > > The **Coaching** page in the sidebar opens the coaching performance view for > your organization. Individual coaching **sessions** live on each agent's > performance page. See [Run a coaching session](/guides/coaching/run-a-session). ## From evaluation to coaching Coaching is most useful when it's tied to evidence: 1. AI QA (and your reviewers) evaluate interactions and produce scores. 2. On an agent's performance page you spot a pattern worth addressing. 3. You log a **coaching session**, attaching the specific tickets and evaluations that illustrate the point, plus your notes. 4. Later evaluations show whether the coaching landed. ## Who can do what * **Coaches and admins** create and edit coaching sessions. * **Agents** see their own coaching sessions read-only. ## Related * [Run a coaching session](/guides/coaching/run-a-session) * [Use team and agent performance](/guides/insights/team-and-agent-performance) # Run a coaching session Source: https://docs.rulebase.co/guides/coaching/run-a-session Log a 1:1 coaching session for an agent, grounded in real tickets and their QA scores. A coaching session is where you capture a one-on-one coaching conversation with an agent. Ground it in their QA results: pull in the specific tickets and evaluations ("evidence") that prompted the conversation, add your notes, and the agent has a clear record to work from after you meet. ## Before you start * **Coaching** must be enabled for your organization ([Settings > Features](https://app.rulebase.co/settings/features)). * You need a coach or admin role. Agents can view their own sessions but can't create or edit them. * Have the tickets or evaluations you want to reference in mind; sessions are most useful when grounded in real interactions. ## Create a session Coaching sessions live on the agent's performance page, not the Coaching page. The score trend, the criteria the agent keeps failing, and the tickets behind both are already in front of you there. [Use team and agent performance](/guides/insights/team-and-agent-performance) covers reading that page before you book the 1:1. 1. Open the agent's performance page ([Insights > Performance](https://app.rulebase.co/insights/performance?view=teams), then drill into the agent, or open the agent from any evaluation). 2. In the **Coaching sessions** timeline, click **New session**. 3. Pick the coaching **date**. New sessions default to now; change it to when the coaching happened or is planned. 4. Save. The session opens so you can add notes and evidence. You can only mark a session as **delivered** once its meeting date is in the past (or now). If the date is still in the future, update it to when coaching actually happened first. ![Creating a coaching session from an agent's performance page](https://mintcdn.com/rulebase/r5Wr8jebJAzAsiac/images/coaching-session-new.png?fit=max\&auto=format\&n=r5Wr8jebJAzAsiac\&q=85\&s=ea4135cf52267114e49dd7572c0a88d9) > [!TIP] > > A session dated in the future shows up as **upcoming** on both the agent's > timeline and your own upcoming-coaching list, so you can plan sessions ahead > of time. ## Add evidence: tickets and evaluations Evidence links a coaching point to a real ticket and its QA result. 1. In the open session, add a ticket by its conversation. 2. Rulebase attaches the ticket's QA score. If you don't choose a specific evaluation, it automatically uses the best completed evaluation for that ticket and this agent, preferring a human evaluation over an AI one, and the most recent. 3. Add per-ticket **feedback** to explain what you want the agent to notice. You can also attach a ticket to a session directly from a conversation or evaluation using **Add to coaching**, without opening the agent's page first. > [!NOTE] > > An evaluation can only be attached as evidence when it belongs to the same > ticket **and** the agent being coached. Rulebase blocks mismatched evidence so > a session never references another agent's score. ## Write notes Session notes use markdown, so you can structure the conversation with headings, bullets, and links. Keep notes specific and tied to the evidence you attached. ![A coaching session with notes and a Tickets panel for attached evidence](https://mintcdn.com/rulebase/aOYyUmDGmyfYo0gr/images/coaching-session.png?fit=max\&auto=format\&n=aOYyUmDGmyfYo0gr\&q=85\&s=6fa689e09eafee9c4b3f4815a5a43df9) ## Share a session Use **Copy link** in the session to share a direct URL (`/coaching/sessions/{id}`). The link resolves the session and opens it on the agent's performance page, so recipients land in the right place. ## What the agent sees The agent opens the same **Coaching sessions** timeline on their own performance view, but read-only: they can read your notes, open the linked tickets, and see the QA scores, but they can't add, edit, or delete anything. There's no acknowledgement step; the record stays available. ## Related * [Coaching overview](/guides/coaching/overview) * [Use team and agent performance](/guides/insights/team-and-agent-performance) * [Schedule recurring exports](/guides/insights/schedule-exports) * [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster) # Configure complaint taxonomy Source: https://docs.rulebase.co/guides/compliance/complaint-taxonomy Define the complaint categories Rulebase matches tickets against, write descriptions and keywords it can classify from, and set the risk level that becomes each case's priority. Complaint detection has no built-in idea of what counts as a complaint at your firm. It works entirely from the categories you define: a ticket is compared against them, and a case is opened only when one of them fits. The taxonomy therefore decides what your complaints register contains. Too few categories and real complaints go undetected; overlapping categories and every case arrives carrying several tags. The taxonomy lives under **Settings > Complaints**, separate from the complaint cases your team works under **Compliance**. ## Before you start * You need an Administrator or Quality Analyst role. **New** and **Edit** are visible but disabled for everyone else, including the Compliance Analyst role, so ask an administrator if you cannot click them. * Decide whether a category applies across the whole organization or to one workspace before you create it. Detection only considers categories that apply to the ticket's workspace. * If your firm already reports complaints against a published taxonomy, use those names here. The category name is what appears as the tag on every case. ## Add a category 1. Go to **Settings > Complaints**. 2. Click **New complaint type**. 3. Fill in the **Name** and **Description**. 4. Leave **Workspace** as **All workspaces**, or pick the single workspace the category applies to. 5. Add the wording customers use with **Add keyword**, one phrase per row. 6. Set the **Risk level**. 7. Click **Save**. The new type appears in the list with its description, workspace scope, channels, and risk level. Select a type to edit it on its own page. The page's actions menu also lets you delete it. ## What each field does * **Name** — the category as it appears on a case and in the **Category** filter. Short and recognisable. * **Description** — the definition detection classifies against. This is where the work goes: say what belongs in the category and, where it helps, what does not. `Complaints about a fee, charge, or interest amount the customer did not expect or believes was applied incorrectly` gives the model something to judge; `Fee issues` does not. * **Workspace** — **All workspaces**, or one connected help desk. A workspace-scoped category is invisible to tickets from anywhere else, which is the way to run a different taxonomy for a separate brand or entity. * **Keywords** — the phrases your customers actually use, such as `payment blocked` or `cannot withdraw`. They are supplied to the model alongside the name and description as hints about wording. * **Risk level** — **Low**, **Medium**, or **High**. It becomes the priority of any case matched to this category. > [!NOTE] > > Keywords are hints, not filters. A ticket can be classified into a category without containing any of > its keywords, and containing one does not force a match. Use them to teach the vocabulary your > customers use, and rely on the description for the actual boundary of the category. ## How risk level drives the case The risk level you set here is the same value the complaints list calls priority. When a ticket matches more than one category, the highest risk level among them wins. | Risk level | On the case | | - | - | | **High** | Groups under **High** on the complaints list and can raise an immediate email or Slack alert. | | **Medium** | Groups under **Medium**. No alert. | | **Low** | Groups under **Low**. No alert. | Alerting for high cases is configured once for the organization in [Configure notifications](/guides/settings/configure-notifications), so the risk level you choose here decides which cases trigger those alerts. ## Keep the taxonomy working Aim for a small set of categories that a person could sort complaints into without hesitating. When two categories overlap, detection tends to apply both, and the case's priority quietly follows the more severe one. Changes take effect on future detection only. Editing a description does not reclassify complaints that have already been raised, and **Delete** takes a category out of detection from that point onward while cases already tagged with it keep their tag. If a category is producing cases you do not consider complaints, tighten its description rather than deleting it, then check the next few days of detections on the complaints list. A category with no keywords still works. A taxonomy with no categories does not: detection has nothing to match against, and no cases are created at all. ## Related * [Work a complaint case](/guides/compliance/work-a-complaint-case) * [Configure notifications](/guides/settings/configure-notifications) * [Rulebase glossary](/guides/concepts/glossary) # Populate a compliance test Source: https://docs.rulebase.co/guides/compliance/populate-tests Draw a random sample of conversations, disputes, cases, or alerts into a test sheet, run the test, and verify the sample before reviewing exceptions. Populate a compliance test with a random sample from its testing period. Rulebase supports tests based on conversations, disputes, compliance cases, and compliance alerts. ## Before you start Confirm the compliance program has the correct target record type and the test has the intended date range. The **Populate test** dialog uses those settings for the sample. The program and its tests are created first, in [Set up a compliance testing program](/guides/compliance/set-up-a-testing-program). If you have no test to open yet, start there. ## Add a random sample 1. Go to **Compliance > Testing**. 2. Open the program and test. 3. Click **Build sample** for an empty sheet or **Add to sample** for an existing sample. 4. Set the available filters for the program's record type. 5. Enter a **Sample size** from 1 to 500. 6. Click **Populate**. Rulebase samples eligible records at random and adds them to the current test sheet. Running **Add to sample** again skips records already present in the sheet. If fewer eligible records remain than requested, the confirmation reports the shortfall. ![Populate test dialog with interaction type, transcript search, and sample size](https://mintcdn.com/rulebase/LmDNkLxqrTRcrtJn/images/compliance-populate-test.png?fit=max\&auto=format\&n=LmDNkLxqrTRcrtJn\&q=85\&s=94aefd2b3b21ba4a79cdd49fb1939a72) ## Filter conversation samples Conversation-based tests provide these filters: * **Interaction type**: include any interaction, tickets, or sales calls. * **Transcript search**: describe relevant language or behavior, such as `customer sounds vulnerable`. **Transcript search** matches message content that supports the condition. Keep the description focused on language likely to appear in the interaction. ## Filter other record types Available filters depend on the compliance program's target: | Record type | Available filters | | - | - | | Disputes | The test period and sample size | | Compliance cases | **Status** and **Source** | | Compliance alerts | **Status**, **Source**, and **Alert type** | Leave an optional filter blank to sample across all eligible values for that field. ## Run and monitor the test After the sample is populated, click **Run test** to fill the generated cells. A cell can show **Queued**, **Generating**, **Thinking**, **Pulling data**, or **Failed** while processing. A cell returns to **Queued** when Rulebase retries generation. Select a completed generated cell to review its reasoning. Citation links open the supporting conversation message or referenced policy or framework document when that source is available. If a generated cell remains **Failed**, confirm the source record is still available. After resolving the problem, select the affected range and click **Run test** again. Do not treat a failed generation as a compliance result. Once every cell has filled, the sheet is complete. [Run and analyze a compliance test](/guides/compliance/run-and-analyze-a-test) covers reviewing the answers, marking exceptions, and generating the analysis report. ## Verify the sample Before reviewing exceptions, confirm: * The rows fall within the test period. * The record type matches the compliance program. * The sample size matches the requested count or the number of eligible remaining records. * Repeated population did not add records already in the sheet. Decide whether a control passed, failed, or was not applicable from the citations and the underlying record, not surface metadata alone. # Run and analyze a compliance test Source: https://docs.rulebase.co/guides/compliance/run-and-analyze-a-test Fill a populated test sheet, generate the analysis report for the cycle, read the per-question breakdown, and work the exceptions before you sign the result off. A populated test is a sheet of sampled records with an empty column for each question in the program's test scripts. What remains is to fill those columns and turn the answers into a result you can hand to a second line reviewer or an examiner. Both happen on the test itself: the **Testing** tab holds the working paper, and the **Analysis** tab holds the report generated from it. ## Before you start * The test needs a sample in its sheet. Drawing one is [Populate a compliance test](/guides/compliance/populate-tests). * **Compliance** must be enabled for your organization ([Settings > Features](https://app.rulebase.co/settings/features)). Without it the **Compliance** section, and the test, are not reachable. * Know the population figure you intend to stand behind for the period. Rulebase records it, but it does not work it out for you. ## Fill the sheet **Run test** sits at the top right of the **Testing** tab and fills the empty generated cells for the rows in the sheet. It appears once the sheet has rows, and it is disabled when there is nothing left to fill: either because every generated cell has an answer, or because the range you have selected is already complete. Select a range first when you want to re-run part of the sheet rather than all of it. ![Test header with Chat, Add to sample, and an enabled Run test button](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-run-test-button.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=ae4c33ef3ea46e94889927ed455a6b2e) The cell states, the reasoning behind each answer, and what to do about a **Failed** cell are covered in [Populate a compliance test](/guides/compliance/populate-tests). Work through those before generating a report: the analysis reads whatever is in the sheet at the moment you generate it, and a gap in the sheet becomes an unexplained gap in the report. A finished sheet carries one row per sampled record and an answer in every question column, and **Run test** greys out because there is nothing left to fill. ![Filled compliance test sheet with ticket, agent, customer, and date columns beside Yes and No answers for each test question](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-test-sheet-filled.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=40cc142bf5035da04fbaf723c44fe2ab) ## Record the counts that describe your coverage Open the **...** menu beside the test name, then **View details** to see the **Name**, **Test period**, **Population size**, and **Sample size**. ![Test details dialog with name, test period, population size, and sample size fields](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-test-details-dialog.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=4756f2cba0b390a1539555a07c86f20d) **Population size** and **Sample size** are yours to maintain. Rulebase does not derive them from the sheet, and it will not let the sample exceed the population. They are passed to the AI as context while cells are filled, and they are the numbers a reviewer will read as your coverage claim for the period, so fill them in from your own population query rather than leaving them at whatever the last cycle held. **Sample size** has one practical side effect: it becomes the default sample size in the **Populate test** dialog, which defaults to 20 when the field is empty. Setting it before you draw the sample is the tidier order. ## Generate the analysis 1. Open the **Analysis** tab. On a test with no report yet, it reads **No analysis generated**. 2. Click **Generate analysis**. 3. Leave the tab open. Rulebase shows the steps it is taking while it reads the sheet, and the report appears in place when it finishes. Generation reads the sheet's rows and the distribution of answers in each column, the program's controls and test scripts, the frameworks attached to the program, and the test period. It does not re-examine the underlying tickets, cases, or alerts; the answers in the sheet are its evidence, so the sheet has to be finished first. ## What the report contains ![Analysis report with interactions reviewed, findings identified, a severity bar, key findings, and the start of the test script breakdown](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-analysis-report.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=a5888794886c7d567ddb700512253fda) * **The counters** — how many records were reviewed, labelled for the program's record type, and how many findings the report raised. * **Findings identified** — the severity split across **High**, **Medium**, and **Low**. * **Key findings** — the findings in severity order, written as statements about what was observed. * **Test script breakdown** — one block per control, with a one-line summary and, for each question, the split of **Yes's**, **No's**, and **N/A**. * **Recommendations** — a short list of remedial actions tied to the findings. ![Test script breakdown grouped by control with per-question answer splits, followed by the recommendations list](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-analysis-breakdown.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=5ab3ed29144ca5c12607eb8c9c36a9df) The counts come from the cells; the prose is model-written. Read the findings and recommendations as a well-informed draft of your write-up rather than as the write-up itself, and edit your own summary from them. > [!NOTE] > > A report covers one test period. Rulebase does not compare periods for you, so quarter-on-quarter > trending is something you assemble from the reports or from the sheet exports. ## Work the exceptions Every **No** and every **N/A** in the breakdown is a question to answer, not yet a finding. 1. In **Test script breakdown**, click the question you want to examine. 2. The **Question breakdown** panel opens with the answer split, the total row count, and the rows grouped by answer. **Show 5 more** expands a group beyond its first five rows. 3. Note the records in the **No** and **N/A** groups, then open those rows back in the sheet to read the cell's reasoning and follow its citations to the source record. ![Question breakdown panel listing the tickets behind the Yes and No answers for one test question](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-question-breakdown.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=ce1c12152f1e339edd337c4d72051b18) An **N/A** usually means the control could not apply to that record: no fee was discussed, so nothing could be disclosed. That is a legitimate answer, but it also reduces your tested count for that question, so record it as such rather than folding it into the passes. ## Before you sign the cycle off * Every generated cell has an answer. Nothing is still **Failed**. * Each **No** and **N/A** has been checked against the source record, not just the cell. * **Population size** and **Sample size** match the methodology you documented on the program. * The report was generated after the last change to the sheet. If you corrected a cell afterwards, regenerate. * The findings you are reporting are ones you can evidence from specific records. ## Keep the working papers **Download PDF** exports the report as it appears. **Regenerate** produces a new version (useful after you have fixed cells), and earlier completed reports stay available under **Version history**, so a superseded conclusion is still retrievable. **Delete report** in the **...** menu next to the report's author removes a version outright. For the sheet itself, use the download icon in the toolbar above the grid, **Export sheet (CSV)**. That file is the row-level record behind the report's counts. ## Related * [Set up a compliance testing program](/guides/compliance/set-up-a-testing-program) * [Populate a compliance test](/guides/compliance/populate-tests) * [Unit21](/guides/integrations/unit21) * [Rulebase glossary](/guides/concepts/glossary) # Set up a compliance testing program Source: https://docs.rulebase.co/guides/compliance/set-up-a-testing-program Create a testing program, agree the controls and test scripts it runs, define how its population is sampled, and add the periodic tests that hold each cycle's work. A testing program is the standing definition of one piece of monitoring: which records it examines and which control questions it asks of each. Each cycle of that program is a **test**, holding a sheet of sampled records with one column per question. Finish the definition before anyone draws a sample, so cycles stay comparable across the year. ![Compliance > Testing showing a monthly UDAAP testing program in the programs list](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-testing-programs-list.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=c1a7cc93668f7a48a7687abae6999c3e) > [!NOTE] > > Programs live under **Compliance > Testing**. Parts of the flow say *test* where they mean > *program*: the create page is headed **Create a test** and the setup wizard reads **Set up > test**. You are building a program in both. Individual tests come after the program is > confirmed. ## Before you start * **Compliance** must be enabled for your organization ([Settings > Features](https://app.rulebase.co/settings/features)). If **Compliance** is missing from the sidebar, ask an administrator to switch it on. * The records you intend to test need to be reaching Rulebase already: tickets and sales calls from your help desk, or cases and alerts from a monitoring tool such as [Unit21](/guides/integrations/unit21). * Have your control language to hand. Rulebase drafts controls and questions, but the wording that ends up in the program is what your testing will be judged against, so edit it before you accept it. ## How programs, tests, and samples relate * **Program**: the reusable definition. Record type, controls, test scripts, population and sampling rules, frequency. * **Test**: one cycle of that program, bounded by a start and end date, holding a sheet and its own analysis. * **Sample**: the rows inside a test's sheet, the specific records drawn for that period. Drawing the sample is covered in [Populate a compliance test](/guides/compliance/populate-tests), and reading the result in [Run and analyze a compliance test](/guides/compliance/run-and-analyze-a-test). ## Create the program 1. Go to **Compliance > Testing**. 2. Click **New**. 3. Either describe the program in **Describe what you would like to test** and submit it (optionally attaching a policy or procedure with **Upload**), or click a framework card under **or select from a program**. Each card shows how many control modules it brings. 4. Rulebase creates the program and opens the setup wizard on the **Details** step. ![Create a testing program page with a prompt box and framework cards showing their control module counts](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-create-program.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=2be430bfa2a9f567ef0001d84f1a1795) Both routes land in the same wizard with the same three steps: **Details**, **Population**, and **Risk factors**. The framework route pre-selects a control library instead of inferring one from your description. ## Confirm the details The **Details** step fixes the program's scope: * **Test name**: the program name, and the name that appears on every test and report it produces. * **Description**: what the program covers. Rulebase uses it as context when it fills a test. * **Applies to**: the record type the program examines: **Interactions**, **Disputes**, **Cases**, or **Alerts**. It decides which filters the **Populate test** dialog offers and which columns a new test sheet is built with, so settle it before you sample: sheets that already exist keep the columns they were created with. * **Frequency**: **Weekly**, **Monthly**, or **Quarterly**. It sets the default period for each new test. Underneath, the control library shows every control in the framework with a checkbox. Checked controls are in the program, and each holds its test scripts on the right: the questions Rulebase answers for each sampled record. Add a question in the **Does the agent...?** field and click **Add new**; write them as questions with a defensible yes, because the analysis counts answers per question. ![Program setup Details step with test name, applies to, frequency, the UDAAP control list, and test scripts per control](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-program-setup-details.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=49b86173575175bea6515978c36ad05a) Uncheck a control to take it out of this program. The pencil beside the framework name switches the list into editing mode, where you can rename a control or delete it with the bin icon. A deletion there removes the control from your whole organization, across every program, without asking twice. Unchecking is what you want in almost every case. When the details are right, click **Determine population**. The button stays disabled while a draft question is pending, so finish or discard any half-written one first. ## Define how the population is sampled This step records what the program covers and how much you intend to test; the wording becomes part of your audit record. 1. Click **Add test population**. 2. Give the population a **Population name**, such as `Retail support tickets`. 3. Under **Determine test population**, choose **Natural language** and describe the population, or choose **Determine via attributes** and set a condition on a field such as **Interaction type** or **Channel**. The attributes option appears for the record types that support it, and the panel takes one condition. 4. Save the population, then set **Sample size** as a percentage of it. Check **Test the full population.** instead when the program is a census of every record. ![Population step of program setup with a named test population and a 10% sample size](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-program-setup-population.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=c6553fd1debe5ae4e09328bc24c2164f) Where coverage should scale with volume, open **Advanced controls** and turn on **Define tiered sampling rules and sample size**. Each tier takes a **Population size** band and its own **Sample size**, as a percentage (**%**) or a fixed count (**items**); **Add tier** adds the next band. Rulebase passes these rules to the AI as program context when it fills a test sheet, and they stand as the program's documented methodology. They do not prefill the **Populate test** dialog: you still choose the filters and sample size when drawing each sample. Click **Evaluate risk factors** to continue. At least one population is required. ## Review the risk factors Rulebase reads what it already knows about your interactions and drafts the failure patterns worth watching: a fee described without its condition, dissatisfaction acknowledged but never logged. Each factor has a title and a short explanation. Use **Add instructions** to add your own guidance to one, **Remove** to drop it, and **Add risk factor** to write one Rulebase missed. ![Risk factors step showing drafted risk factors with Add instructions and Remove actions](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-program-risk-factors.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=a3f8f1ee0e639bd53e8f4f8acc4522cd) Treat this step as a review of your own thinking. Risk factors are recorded on the program; they are not fed into cell filling or the analysis report, so anything you actually want tested has to exist as a test script on the **Details** step. **Confirm risk factors** finishes setup and opens the program. ## Add the tests A confirmed program has no tests until you create them; a test holds a period's sample. 1. On the program, click **New test**. 2. Set the **Period**. It defaults to the current period for the program's frequency. 3. Click **Create test**. Rulebase names the test after its period (a full calendar month becomes `June 2026 test`) and opens it on an empty sheet. ![New test dialog with a June 2026 period and the Create test button](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-new-test-dialog.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=174ab2c0759b12b7bad2af00fc9a34a4) ![Program page listing the July and June tests with their periods and authors](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-program-tests.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=0925076406bf287bce42007141cdddc6) > [!TIP] > > Create each test when you are ready to sample it. The period you choose is the window the sample is > drawn from, so a test created for the wrong month quietly produces the wrong population. Next, [Populate a compliance test](/guides/compliance/populate-tests) draws the sample into this sheet. ## Change a program later The **...** menu beside the program name offers **Edit**, which reopens the setup wizard with your confirmations intact, and **Delete**, which removes the program along with its tests. Editing a program does not change tests that have already run: their sheets keep their columns, so a mid-year change to the test scripts shows up in the next test and leaves the last one as it was. Each test row has its own **...** menu with **Rename** and **Delete**. Renaming is cosmetic. The period controls what a test covers, regardless of the name. ## Related * [Populate a compliance test](/guides/compliance/populate-tests) * [Run and analyze a compliance test](/guides/compliance/run-and-analyze-a-test) * [Unit21](/guides/integrations/unit21) * [Rulebase glossary](/guides/concepts/glossary) # Work a complaint case Source: https://docs.rulebase.co/guides/compliance/work-a-complaint-case Find a detected complaint, read the tickets and timeline behind it, record your investigation, and close it, including what closing a linked ticket does to the case. Rulebase opens a complaint case on its own. Every ticket it evaluates is checked for a complaint, and when it finds one it writes a case from that ticket and classifies it against your complaint categories. A person then confirms the classification, investigates, and closes the case with a documented trail behind it. ## Before you start * At least one complaint category must exist. Detection matches tickets against your categories, so an empty taxonomy detects nothing; see [Configure complaint taxonomy](/guides/compliance/complaint-taxonomy). * Anyone who can open Quality assurance can change a complaint's status and add updates. There is no separate complaints permission. > [!NOTE] > > Open the complaints list from **Quality assurance → Complaints**, or at > [app.rulebase.co/complaints](https://app.rulebase.co/complaints). > **Settings > Reviews** holds the complaint types used for detection, while the **Complaints** > view in the ticket lists shows tickets rather than cases. ## Where a case comes from The complaint check runs on tickets (not on sales calls) as part of the evaluation Rulebase performs when a ticket is assessed. It compares the conversation against the complaint categories that apply to that ticket's workspace, using each category's name, description, and keywords. If nothing matches, no case is created, however unhappy the customer sounded. When something does match, Rulebase writes the case's title and summary from the conversation, tags it with every category that matched, and sets the priority from the highest risk level among those categories. A case tagged both `Fees and charges` at high risk and `Service delays` at medium risk is a high-priority case. Repeat contact is handled for you. When a customer writes again, Rulebase checks that customer's unresolved cases first, and if the new ticket is about the same issue it attaches it to the existing case rather than opening a second one. Once a case is resolved it is closed for good: a later contact opens a fresh case instead of reopening it. > [!NOTE] > > Linking is automatic and there is no control for attaching or detaching a ticket by hand. A case's > tickets are exactly the ones detection associated with it. ## Find the case The list opens on the last 7 days, grouped by priority, with a count beside each group. ![Complaints list grouped by Urgent, High, and Medium priority, showing customer, title, category tags, and date](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-complaints-list.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=bfd29794709ccedf17f78edc5efab6bf) Search matches a case's title and summary. **Filter** narrows the list further: | Filter | What it does | | - | - | | **Date** | The window cases were created in. Defaults to the last 7 days. | | **Status** | One of the five workflow statuses, or **All statuses**. | | **Priority** | **High**, **Medium**, or **Low**, or **All priorities**. | | **Customer** | Cases for one customer. | | **Workspace** | Cases from one connected help desk workspace. | | **Category** | Cases carrying one or more of your complaint categories. | For a regulated complaints register, filter by a date range plus a status other than **Resolved** to get your open population for the period. ## Read the case Opening a case shows the summary and the timeline on the left, and the case's properties and tickets on the right. ![Complaint case with its summary, activity timeline, and the linked ticket conversation open in the sidebar](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-complaint-detail.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=402953f97a80e6e255b2e397e05cf06f) The timeline is the case history in order: * **New complaint tagged** — detection created the case, with the originating ticket beneath it. * **Rulebase set Risk level as ...** — the priority detection assigned to the case. * **\ complained** — a later contact from the same customer that attached to this case. * Comments — updates you and your colleagues added. Complaints detected from tickets carry no reference number of their own, so the line above the title reads `#Complaint`. Identify a case by its customer and title, or by its URL. The **Documents** tab holds files you attach. ## Record your investigation Everything you do outside Rulebase (the call to the payments team, the approval, the letter) belongs on the case; the timeline is what a reviewer reads months later. 1. In **Activity**, type into **Add update** and click **Add**. It is recorded as a comment against your name with today's date. 2. To attach evidence, open **Documents** and click **Upload file**. PDFs, images, text, CSV, Word, and Excel files are accepted, up to 25 MB each. **Delete** removes one. 3. Move the status as the work progresses, from the **Status** row on the right. The five statuses map to complaint-handling stages: * **Open** — detected and not yet picked up. * **Under investigation** — someone is working it. * **On hold** — waiting on the customer or another team. * **Provisional credit issued** — money has been returned pending the outcome. * **Resolved** — the case is closed. ## Check the tickets behind it **Complaint details** shows the customer, the complaint date, the status, the category tags, and the assignee. **Tickets** lists every conversation linked to the case; click one to read its thread inline, and use the back arrow to return to the list. ![Complaint details panel with customer, date, status, tags, and assignee, above the linked tickets list](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/compliance-complaint-sidebar.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=0495fd716d8fb88416e45e7e028dd4d5) Read the tickets before you accept the classification. Detection is good at spotting dissatisfaction and less good at knowing which of your categories a firm would file it under, and the tags drive the priority, so a miscategorised case is also mis-prioritised. **Assignee** shows accounts or roles assigned to the case through the API. It is not editable from this page. ## Close the case Set **Status** to **Resolved** when the complaint is answered. Resolving the review on a linked ticket also resolves the case: marking a ticket **Done** sets every unresolved complaint linked to that ticket to **Resolved**, so a colleague finishing their QA work can close your case. After resolution, further contact from the same customer opens a new case rather than reopening this one. If a case was closed too early, you can move it back to any other status from the same menu. High cases can raise an email or Slack alert the moment they are detected, turn this on if you work to a response deadline. See [Configure notifications](/guides/settings/configure-notifications). ## Related * [Configure complaint taxonomy](/guides/compliance/complaint-taxonomy) * [Configure notifications](/guides/settings/configure-notifications) * [Set up a compliance testing program](/guides/compliance/set-up-a-testing-program) * [Rulebase glossary](/guides/concepts/glossary) # Rulebase glossary Source: https://docs.rulebase.co/guides/concepts/glossary Definitions of core Rulebase terms, grouped by product area, with a link to the guide that covers each one. This page defines Rulebase's core terms, grouped by the part of the product they belong to, with a link to the guide that goes deeper on each. ## Scorecards, sections, criteria, and checks A scorecard is a nested structure: it contains sections, sections contain criteria, and criteria contain checks. Points are set on criteria and roll up into section totals. * **Scorecard** — the whole rubric Rulebase scores against, with a name, a description, and an optional scope. Only a published scorecard evaluates anything; a draft is invisible to AI QA and to reviewers. * **Section** — a themed group of criteria such as `Resolution` or `Communication`. A section's score is the sum of its criteria's points; you do not set it directly. * **Criteria** — the individual item being judged, and the level where points are assigned. Each criterion has score options: **Pass**, **Fail**, and optionally **Partial** or **Auto-fail**. * **Check** — the instruction an evaluator follows inside a criterion. A criterion can hold several checks. Inside a check: * **Condition** — the if/then rule that maps an observation onto one of the criterion's non-passing scores, as in `If ` … `mark as fail`. Every check needs at least one before the scorecard can be published. * **Apply to** — the pill on a check that limits which tickets the check runs on at all, by channel, by SLA breach, by call direction, and so on. It reads **All tickets** until you narrow it. * **Additional instructions** — extra guidance applied while the check is being scored, for exceptions your reviewers hold in their heads. If you catch yourself writing "ignore this check when…", you want **Apply to** instead. A scorecard's top-level points must add up to exactly 100, so scores are comparable across tickets. [Create a scorecard](/guides/quality-assurance/create-a-scorecard) walks through the structure and the point budget, and [Add scorecard additional instructions](/guides/quality-assurance/add-scorecard-additional-instructions) covers additional instructions. ## Tickets, conversations, work items, and interactions The records Rulebase scores go by different names depending on where you see them. * **Ticket** — a customer support conversation from your help desk. This is the term the QA screens use. * **Conversation** — the same object as a ticket, as named by the API and the upload endpoints. [Uploading conversations](/guides/uploads/conversations) and API responses say "conversation". * **Work item** — back-office work an agent did in an internal tool with no customer on the line, such as a fraud review or a KYC check. Work items are scored with the same scorecards as tickets, and a scorecard's **Applies to** scope decides which of the two it covers. See [Uploading work items](/guides/uploads/work-items). * **Interaction** — the umbrella term for customer touchpoints, and the name of the list that holds them. **Interaction type** separates support tickets from sales calls. * **Ticket scope** — whether the customer replied. A ticket where they did is **External**; a one-sided ticket where they never did is **Internal**, and it is skipped unless a published scorecard opts in to internal tickets. ## The kinds of evaluation An evaluation is one agent's score, on one ticket, against one scorecard. A ticket handled by three people can carry three evaluations, so a ticket's score and an agent's score can be different numbers in reports. * **AI QA evaluation** — the score Rulebase produces automatically once a ticket clears the eligibility gates. See [How AI QA works](/guides/concepts/how-ai-qa-works). * **Manual evaluation** — a score a person enters by hand against the same scorecard, either instead of or alongside the AI result. Anyone with permission to perform evaluations can create one, and a manual request also works on tickets automatic QA passed over. * **Evaluation validation** — review of a trainer's manual evaluation by someone who can provide model-training feedback. The trainer's evaluation counts immediately. If the validator publishes corrections, that evaluation replaces the trainer's result. * **Calibration** — assigning several reviewers to evaluate the same tickets so you can compare how they scored and align on standards. Each reviewer produces a real evaluation. Switch it on under [Settings > Features](https://app.rulebase.co/settings/features). * **Criterion feedback** — a thumbs up or thumbs down on one criterion of an AI result, with a note. Feedback can re-score that single criterion; a manual evaluation replaces the whole judgement. [Give feedback on AI QA](/guides/quality-assurance/give-feedback-on-ai-qa) covers it. * **Auto-fail** — a score option a criterion can carry. Selecting it fails the whole evaluation rather than costing that criterion's points. ## Eligibility, coverage rules, and scoping Four mechanisms control which tickets QA covers. Each answers a different question and is set in a different place: | Mechanism | The question it answers | Where you set it | | - | - | - | | **Eligibility** | Is there anything on this ticket worth scoring? | [Settings > Evaluations](https://app.rulebase.co/settings/evaluations) | | **Coverage rules** | Of the tickets worth scoring, which ones do we spend an evaluation on? | [Settings > Evaluations](https://app.rulebase.co/settings/evaluations) | | **Scorecard scoping** | Which rubric applies to this ticket and this agent? | The scorecard's **Add scoping** row | | **Apply to** | Should this one check run on this ticket? | The check card inside a criterion | Eligibility is several settings, grouped under **Eligibility** and **Timing** in evaluation settings: * **Coverage rules** — sampling. A rule keys on something about the ticket (channel, brand, team, or group) and sets a target, either a percentage of matching tickets or a fixed count per day, week, or month. No rules at all means no sampling limit, not zero coverage. * **Eligibility instructions** — plain-language rules Rulebase applies to the conversation itself, to filter out tickets nobody would score: spam, a customer who typed "hello?" and vanished, a bot handoff with no human reply. They gate automatic QA only. * **Eligibility window** — an age limit measured from when the ticket started, so connecting a help desk with years of history does not flood QA. * **Wait for all linked tickets to close** — holds evaluation until every linked ticket in the thread is resolved, for teams that split one issue across several tickets. * **Evaluation delay** — how long after close Rulebase waits before evaluating. The delay is measured from when the ticket closes and decides when the evaluation runs; the eligibility window is measured from when the ticket started and decides whether it runs at all. [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility) is the guide for all of these, and the eligibility tester it describes will tell you which mechanism is holding a specific ticket back. ## AI agents An **AI agent** is an automation you build and run inside Rulebase, under [Home > Agents](https://app.rulebase.co/home/agents). The **Agents** permission group governs them: a permission checklist that says **Agents** means AI agents. [Create a managed agent](/guides/agents/create-a-managed-agent) covers building one. ## Agents, teams, groups, roles, and memberships * **Agent** — a support rep you run QA and performance reports on, listed under [Settings > People > Agents](https://app.rulebase.co/settings/people/employees). Viewing agents and their reports requires the **View agents and performance reports** permission, in the Insights group. * **Agent roster** — the list of agents Rulebase evaluates. Someone missing from it is not evaluated, and someone still on it after they left keeps appearing in team performance views. See [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster). * **Team** — a named group of agents with an optional team lead, maintained by you in Rulebase. Teams drive grouping in QA reports and exports, and an agent belongs to one team at a time. See [Create and manage teams](/guides/access-management/create-and-manage-teams). * **Group** — a grouping that mirrors your help desk's own groups rather than one you build in Rulebase. Tickets and agents arrive already assigned to one, and groups are offered for coverage rules, scorecard scoping, and filtering in the same menus as teams. * **Role** — the bundle of permissions someone carries. System roles ship with Rulebase and cannot be edited; custom roles are yours. [Roles and permissions](/guides/roles-and-permissions) is the dictionary of what each permission grants. * **Membership** — one person's login in your organization, carrying their role and optionally a data scope that narrows what they can see. Membership is separate from the agent roster: an agent with **Not invited** in the roster's **Membership** column is still evaluated normally, they just cannot sign in. * **Queue** — a group of members used for assignment routing, such as the reviewers who share evaluation validation work. Queues are about assignment; teams and groups are about reporting. > [!NOTE] > > Teams and groups appear in the same scoping and filter menus. Teams are created > and renamed in Rulebase. Groups follow your help desk, so a group that is > missing or misnamed is usually fixed there rather than here. ## Related * [How AI QA works](/guides/concepts/how-ai-qa-works) * [Create a scorecard](/guides/quality-assurance/create-a-scorecard) * [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility) * [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster) * [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) * [Roles and permissions](/guides/roles-and-permissions) # How AI QA works Source: https://docs.rulebase.co/guides/concepts/how-ai-qa-works Follow a ticket from close to score: the eligibility gates, the evaluation delay, scorecard matching, scoring, and how human feedback shapes what comes next. Automatic QA looks like one step from outside: a ticket closes, and later it carries a score. Inside, it is a short pipeline with a decision at every stage. **Nothing is being evaluated** is a gate question: eligibility settings and scorecard scope. **The score is wrong** is a rubric question: your criteria and their checks. ```mermaid theme={null} flowchart TD A[Ticket closes in your help desk] --> B{Eligibility gates} B -->|Blocked| S[Evaluation skipped, with a reason] B -->|Passed| C[Evaluation scheduled for close time plus your evaluation delay] C --> D{Eligibility instructions read the conversation} D -->|Nothing worth scoring| S D -->|Worth scoring| E[AI scores each eligible agent against each matching scorecard] E --> F[QA score lands on the ticket and the agent] F --> G[Criterion feedback from your reviewers] G -.->|shapes later evaluations| E ``` ## It starts when the ticket resolves Resolution triggers automatic QA, because an open ticket can still turn around. Rulebase sweeps recently resolved tickets on a schedule, so a few minutes between close and any visible QA state is normal. Two things must be true before a ticket is a candidate: it reached Rulebase through your help desk connection, and an agent sent at least one customer-facing message on it. A ticket where nobody replied has no agent work to judge. ## The gates run in a fixed order Rulebase applies a series of gates and stops at the first that says no. The order matters: the reason shown on a skipped ticket is the *first* thing that blocked it, and others may also apply: 1. **Evaluation credits.** Your organization has evaluations left to spend. 2. **Eligibility window.** The ticket started inside the age limit you set. 3. **No evaluation already exists or is in flight.** A completed AI evaluation, or another request already queued, stops a second one. 4. **Linked tickets.** If you have asked Rulebase to wait, every linked ticket in the thread has to be resolved too. 5. **Ticket scope.** The customer replied, or a published scorecard has opted in to internal tickets. 6. **Scorecard match.** At least one published scorecard's scope covers this ticket. 7. **Eligible agents.** At least one agent on the ticket is on the QA roster, is a human agent (bots do not qualify), wrote enough to judge, and falls inside a matching scorecard's people scope. 8. **Coverage rules.** Your sampling rules selected this ticket, and selected these agents. Scorecard match and eligible agents catch new accounts for opposite reasons: a scorecard scoped too narrowly covers no tickets; a roster missing people leaves a well-scoped scorecard with nobody to grade. A blocked ticket records its reason on the ticket. [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) is the gate-by-gate lookup for a specific ticket. ## The delay controls timing only Once a ticket clears the gates, Rulebase schedules the evaluation for its close time plus your **Evaluation delay**. The gates it passed are not re-run while it waits. The delay exists because tickets reopen. If a customer replies "actually, one more thing" twenty minutes after close, an evaluation that already ran judged half a conversation. So Rulebase rechecks resolution when the delay expires, and a ticket that reopened in the gap is set aside. Aim for long enough that reopens settle, short enough that reviewers get their scores within the day. ## Eligibility instructions read the conversation The gates so far read metadata; the last one reads the conversation. When the delay expires, Rulebase prepares the conversation, transcribing call recordings and reading image attachments as scorable evidence. Then it applies your **Eligibility instructions** to the transcript itself. This filter keeps QA off tickets where nobody would score anything. The default asks for two things: the customer expressed a real request, and a human agent had a two-way exchange about it. It catches spam, a "hello?" that went nowhere, and a bot handoff with no human reply. > [!NOTE] > > This gate and your coverage rules apply to **automatic** QA only. A manual > request bypasses both, which is how you get a score on a specific ticket while > you are still tuning the settings that filtered it out. ## The AI scores against the matching scorecard Rulebase produces one evaluation per eligible agent per matching scorecard, so a ticket handled by three people can carry three evaluations, each with its own score. Within an evaluation, the AI works bottom-up through the structure in [Create a scorecard](/guides/quality-assurance/create-a-scorecard). It reads each check against the conversation, applies the check's conditions to score the observation a pass, partial, or fail, and takes any **Additional instructions** on the check into account. Criteria carry the points, sections roll up from their criteria, and the total lands against the scorecard's 100. An auto-fail is the exception: selecting one fails the evaluation outright. Every result includes its reasoning, so you can see which check produced each score. ## The score lands on the ticket and on the agent When the evaluation completes, the score attaches in two places at once: the ticket carries its QA result with the per-criterion breakdown, and the agent's performance views and Insights dashboards pick up the same numbers. Coaching draws on them too: attaching a ticket as evidence in a session brings its QA score along. Because the same rubric applies across a sampled slice of your volume, trends and team differences are comparable. ## Feedback shapes the next evaluation When a reviewer thumbs down a criterion and explains what the AI got wrong, Rulebase turns that note into a written correction in your organization's feedback library, scoped to the scorecard and criterion it came from. Later evaluations retrieve it when they hit a comparable situation. Two tools apply to a score you disagree with: * **Criterion feedback** corrects one result and teaches by example. Reach for it when the AI missed context you can describe in a sentence or two. See [Give feedback on AI QA](/guides/quality-assurance/give-feedback-on-ai-qa). * **Additional instructions** change the rule everywhere, for every future evaluation of that check. Reach for them when the same exception keeps coming up. See [Add scorecard additional instructions](/guides/quality-assurance/add-scorecard-additional-instructions). Neither reaches back across your history. Criterion feedback re-scores only the one criterion you flagged on the evaluation you flagged it on; additional instructions apply from the next evaluation onward. Every evaluation that already ran keeps its score, so to judge old tickets under new guidance, re-evaluate them. ## Which setting controls which stage | Stage | What decides it | Where to change it | | - | - | - | | Whether the ticket is a candidate | Ticket resolved, agent replied, ticket reached Rulebase | Your help desk connection under [Settings > Connections](https://app.rulebase.co/settings/connections) | | How far back Rulebase looks | **Eligibility window** | [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility) | | Whether one-sided tickets count | **Ticket scope** on a scorecard | [Create a scorecard](/guides/quality-assurance/create-a-scorecard) | | Which rubric applies | Scorecard **Add scoping**, and publishing the scorecard | [Create a scorecard](/guides/quality-assurance/create-a-scorecard) | | Who gets evaluated | The QA agent roster, plus a scorecard's people scope | [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster) | | How much volume is covered | **Coverage rules** | [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility) | | When evaluation starts | **Evaluation delay**, and **Wait for all linked tickets to close** | [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility) | | Whether the conversation is worth scoring | **Eligibility instructions** | [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility) | | What gets judged, and how it scores | Sections, criteria, checks, and conditions | [Create a scorecard](/guides/quality-assurance/create-a-scorecard) | | Exceptions to a single check | **Additional instructions** | [Add scorecard additional instructions](/guides/quality-assurance/add-scorecard-additional-instructions) | | How a specific wrong result is corrected | Criterion feedback | [Give feedback on AI QA](/guides/quality-assurance/give-feedback-on-ai-qa) | ## Reading the pipeline backwards Start from a symptom: * **Nothing at all is being evaluated.** Start at the scorecard. An unpublished scorecard, or one scoped past the tickets you care about, blocks everything downstream. It is the top cause on new accounts. * **Some tickets are scored and similar ones are not.** That is a sampling or roster issue, and the rubric is fine. Coverage rules skip tickets on purpose, and an agent who never made the roster produces the same silence. * **Scores exist but they are wrong.** Nothing in eligibility will help. The answer is in the criterion that scored wrong, its checks and conditions, and whether your exception is written down as an additional instruction. For a specific ticket, don't reason it out from here. The eligibility tester shows every gate at once for a real ticket, and [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) walks through reading its result. ## Related * [Rulebase glossary](/guides/concepts/glossary) * [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility) * [Create a scorecard](/guides/quality-assurance/create-a-scorecard) * [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) * [Give feedback on AI QA](/guides/quality-assurance/give-feedback-on-ai-qa) # Getting started Source: https://docs.rulebase.co/guides/getting-started What Rulebase does, how your conversations reach it, what happens once they are scored, where the results appear, and which guide to read first. Rulebase is an AI-assisted quality assurance tool for customer support teams. Every ticket your team closes can be scored against the same rubric a QA lead would apply by hand, so coverage no longer depends on how many hours someone had that week to read tickets. The judgement still belongs to people. Rulebase produces a score and the reasoning behind every part of it; your reviewers correct what it got wrong, and those corrections shape the evaluations that come after. ```mermaid theme={null} flowchart LR A[Help desk or phone system] --> D[Rulebase] B[File uploads] --> D C[REST API] --> D D --> E[QA review on the ticket] D --> F[Performance and Insights reporting] D --> G[Coaching] ``` ## How your conversations reach Rulebase Conversations reach Rulebase three ways; most teams use the first and never touch the others. * **A native connection to your help desk or phone system.** [Zendesk](/guides/integrations/zendesk), [Intercom](/guides/integrations/intercom), and [Freshdesk](/guides/integrations/freshdesk) bring in tickets and chats, and [Aircall](/guides/integrations/aircall) attaches call recordings to them. You set one up under [Settings > Connections](https://app.rulebase.co/settings/connections) once, and Rulebase keeps syncing from then on. * **File uploads.** Recorded calls can be sent to Rulebase as audio files or ZIP archives, which is the path for a phone system with no native connection. See [Uploading conversations](/guides/uploads/conversations). * **The REST API.** Push conversations from a help desk we don't connect to natively, or file back-office work from your own admin tools as [work items](/guides/uploads/work-items). Start at [Set up API access](/guides/api-access) for keys and the base URL for your region. Which one you use shapes the rest of your setup. A native connection brings your agents and their help desk groups in alongside the tickets, so the lists you later scope scorecards and coverage rules by are already filled in. Records you upload or push carry whatever agent identity you send with them, which puts more weight on getting that right at the source. Either way, someone who is not on the QA roster is not evaluated, so check that list early. ## What happens to a ticket once it is in **Eligibility decides whether the ticket is considered at all.** Rulebase does not evaluate everything that closes. Coverage rules sample a share of your volume, an eligibility window keeps a years-deep help desk backlog from flooding QA on day one, and plain-language eligibility instructions filter out tickets where nobody would score anything: spam, a customer who typed "hello?" and vanished. These live under [Settings > Evaluations](https://app.rulebase.co/settings/evaluations). **A scorecard decides how it is judged.** A scorecard is the rubric: sections like resolution and tone, the criteria under each one, and the specific checks an evaluator has to make before awarding points. Its scope decides which tickets and which people it covers. Only a published scorecard evaluates anything, and an account with no published scorecard produces no scores at all. An empty QA list on a new account usually means no published scorecard. **The AI scores, and humans correct.** Rulebase produces one evaluation per eligible agent per matching scorecard, so a ticket three people touched can carry three separate scores. Each result comes with the reasoning for every criterion. When a reviewer disagrees, they can thumbs down the single criterion that went wrong and explain why, or replace the whole judgement with a manual evaluation. [How AI QA works](/guides/concepts/how-ai-qa-works) walks the same path in order, gate by gate, and maps each stage to the setting that controls it. The [glossary](/guides/concepts/glossary) defines the terms, including the two meanings of "agent". ## Where the results show up The same evaluation surfaces in three places: * **On the ticket**, under its **Review** tab, with the per-criterion breakdown and the reasoning behind each score. This is where a reviewer works, and where they leave feedback or write a manual evaluation. The **Interactions** list has QA and evaluation views for working through them in bulk. * **In Insights**, where the same numbers roll up into agent and team performance pages, the built-in Analytics reports, and any dashboards you build yourself. This is the view for spotting that one team's resolution scores have been sliding for a month. [Insights overview](/guides/insights/overview) explains which of those surfaces answers which question. * **In coaching**, where a session pulls in specific tickets and their scores as evidence for a 1:1. Coaching is switched on per organization under [Settings > Features](https://app.rulebase.co/settings/features), so if it is missing from your sidebar that is why. Scores also leave the product. The Evaluations list exports to an Excel workbook with criterion scores and written feedback in separate columns, so teams can pull QA data into whatever they already report from. ## Where to go next Pick the path that matches what you are here to do. ### You are setting Rulebase up for the first time Work in this order; each step depends on the one before it. 1. Connect your source under [Settings > Connections](https://app.rulebase.co/settings/connections), or read [Set up API access](/guides/api-access) if you are pushing data in. 2. Check the people Rulebase will evaluate with [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster). 3. Build your rubric with [Create a scorecard](/guides/quality-assurance/create-a-scorecard), and publish it. 4. Decide how much volume to cover with [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility). 5. Bring in your reviewers and coaches with [Invite members](/guides/access-management/invite-members) and [Roles and permissions](/guides/roles-and-permissions). Before you call it done, prove the setup on real tickets. The eligibility tester in [Settings > Evaluations](https://app.rulebase.co/settings/evaluations) runs every gate against a ticket you pick and tells you which one blocked it, so try one ticket you are certain should be scored and one you are certain should not. ### You have a score in front of you and want to understand it * [Rulebase glossary](/guides/concepts/glossary) — definitions of Rulebase terms, including the two kinds of agent * [How AI QA works](/guides/concepts/how-ai-qa-works) — the full path from ticket close to score * [Give feedback on AI QA](/guides/quality-assurance/give-feedback-on-ai-qa) — correcting a criterion the AI got wrong * [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) — for the score that never appeared ### You are wiring Rulebase into your own systems * [Set up API access](/guides/api-access) — keys, regional base URLs, and your first request * [Uploading conversations](/guides/uploads/conversations) — recorded calls * [Uploading work items](/guides/uploads/work-items) — back-office work with no customer on the line * [MCP](/guides/mcp) — connecting an AI client to your Rulebase data # Assign roles from your identity provider Source: https://docs.rulebase.co/guides/identity-provider-role-assignment Map identity provider groups to Rulebase roles in the Admin Portal, using Directory Sync or SSO depending on how your organization manages access. Rulebase can assign member roles from your identity provider groups. This lets your IT team manage Rulebase access in the same place they manage employee groups, instead of updating each member manually in Rulebase. Use this guide if your organization uses Single Sign-On (SSO), Directory Sync, or both. ## Before you start Make sure you have: * Administrator access in Rulebase * Administrator access to your identity provider * The identity provider groups you want to map to Rulebase roles * A list of the Rulebase roles each group should receive You can review available roles in [Roles and Permissions](/guides/roles-and-permissions). ## Open the Admin Portal The Admin Portal is available from your Rulebase settings page. 1. Open your Rulebase members settings: * [US region: app.rulebase.co/settings/members](https://app.rulebase.co/settings/members) * [EU region: eu.app.rulebase.co/settings/members](https://eu.app.rulebase.co/settings/members) 2. Click **Admin portal**. 3. Choose the SSO or Directory Sync setup flow for your organization. If you do not see the Admin Portal option, contact your Rulebase administrator or reach out to Rulebase. ## Choose a role assignment method If your organization uses Directory Sync, use Directory Sync group role assignment. Directory Sync keeps group membership changes up to date without waiting for users to sign in again. If your organization only uses SSO, use SSO group role assignment. SSO role changes are applied the next time a user signs in with SSO. Avoid configuring both methods for the same role workflow unless your Rulebase team has confirmed the intended setup. ## Assign roles with Directory Sync Directory Sync role assignment is available for SCIM-based directories and Google Workspace directories. 1. In your identity provider, create or confirm the groups you want to use for Rulebase access. 2. Push or sync those groups to Rulebase through your directory setup. 3. In the Rulebase Admin Portal, continue the Directory Sync setup. 4. When prompted, map each identity provider group to the appropriate Rulebase role. 5. Save the setup. After you finish, reach out to Rulebase so we can verify the mappings and complete any Rulebase-side configuration. ## Assign roles with SSO Use SSO role assignment when your organization does not use Directory Sync for Rulebase. 1. In your identity provider, create or confirm the groups you want to use for Rulebase access. 2. Configure your SSO application to include group membership data in the SAML or OIDC response. 3. In the Rulebase Admin Portal, continue the SSO setup. 4. When prompted, map each identity provider group to the appropriate Rulebase role. 5. Save the setup. 6. Ask affected users to sign out and sign back in with SSO. After you finish, reach out to Rulebase so we can verify the mappings and complete any Rulebase-side configuration. # Manage charts on a custom dashboard Source: https://docs.rulebase.co/guides/insights/manage-dashboard-charts Duplicate, move, copy, filter, and export data from custom dashboard charts. Use a chart's menu to duplicate it, move it to another custom dashboard, or show it on more than one dashboard. You can also add repeated chart-level filters and export conversation drilldowns. This guide covers the charts. For how boards sit alongside reports, Radar, and the performance pages, see [Insights overview](/guides/insights/overview). ## Duplicate a chart for independent edits 1. Open the custom dashboard. 2. Open the chart's three-dot menu. 3. Click **Duplicate**. Rulebase adds a new chart to the current dashboard with `(copy)` in its name. The duplicate has its own chart configuration, so you can edit it without changing the original. ## Move a chart to another dashboard 1. Open the chart's three-dot menu. 2. Under **Dashboard**, open **Move to**. 3. Select the destination dashboard. The chart is removed from the current dashboard and added to the destination. ## Copy a chart to another dashboard 1. Open the chart's three-dot menu. 2. Under **Dashboard**, open **Copy to**. 3. Select the destination dashboard. The chart remains on the current dashboard and also appears on the destination. Both dashboard cards reference the same chart configuration. If each dashboard needs an independently editable version, use **Duplicate** first, then move the duplicate. ![Custom dashboard chart menu with Duplicate, Move to, and Copy to actions](https://mintcdn.com/rulebase/LmDNkLxqrTRcrtJn/images/dashboard-chart-actions.png?fit=max\&auto=format\&n=LmDNkLxqrTRcrtJn\&q=85\&s=59ea8d47a6b528a2dd206ae693923e74) ## Add repeated chart filters 1. Open the chart's three-dot menu. 2. Click **Edit**. 3. Under **Filters**, click **Add filter**. 4. Choose **Brand**, **Partner**, or **Agent joined date** when available for the chart's data source. 5. Set the operator and value. 6. Repeat **Add filter** for each additional condition. 7. Save the chart. Repeated filters let you express multiple conditions for the same field. The available filter types depend on the chart's metric and data source. ## Export a conversation drilldown 1. Select a chart segment or value that opens a conversation drilldown. 2. Apply any drilldown filters or search terms. 3. Use the display control to choose the visible columns. 4. Click **Export as CSV**. The CSV includes every conversation matching the chart and the current drilldown filters, not only the rows currently loaded on screen. It uses the visible columns and the workspace timezone. **Export as CSV** produces one file, once. When the same extract is needed every week, or QA data has to land in a warehouse, set it up under [Schedule recurring exports](/guides/insights/schedule-exports) instead. # Insights overview Source: https://docs.rulebase.co/guides/insights/overview How boards, reports, and performance pages divide up the work of answering questions about QA results. Insights holds the QA program's numbers, split across three surfaces. Boards chart a metric over time. Reports write an answer out in prose and send it on a schedule. Performance pages break the same numbers down by team and by agent. All of them read the same [evaluations](/guides/concepts/glossary#the-kinds-of-evaluation), so when two screens disagree it is the filter or the grouping that differs, not the data. **Insights** in the sidebar opens on **Boards**. ![The Insights Boards page with chart templates, the Radar and Performance cards, and the All boards section](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/insights-boards-home.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=473afced3fd85b58de43b07de06734fc) ## Which surface answers your question | What you are asking | Open | What you get | | - | - | - | | How has QA score moved this month, and does one channel lag? | **Boards** | Charts you choose, filter, and keep | | Can something summarize the week and email it to the team? | **Reports** | A written report Rulebase generates on a schedule | | How is this team doing, and who needs coaching? | **Performance** | A team list, a page per team, a page per agent | | What does the whole operation look like right now? | **Radar** | One ready-made page of headline numbers and six standing charts | ## Before you start * Everything here is built from evaluated interactions. Until a published scorecard has produced evaluations, these pages have nothing to draw. * Who appears in performance views comes from the agent roster, not from who has a login. See [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster). * Reading Insights needs a role carrying **View organization-wide reports and analytics** and, for the performance pages, **View agents and performance reports**. Every system role except **Agent** has both; agents get their own evaluations and performance instead. * Grouping boards into sections, and managing boards you did not create, needs the **Manage organization settings** permission, in practice an admin. ## Boards are yours to arrange A board is a custom dashboard: a set of charts you place, filter, and re-use. Starting from a template is faster than building one chart at a time, because the template arrives with the charts that usually belong together: a headline number, a trend, and a channel breakdown. 1. Open **Insights**. 2. In the **Templates** row, click the template you want. 3. In **Create board from template?**, click **Create board**. Rulebase creates the board and opens it. From there the date range at the top applies to every chart on the page, and **Share** hands the board to colleagues. ![A board created from the Average QA score template, showing the headline score, trend, and channel charts](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/insights-board-average-qa-score.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=596226a22d263d4f436a1783930fe11a) > [!TIP] > > The **Templates** row shows only the first four templates. **New** > **New board** > opens the full set with a search box and a **Blank board** option, and it is > where **SLA** appears once you have SLA policies configured. Charts are edited on the board itself rather than here: duplicating one, moving it to another board, adding repeated filters, and exporting a drilldown are all covered in [Manage charts on a custom dashboard](/guides/insights/manage-dashboard-charts). ## Reports read like a written answer A report is a written answer rather than a chart. Rulebase runs it on your schedule, fills it from your QA data, and delivers it by email, Slack, or both. Use one for the recurring "how did last week go" question. Click **New report** on the **Reports** tab and describe what you want. The composer opens with `Send me a weekly report of` already typed, and the conversation that follows is where the report's subject, cadence, and recipients get settled. Reports you already have appear under **Your reports** on the same page. ![The Insights Reports tab showing report templates grouped by section](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/insights-reports-templates.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=16919278dda95e28a9833dc0e3cf0726) > [!NOTE] > > The template cards on this page are samples. Clicking one opens a preview of what > that kind of report looks like, filled with example content; it does not create > the report. Use **New report** to get a real one, and mention the template's name > if you want something shaped like it. ## Performance answers "who" The **Performance** card leads to a list of teams with their QA scores, each team opens onto its agents, and each agent has a page carrying their tickets, evaluations, and coaching history. Use it when a board tells you something moved and you need to know who it moved for. [Use team and agent performance](/guides/insights/team-and-agent-performance) covers that route end to end. ## Radar **Radar** is the ready-made view: a row of headline metric cards over six standing charts: QA score overview, scorecard adherence, overall trends, handling time on service issues, complaints by volume, and your top service issues. You cannot add to it or rearrange it; it is the same page every time you open it, so changes stand out. The date range picker at the top right is yours to move. Radar is also your homepage until you choose otherwise, so signing in usually lands you here. Any board's card menu offers **Pin**, to keep it in the **Pinned** section at the top of Insights, and **Make homepage** if a board you built deserves that spot instead. Setting a team as your homepage does the same thing, and either one replaces Radar as your landing page. ## Export the underlying rows When someone needs the underlying tickets in a spreadsheet: * One-off analysis: filter the Evaluations list and export a workbook. See [Export QA evaluation data](/guides/quality-assurance/export-qa-evaluations). * A recurring spreadsheet in an inbox, or QA data landing in your warehouse: [Schedule recurring exports](/guides/insights/schedule-exports). * The conversations behind a single chart segment: open the drilldown and use **Export as CSV**, as described in [Manage charts on a custom dashboard](/guides/insights/manage-dashboard-charts). ## Related * [Use team and agent performance](/guides/insights/team-and-agent-performance) * [Schedule recurring exports](/guides/insights/schedule-exports) * [Manage charts on a custom dashboard](/guides/insights/manage-dashboard-charts) * [Export QA evaluation data](/guides/quality-assurance/export-qa-evaluations) * [Rulebase glossary](/guides/concepts/glossary) # Schedule recurring exports Source: https://docs.rulebase.co/guides/insights/schedule-exports Send QA spreadsheets to an inbox on a schedule, or write QA evaluation analytics to your own Amazon S3 bucket every week. Two recurring exports are configured on the same page: an email export that mails a filtered workbook on a schedule, and a cloud storage export that writes a CSV to a bucket you own. Use them when somebody needs the same spreadsheet every Monday, or when your analysts want QA results in the warehouse next to everything else. Both live under [Settings > Export](https://app.rulebase.co/settings/exports), at the bottom of the settings sidebar under **Administration**. ![The Export settings page with an email export schedule and the cloud storage export section](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/insights-export-settings.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=c07bf821a8a30ba454aa77d23abb8435) ## Which one you want | | Email exports | Cloud storage export | | - | - | - | | **What arrives** | An Excel workbook attached to an email | A CSV plus a manifest in your S3 bucket | | **When** | Daily, or weekly on a day and time you choose | Weekly, plus **Run now** whenever you like | | **What it covers** | Quality assurance, evaluations, interactions, complaints, or coaching sessions, filtered how you like | QA evaluation analytics, whole workspace | | **Who can set it up** | Anyone except agents | Admins only | | **Best for** | A recurring report for people | Feeding a warehouse or BI tool | ## Before you start * Email exports are available to everyone except agents. Cloud storage exports require the **Manage organization settings** permission. * Decide the filters before you create a schedule. A recurring export repeats the filters it was created with, so "resolved tickets, last week, one team" is a choice you make once. * For the S3 route, have a bucket and a dedicated IAM user's keys ready. Rulebase validates them before saving. ## Create a recurring email export The dialog builds the export and previews it at the same time. The rows you are about to schedule appear beside the fields as you change them. 1. Click **New export**. 2. Pick the **Dataset**. 3. Add filters with **Add filter** until the preview on the right shows the rows you want. 4. Open **Columns** and choose the fields to include. Skip this step if you picked **Quality assurance**, which exports a fixed set of columns. 5. Set the **Schedule**: daily, or weekly on a chosen day, at a time in your workspace timezone. 6. Enter one or more **Recipients**. 7. Click **Create**. ![The New recurring export dialog with dataset, columns, schedule, recipients, and a live row preview](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/insights-new-export-dialog.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=1312cefbc1826d6508f18edf4e43120c) The new schedule appears under **Email exports** with its cadence and recipients, and the first delivery lands at the next occurrence rather than immediately. The datasets differ in the shape of a row: * **Quality assurance** — one row per ticket, with its AI and human evaluation results. This is the only dataset without a **Columns** control; it ships a fixed set of ticket-level fields. * **Evaluations** — one row per agent per evaluation, with scorecard scores and review details. Pick this when you are analysing people rather than tickets. * **Interactions** — every customer interaction across channels, including the ones QA never scored. * **Complaints** — one row per complaint, with its status, priority, categories, and related customer context. * **Coaching sessions** — one row per coaching session, with the agent, coach, status, topics, and evidence ticket IDs. Notes are optional and off by default. > [!TIP] > > Filters come from the same menus as the Evaluations, Tickets, Complaints, and > Coaching sessions lists, so the > fastest way to design an export is to get the list showing exactly what you want > first, then rebuild those filters here. ## What recipients receive Each run generates a fresh workbook and attaches it to an email titled `[RULEBASE] export for `. Nobody has to sign in to open it, and recipients do not need to be Rulebase members; any valid address works. Rulebase membership and data scopes do not restrict who can read a delivered workbook, so verify every recipient before saving the schedule. Rulebase builds the file using the permissions and data scope of whoever created the schedule. A schedule created by someone scoped to one team keeps producing a one-team file even after it is handed over, so recreate it under the right account rather than editing the recipients when scope is the problem. ## Manage or stop a schedule Everyone in the workspace sees the schedules that exist, but the row menu only opens for the person who created a schedule and for admins: * **Send now** — runs it immediately, without touching the schedule. Useful for confirming the filters produce what you expected. * **Edit** — reopens the dialog, preview included. * **Delete** — removes the schedule. Files already delivered stay in people's inboxes. ## Send QA analytics to your own S3 bucket The cloud storage export writes machine-readable QA evaluation data into a bucket you control, every week. Rulebase only ever writes, so the IAM user you create needs a single permission: ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Sid": "RulebaseExportWrite", "Effect": "Allow", "Action": ["s3:PutObject"], "Resource": "arn:aws:s3:::YOUR_BUCKET/YOUR_PREFIX/*" } ] } ``` With the bucket and keys ready: 1. Under **Cloud storage export**, click **New data export**. 2. Select **Amazon S3** as the provider. 3. Enter the **Bucket name** and choose its **Region**. 4. Optionally set a **Folder prefix**, such as `rulebase/qa-analytics`. 5. Enter the **Access key ID** and **Secret access key**. 6. Click **Test**. Rulebase writes a test object and reports the key it wrote. 7. Click **Start data export**. The save button stays disabled until a test passes, so a typo in a key fails here rather than three days later. The secret is encrypted and never shown again, so editing an export means re-entering it. ## What lands in the bucket Each run writes three objects under your prefix, partitioned by date: ```text theme={null} evaluations/dt=YYYY-MM-DD/evaluations__to__.csv evaluations/dt=YYYY-MM-DD/evaluations__to__.manifest.json evaluations/dt=YYYY-MM-DD/_SUCCESS ``` The CSV carries one row per evaluation: `evaluation_id`, `conversation_id`, `ticket_id`, `evaluated_at`, `source` (whether AI or a person produced it), `agent_employee_id`, `agent_name`, `agent_email`, `team_lead_name`, `evaluator_email`, `conversation_started_at`, and `conversation_external_created_at`. A team can have several leads, so `team_lead_name` holds every lead of the agent's team, comma-separated. The manifest repeats the column list with a row count and the exported window, so a downstream job can check completeness without parsing the CSV, and `_SUCCESS` marks the partition as finished, the convention most warehouse loaders look for. Run keys include the run id, so a re-run never overwrites a good upload. That also means deleting the export in Rulebase leaves everything already written in place; clean the bucket yourself if you need the data gone. ## Keep an eye on it The row for each cloud storage export shows its status, and its menu carries **Run now**, **Edit**, **Pause**, and **Delete**. A failed run keeps its weekly slot and retries on the next one, with the error recorded against the export, so a transient S3 problem does not need a manual restart. **Pause** is the control to use when the destination is being rebuilt: it stops the schedule without discarding the configuration. ## Related * [Export QA evaluation data](/guides/quality-assurance/export-qa-evaluations) * [Run a coaching session](/guides/coaching/run-a-session) * [Insights overview](/guides/insights/overview) * [Manage charts on a custom dashboard](/guides/insights/manage-dashboard-charts) * [Roles and permissions](/guides/roles-and-permissions) # Use team and agent performance Source: https://docs.rulebase.co/guides/insights/team-and-agent-performance Move from the team list to a team page to one agent's tickets, evaluations, and coaching history. The performance views are the part of Insights that answers "who". They start with every team side by side, narrow to one team and its agents, and end on a single agent's page where their tickets, evaluations, and coaching history sit together. Open **Insights** and click the **Performance** card to get started. ![The Performance teams list with QA score, AI evaluations, and manual evaluations cards above a table of teams](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/insights-teams-performance.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=ccf995cb8302e9074d49f8ca964d37c0) ## Before you start * Who appears here comes from the agent roster, not from who has a login. An agent missing from a team is usually missing from the roster or unassigned: [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster) is the fix. * Manager, team lead, quality analyst, compliance analyst, member, and admin roles all carry **View the agent roster**, which is what these pages need. Agents are the exception: they see their own evaluations and performance rather than the team list. * A membership scoped to one team sees only that team here. * Scores only appear for periods that were actually evaluated. A week with no evaluations reads as **No QA evaluations in this period** rather than zero. ## Start with the team list The list is a fixed four-week view. The cards along the top summarize the whole workspace over that window rather than any one team: * **QA score** — the average across every evaluation in the last four weeks. * **AI evaluations** — how many evaluations AI QA produced. * **Manual evaluations** — how many a person entered by hand. Underneath, each team gets a row with its **Team lead**, how many **Agents** it holds, and its **Avg score** with the change against the previous four weeks. ## Open a team to see its agents 1. Click the team's row. 2. Read the **Summary** at the top before the numbers. Rulebase writes it from that period's results, naming what improved, what slipped, and which agent sits lowest. 3. Use the date control in the header to move to another week. ![A team performance page with its written summary, QA score cards, agents table, and the metrics rail](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/insights-team-detail.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=42c80318a871def75acf73ef905b0a03) The three cards under the summary are the team's headline: **QA score** with its change in points, **QA score breakdown** as the share of evaluations landing in the high, medium, and low bands, and **Evaluations** as the number of evaluations behind all of it. Hover a number to see the counts it was calculated from. The right-hand rail splits the score by cause rather than by person: **Scorecard pass rate** per criterion, **Critical error rate**, **QA score by channel**, and **Lowest-scoring contact reasons**. > [!NOTE] > > The date control moves in whole weeks. Clicking any day in the calendar selects > that Monday-to-Sunday week, and every comparison on the page is against the week > before it. The four-week window on the team list is separate and cannot be > changed. ## Follow one agent into their tickets and evaluations The **Agents** table on a team page is sorted by QA score, so the people who need attention are at the bottom of it. 1. Click an agent's name. 2. Stay on **Overview** to read their summary and the same headline cards, now scoped to them. 3. Switch tabs to get from the summary to the underlying work. ![An agent performance page with the Tickets tab open, showing their tickets beside the QA score cards and metrics rail](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/insights-agent-detail.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=54c631831a2e6fb0428aa408bc71151a) | Tab | What it holds | | - | - | | **Overview** | The written summary, headline cards, and the metrics rail | | **Tickets** | Every ticket they handled in the period, with search, filters, and display controls | | **Evaluations** | Their evaluations, openable one by one to see criterion scores and feedback | | **Coaching** | Their coaching sessions, plus **New session** to log another | | **Reports** | Their weekly performance reports, chosen from the selector in the header | The date control lives on **Overview**, and moving to **Tickets** or **Evaluations** swaps it for that list's own search, filter, and display controls. The period you picked still applies; it just stops being adjustable from there. Both lists open the ticket or evaluation in place. The rail on an agent's page carries a few numbers the team page does not: **Resolved tickets**, **Average handling time**, **Evaluation coverage**, and **Coaching sessions**. Read the score together with **Evaluation coverage**: 90 across eight evaluations means less than 90 across eighty. ## Turn what you found into coaching Coaching sessions are created from this page rather than from the Coaching page; the evidence you want to attach is already in front of you. Open **Coaching**, click **New session**, and pull in the tickets and evaluations you just read. [Run a coaching session](/guides/coaching/run-a-session) covers writing one that the agent can act on, and the sessions you log show up in the **Coaching sessions** count on this same page. ## When the numbers look wrong Before escalating a surprising score, confirm the boring explanations: * The period is a week, not the last seven days. A Monday morning reading is one day of data. * The agent is on the roster and on the team you expect. Evaluations stay with the team the agent belonged to when they were scored, so somebody who changed team mid-period shows up under both. * Evaluations exist for the period. **Evaluation coverage** near zero means the score is built on a handful of tickets. * Your own membership is not scoped more narrowly than you assumed. If a specific agent is absent rather than wrong, [An agent is missing from reports](/guides/troubleshooting/agent-missing-from-reports) walks the causes in order. When a score itself is the surprise, [Why is this score different from what I expected?](/guides/troubleshooting/why-is-my-score-different) covers how the same tickets can produce different averages. ## Related * [Insights overview](/guides/insights/overview) * [Run a coaching session](/guides/coaching/run-a-session) * [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster) * [Set score bands and the contest window](/guides/settings/set-score-bands-and-contest-window) * [An agent is missing from reports](/guides/troubleshooting/agent-missing-from-reports) * [Rulebase glossary](/guides/concepts/glossary) # Use Instruct Source: https://docs.rulebase.co/guides/instruct/use-instruct Ask Rulebase questions about your own QA and support data, then keep, expand, and share the answers it builds. Instruct is the chat inside Rulebase that can read your workspace. Ask it which issues drove the most tickets last month, have it pull a ticket apart and explain what the agent missed, or point it at a policy document and ask what your team is supposed to do. It answers by querying your own data rather than guessing, and when the answer needs more than a paragraph it builds something you can open and send to someone else. ## Before you start Instruct is granted per role, and it is not on for everyone by default: administrators have it, and every other role needs it added. If your sidebar has no **Chats** section and no **Instruct** button sits in the page header, that is what you are missing: an admin can grant the **Instruct** permission to your role under [Settings > People > Roles](https://app.rulebase.co/settings/people/roles). Instruct reads what has already synced into Rulebase, so a workspace in its first week will tell you it could not find something rather than inventing a number. ## Start a chat, from anywhere There are two ways in. **A full chat page**, from **New chat** at the bottom of the **Chats** section in the sidebar. Use it for analysis you expect to come back to, or want to send to someone. **An overlay**, from the **Instruct** button in the page header or Cmd+J (Ctrl+J on Windows) anywhere in the app. Esc tucks it away. The overlay opens on top of the screen you were reading and takes that screen with it, so on a ticket, a dashboard, or a scorecard you can ask "what went wrong on this one?" without naming it. Either way, type the question the way you would ask a colleague. Working out which tickets, evaluations, or records to read is Instruct's job, not yours. ## Point it at something specific Type `@` in the composer to search your workspace and drop a real record into the message. Tickets, teammates, agents on your roster, customers, companies, onboarding cases, and knowledge base documents all resolve this way, and a mentioned record means Instruct reads the ticket you meant instead of searching for one that matches your description. The settings icon beside the paperclip opens three more choices: * **Models** — which model answers. Leave it on the default unless you have a reason to change it. * **Skills** — a saved set of instructions for a recurring job. Attaching one tells Instruct how your team wants that job done. * **Knowledge base** — a specific document to read, for when the answer should come from your own policy rather than from ticket data. ![Instruct composer toolbar open showing Models, Skills, and Knowledge base](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/instruct-composer-toolbar.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=ab64b65c30a4cd7f017d7be667a3fad3) > [!TIP] > > If an answer comes back close but not right, reply in the same chat rather than > starting over. Everything already established in the thread still applies, so > "split that by team" or "only the last 30 days" is a short follow-up instead of a > rebuilt question. ## Work with what it builds Above each answer sits a line like *Ran 4 commands, retrieved 2 schemas, ran 7 queries*. Expand it when a number looks wrong and you get the actual queries. When the answer deserves a layout rather than a paragraph, Instruct builds a **canvas**: a small interactive view rendered from your data, with headline figures, tables, and rows you can click into for the tickets behind them. ![An Instruct answer with a Ticket volume and top issues canvas showing total tickets, the top issue, and a table of issues by volume](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/instruct-canvas-answer.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=a416bb521565482969fd75b63cddd5d9) The icons in the card's corner copy a link to the canvas and open it larger; canvases Instruct produced as files get a download icon too. Ask for a change ("group these by week", "add average handling time") and the canvas is rewritten in place rather than repeated further down the chat. Not everything arrives as a canvas. Instruct also writes to the product when you ask it to: dashboards and the charts on them, sheets, documents saved into Knowledge. Ask it to set up QA evaluations for a list of tickets, or attach a CSV of ticket IDs, and it comes back with a confirmation card where you drop the rows you did not want before anything is submitted; nothing is written on its own initiative. Ask for "an agent that emails us a weekly summary" and you will usually get a **report** (a saved, scheduled summary) rather than an agent. Recurring deliverables are what reports are for. Ask for something that watches for events and acts on them and you get a managed agent instead. See [Create a managed agent](/guides/agents/create-a-managed-agent) for that side of it. ## Share the result **Share** in the chat header copies a link to the whole conversation, canvases included. The share icon on a canvas copies a link to just that canvas, on its own page. Which one you want depends on whether the reasoning matters or only the result does. Both links need the recipient signed in to your Rulebase organization; neither is a public URL. They do not need Instruct access of their own to read what you send, so a link is a reasonable way to get an answer in front of someone whose role does not include Instruct. ## Keep your chat list usable Chats live under **Chats** in the sidebar: anything you have pinned, then your five most recent. A chat is named after your opening message, which stops being useful somewhere around the tenth one, so from the `...` menu on a chat you can **Rename** it to something you will recognise next week or **Archive** it to take it out of your history. The pin icon that appears on hover keeps the ones you return to at the top. ## Related * [Rulebase glossary](/guides/concepts/glossary) * [Create a managed agent](/guides/agents/create-a-managed-agent) * [MCP](/guides/mcp) # Aircall Source: https://docs.rulebase.co/guides/integrations/aircall Authorize Aircall so Rulebase pulls call recordings onto the support tickets they belong to and transcribes them for review. Connect Aircall to Rulebase to review the calls behind your support tickets. Aircall calls do not arrive as records of their own. Each call is attached to the conversation for the ticket it belongs to, so the call is reviewed alongside the emails and chats on the same customer issue. ## Prerequisites * You must be an Aircall admin or account owner to complete the installation. * Zendesk should be connected, because Rulebase's automatic call matching runs on Zendesk tickets. * You must have the Administrator role in Rulebase. ## Installation ### Step 1: connect Aircall 1. Log in to your [Rulebase dashboard](https://app.rulebase.co) 2. Go to **Settings** > **Integrations** 3. Find **Aircall** in the list and click **Add** 4. Sign in with your Aircall admin credentials 5. Authorize Rulebase to access your Aircall account ## How a call reaches a conversation Rulebase finds Aircall calls in two ways, and both start from a ticket: * **A recording link on the ticket.** Wherever ticket content contains an Aircall recording link, Rulebase fetches that call from Aircall. * **Phone number and time.** When a Zendesk ticket is updated, Rulebase looks for an Aircall call to or from the customer's phone number around the time the ticket was created, then links the closest match. Either way, the call becomes a call in the Rulebase conversation and carries its direction, the customer's phone number, and the recording. Rulebase transcribes the recording, so the call can be scored and searched like any other interaction. The Aircall user who handled the call is matched to a Rulebase employee by email address, which attributes the call to that agent for QA. ## When a call does not appear * **No phone number to match on.** Rulebase reads the customer's phone from the Zendesk requester, or from the `Caller Phone number` line in the Aircall note on the ticket. A ticket with neither cannot be matched. * **The call sits far from the ticket.** Matching is anchored to when the ticket was created, so a call placed well before or well after that may fall outside the window Rulebase searches. * **No recording in Aircall.** Missed calls, and calls made with recording turned off, leave nothing to attach or transcribe. * **A different email in Aircall.** If an agent's Aircall email is not the email on their Rulebase employee record, the call still attaches but is credited to a separate employee. # Alloy Source: https://docs.rulebase.co/guides/integrations/alloy Store your Alloy API token and secret in Rulebase so a Rulebase workflow can be pointed at your Alloy account. Connect Alloy to Rulebase so Rulebase workflows can be configured against your Alloy account. ## Prerequisites * You must have an Alloy API token and API secret. * You must have the Administrator role in Rulebase. ## Connect Alloy 1. Log in to your [Rulebase dashboard](https://app.rulebase.co). 2. Go to **Settings** > **Connections**. 3. Find **Alloy** and click **Add connection**. 4. Enter your Alloy API token. 5. Enter your Alloy API secret. 6. Click **Connect**. ## Configure Alloy workflows After connecting, work with your Rulebase account team to configure which Alloy data your workflows can access and how they use it. Coordinate credential rotations with them so those workflows stay connected. # ComplyAdvantage Source: https://docs.rulebase.co/guides/integrations/complyadvantage Connect ComplyAdvantage to Rulebase with an API key and account region. Connect ComplyAdvantage to Rulebase with an API key and the region where your account is hosted. ## Prerequisites * You need a ComplyAdvantage API key. * You need to know the region where your ComplyAdvantage account is hosted. * You need the Administrator role or permission to manage integrations in Rulebase. ## Choose the correct account region Rulebase uses your region to select the corresponding ComplyAdvantage API: | Region | API base URL | | - | - | | Europe | `https://api.complyadvantage.com` | | United States | `https://api.us.complyadvantage.com` | | Asia Pacific | `https://api.ap.complyadvantage.com` | Ask your ComplyAdvantage account administrator if you do not know which region to select. ## Connect ComplyAdvantage in Rulebase 1. Log in to your [Rulebase dashboard](https://app.rulebase.co). 2. Go to **Settings** > **Connections**. 3. Find **ComplyAdvantage** under **Risk and compliance**. 4. Click **Add connection**. 5. Select your account **Region**. New connections default to **Europe**. 6. Enter the **API key**. 7. Click **Connect**. Rulebase encrypts the API key before storing it. The selected region is stored with the connection so Rulebase uses the correct API endpoint. ## Update your connection To rotate the API key or change regions, submit the new configuration from **Settings** > **Connections**. Rulebase replaces the stored ComplyAdvantage connection for your organization. # Freshdesk Source: https://docs.rulebase.co/guides/integrations/freshdesk Connect Freshdesk with your subdomain and API key so Rulebase syncs email, chat, and phone tickets with their replies, notes, and SLA targets. Connect Freshdesk to Rulebase to evaluate tickets your team handles in Freshdesk. ## Prerequisites * You must have a Freshdesk API key. Rulebase sees exactly what the agent who owns that key can see, so use a key from an account with access to every ticket you want reviewed. * You must know your Freshdesk subdomain, for example `my-company` for `my-company.freshdesk.com`. * You must have the Administrator role in Rulebase. ## Installation ### Step 1: connect Freshdesk 1. Log in to your [Rulebase dashboard](https://app.rulebase.co) 2. Go to **Settings** > **Integrations** 3. Find **Freshdesk** in the list and click **Add** 4. Enter your Freshdesk subdomain 5. Enter your Freshdesk API key 6. Click **Add** ## What gets synced Rulebase checks Freshdesk every fifteen minutes for tickets that changed, and imports each one as a conversation: * **The description and every reply** become the messages in the conversation. Private notes come across as internal notes, so reviewers can see them without treating them as customer-facing. * **Attachments** on the ticket and on individual replies are downloaded and kept with the message they belong to. * **The requester** becomes a Rulebase customer, and the ticket's company becomes a Rulebase company. * **Agents** become Rulebase employees, matched by their Freshdesk email; that match attributes QA scores. * **Priority, type, status, and resolution time** carry over, so tickets can be filtered and used in scorecard rules. Resolved and closed tickets both count as resolved in Rulebase. * **Custom ticket fields** become Rulebase conversation fields with the same names. Rulebase also imports your Freshdesk SLA policies and business hours, and uses them to flag response and resolution breaches on synced tickets. ## Ticket types Rulebase imports Only tickets whose Freshdesk source is **Email**, **Outbound Email**, **Phone**, or **Chat** are imported. Tickets created through any other Freshdesk source, such as the customer portal or the feedback widget, are not imported. If a ticket you expected is missing, check its source first, then confirm the API key's agent can open that ticket in Freshdesk. # Google Drive Source: https://docs.rulebase.co/guides/integrations/google-drive Share folders, Docs, and Sheets with the Rulebase service account so Rulebase workflows can read them, write back to spreadsheets, and import content into the knowledge library. Use Google Drive with Rulebase by sharing Drive content with the Rulebase service account. Access is controlled entirely by what you share in Google Drive; there is no OAuth screen and no Connect step. ## Prerequisites * You must have permission to share the Google Drive folder, Google Doc, or Google Sheet Rulebase should use. * You must have the Administrator role in Rulebase. ## Installation ### Step 1: share content with Rulebase 1. Log in to your [Rulebase dashboard](https://app.rulebase.co) 2. Go to **Settings** > **Connections** 3. Open **Google Drive** and copy the Rulebase service account email 4. In Google Drive, share the folder or file with that email address Sharing a folder covers the files inside it, so a single shared working folder is usually easier to manage than a list of individual files. ### Step 2: import into Knowledge (optional) Sharing alone does not add library documents. To make Drive content searchable during AI QA evaluations: 1. Go to **Knowledge** 2. Open **Manage knowledge base** 3. Choose **Import** > **Google Drive** 4. Paste the shared folder or file URL 5. Click **Import** Only content under that URL is imported. Rulebase never scans your whole Drive. ## What Rulebase can do with shared content ### Workflows and agents A Rulebase workflow acting on Drive can find and list folders, create folders, and read a file's metadata and text. Text extraction covers Google Docs, PDFs, spreadsheets, CSVs, and other text-like files. For spreadsheets, a workflow can also copy a template, inspect a spreadsheet's tabs and headers, read a range of values, and write values back. Nothing is read on a schedule for workflows. Rulebase opens a file at the moment a workflow runs, so the workflow always sees the current version. ### Knowledge library Imported Drive files become knowledge base documents (one document per file). Sources refresh once a day on the knowledge base sync schedule, and you can run **Sync** manually from **Manage knowledge base**. Files that are deleted or unshared are removed from the library on the next sync. ## Choose the right sharing level Share as **Viewer** when a workflow or knowledge import only needs to read source material such as policies or reference sheets. Share as **Editor** only where a workflow needs to create a folder or write to a spreadsheet, since that same access lets it overwrite the ranges it writes to. If a workflow or import reports that it cannot find a file you can see yourself, the file is almost certainly shared with your account rather than with the service account email. # Gradient Labs Source: https://docs.rulebase.co/guides/integrations/gradient-labs Connect Gradient Labs to Rulebase with a management and an integration API key. Gradient Labs assigns each API key exactly one role, so one key cannot do both jobs: * **Management** — updates procedures, tools, resources, and other agent configuration. * **Integration** — runs conversations, including test runs against your agent. Submit both keys to use everything Rulebase can do with Gradient Labs. If you only have one, submit it now and add the other later — you keep the capabilities of the keys you have shared. ## Prerequisites * You must be a Gradient Labs Admin. * You must have the Administrator role or `integrations:manage` permission in Rulebase. ## Installation ### Step 1: Create a management API key in Gradient Labs (for agent configuration) 1. Sign in to Gradient Labs. 2. Go to **Integrations** > **API**. 3. In the **API Keys** section, click **New API key**. 4. Give the key a name, choose the **Management** role, and create the key. 5. Copy the key immediately. Gradient Labs shows it only once. ### Step 2: Create an integration API key in Gradient Labs (for conversations) 1. In **Integrations** > **API**, click **New API key** again. 2. Give the key a name, choose the **Integration** role, and create the key. 3. Copy the key immediately. Gradient Labs shows it only once. Requests sent with the wrong role fail with `permission_denied`, so keep the two keys distinct. See [Gradient Labs API authentication](https://api-docs.gradient-labs.ai/pages/api-reference/authentication) for details about API-key roles and authentication. ### Step 3: Connect Gradient Labs in Rulebase 1. Log in to your [Rulebase dashboard](https://app.rulebase.co). 2. Go to **Settings** > **Connections**. 3. Find **Gradient Labs** and click **Add connection**. 4. Paste the **Management API key**, the **Integration API key**, or both. 5. Click **Connect**. ## Next steps * To rotate or add a key, create it in Gradient Labs and submit it from **Settings** > **Connections**. A key you leave blank keeps its stored value. * See [Roles and permissions](/guides/roles-and-permissions) to control which Rulebase users can manage connections. # Intercom Source: https://docs.rulebase.co/guides/integrations/intercom Authorize Rulebase in Intercom to sync conversations and tickets, along with the teammates and teams that QA scores are attributed to. Connect Intercom to Rulebase via OAuth2 to evaluate the conversations and tickets your team handles in Intercom. ## Prerequisites * You must be an Intercom admin to complete the installation. * You must have the Administrator role in Rulebase. ## Installation ### Step 1: connect Intercom 1. Log in to your [Rulebase dashboard](https://app.rulebase.co) 2. Go to **Settings** > **Integrations** 3. Find **Intercom** in the list and click **Add** 4. Sign in with your Intercom admin credentials in the popup 5. Authorize Rulebase to access your Intercom workspace ## What gets synced Once connected, Rulebase starts syncing: * **Conversations and tickets** across all your inboxes * **Teammates** as Rulebase employees, so QA scores and feedback can be attributed to the agent who handled each conversation * **Teams** as Rulebase groups, so scorecards and routing rules can be scoped to a specific team * **Contacts and companies** to enrich customer and company records * **Conversation custom attributes** as Rulebase conversation fields, so they can be used in scorecard rules and filters ## Next steps * New conversations are picked up in real time. Historical conversations are backfilled in the background after the initial connect. * See [Roles and permissions](/guides/roles-and-permissions) to control which Rulebase users can review Intercom conversations. # Jira Source: https://docs.rulebase.co/guides/integrations/jira Connect Jira with OAuth or an Atlassian API token so Rulebase can sync issues and service request types from your projects. Connect Jira to Rulebase to work with Jira issues and service request types from Rulebase. ## Prerequisites * You must have permission to authorize Jira apps, or you must have an Atlassian API token. * You must have access to the Jira projects Rulebase should read or update. * You must have the Administrator role in Rulebase. ## Installation ### Step 1: connect Jira 1. Log in to your [Rulebase dashboard](https://app.rulebase.co) 2. Go to **Settings** > **Integrations** 3. Find **Jira** in the list and click **Add** 4. Choose **Connect with OAuth**, or enter your Jira URL, email, and API token 5. Authorize Rulebase or submit the API token form ## Authentication options | Method | Use when | Required values | | - | - | - | | OAuth | You can authorize Rulebase through Atlassian | Atlassian account access | | API token | You prefer to connect with an Atlassian API token | Jira URL, email, API token | ## What gets synced Rulebase can sync Jira issues and Jira service request types. Rulebase also creates the Jira webhook it needs to receive issue updates when connected through the standard Jira flow. ## Next steps * Confirm that the connected account can access the projects Rulebase should use. * Configure any Rulebase workflows that should create, link, or review Jira issues. # Langfuse Source: https://docs.rulebase.co/guides/integrations/langfuse Connect Langfuse to let Rulebase read your agent traces. Connect Langfuse to let Rulebase read the traces your agent already records. ## Get started ### Get your Langfuse credentials In Langfuse, open **Project settings** > **API keys** and copy: * Host URL * Public key * Secret key Use project API keys. Rulebase supports Langfuse Cloud and self-hosted Langfuse v4 projects with Observations API v2 enabled. ### Add your credentials to Rulebase In Rulebase, go to **Settings** > **Connections** > **Langfuse** and click **Add connection**. Enter your credentials and click **Connect**. ### Verify the connection Rulebase verifies access to your project and traces, then shows the connected project name. # Linear Source: https://docs.rulebase.co/guides/integrations/linear Connect Linear with OAuth so Rulebase can link Linear issues mentioned in conversations and notes and keep their status current. Connect Linear to Rulebase to see the Linear issues your team references while working a ticket. When an agent pastes a Linear link or types an issue identifier such as `INC-1221` into a conversation or note, Rulebase looks the issue up and adds it to the conversation's linked activity. Issue status updates flow back from Linear, so the linked activity shows where the issue stands without leaving Rulebase. ## Prerequisites * You must be a Linear workspace admin, or otherwise allowed to authorize applications for your workspace. * You must have the Administrator role in Rulebase. ## Installation ### Step 1: connect Linear 1. Log in to your [Rulebase dashboard](https://app.rulebase.co) 2. Go to **Settings** > **Connections** 3. Find **Linear** in the list and click **Add connection** 4. Choose the Linear workspace in the popup 5. Review the requested permissions and click **Authorize** Rulebase installs as an application in your workspace and asks for these permissions: | Scope | Why Rulebase needs it | | - | - | | `read` | Look up issues referenced in conversations and notes | | `write` | Update issues from Rulebase workflows | | `comments:create` | Reply on issues where Rulebase is mentioned or assigned | | `app:assignable` | Let your team assign issues to Rulebase | | `app:mentionable` | Let your team @mention Rulebase in issue comments | ## What gets synced * **Linked issues.** Rulebase scans conversation messages and internal notes for Linear issue URLs (`https://linear.app/your-workspace/issue/INC-1221`) and bare identifiers that match one of your workspace's team keys. Matching issues appear under the conversation's linked activity with their title, identifier, and current state. * **Status updates.** Linear notifies Rulebase when a linked issue changes, so the state shown in Rulebase stays current. * **Rulebase in Linear.** Because Rulebase is installed as an application, your team can mention or assign Rulebase on a Linear issue to ask about related conversations. ## Next steps * Open a conversation that references a Linear issue and confirm the issue appears in its linked activity. * Review how to [manage integrations](/guides/integrations/manage-integrations) if a connection stops updating and needs reconnecting. # Looker Studio Source: https://docs.rulebase.co/guides/integrations/looker-studio Store Looker API credentials and your Looker instance URL in Rulebase so a Rulebase workflow can be pointed at your reporting. Connect Looker Studio to Rulebase so Rulebase workflows can be configured against your reporting environment. ## Prerequisites * You must have API credentials from Looker: a client ID and client secret. * You must know the URL of your Looker instance. * You must have the Administrator role in Rulebase. ## Connect Looker Studio 1. Log in to your [Rulebase dashboard](https://app.rulebase.co). 2. Go to **Settings** > **Connections**. 3. Find **Looker Studio** and click **Add connection**. 4. Enter the client ID and client secret. 5. Enter the URL of your Looker instance, for example `https://example.lookerstudio.com`. 6. Click **Connect**. Enter the instance URL, not a link to a report or dashboard. Rulebase needs an address it can reach the Looker API on, and a shared report link will not work. ## Configure reporting workflows After connecting, work with your Rulebase account team to configure the queries your workflows can run against your reporting environment. # Manage integrations Source: https://docs.rulebase.co/guides/integrations/manage-integrations Check which sources are connected and still syncing, add a second account, and disconnect one without losing the tickets you already have. [Settings > Connections](https://app.rulebase.co/settings/connections) lists every connector Rulebase supports and every connection you have made. Use it to check that a source is connected and still pulling data, to add another account, and to disconnect one without losing the history it already synced. This guide covers the page itself. For the specifics of setting up a particular tool, each connector has its own guide: * **Help desks and ticketing** — [Zendesk](/guides/integrations/zendesk), [Freshdesk](/guides/integrations/freshdesk), [Intercom](/guides/integrations/intercom), [Jira](/guides/integrations/jira) * **Calls** — [Aircall](/guides/integrations/aircall), [XCALLY](/guides/integrations/xcally), [Ringover](/guides/integrations/ringover) * **Knowledge sources** — [Notion](/guides/integrations/notion), [Google Drive](/guides/integrations/google-drive), [Scribe](/guides/integrations/scribe) * **Sales and CRM** — [Salesforce](/guides/integrations/salesforce), [Outreach](/guides/integrations/outreach) * **Risk and compliance** — [Unit21](/guides/integrations/unit21), [Alloy](/guides/integrations/alloy), [Veriff](/guides/integrations/veriff), [ComplyAdvantage](/guides/integrations/complyadvantage) * **Messaging and reporting** — [Slack](/guides/integrations/slack), [Looker Studio](/guides/integrations/looker-studio), [Langfuse](/guides/integrations/langfuse) * **Other** — [Gradient Labs](/guides/integrations/gradient-labs) ## Before you start * You need an admin role, or a role with permission to manage integrations. Without it the **Add** buttons are visible but disabled. * Connecting a source usually means signing in to that tool, so have an account there with permission to authorize apps. ## Read the page The page lists every connector alphabetically with an **Add** button on the right. A connector you have connected gets a row underneath it, one row per account, showing which account it is and when Rulebase last heard from it. ![Zendesk row on the Connections page with a connected subdomain and a Last synced on timestamp](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/integrations-connected-source.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=e4528854715647424b25c74520ec902f) In each row: * **The identifier** tells you which account this is. A Zendesk subdomain, a Slack workspace, a Jira site. You can connect two accounts of the same tool: a second Zendesk subdomain gets its own row under Zendesk, and each syncs independently. * **Last synced on** is your health check. It moves whenever Rulebase records activity for that connection, so on a busy queue you want to see a timestamp from minutes ago, not last Tuesday. > [!NOTE] > > The Connections page does not show connection errors. A connection whose > credentials were revoked, whose token expired, or whose webhook was deleted looks > exactly like a healthy one; the only sign is that **Last synced on** stops > moving. If a source has gone quiet, check that timestamp before assuming Rulebase > lost the data. [Integration stopped syncing](/guides/troubleshooting/integration-stopped-syncing) > walks through the rest. **Slack** and **Front** rows show **Connected on** with the date you authorized them rather than a sync timestamp, and they have no actions menu. Contact support to configure, change, or remove an **XCALLY** connection. ## Look at what a source has been doing When you want more than a timestamp, open the `...` menu on the connection and choose **View activity**. ![Actions menu on a connected Zendesk source showing View activity and Disconnect](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/integrations-actions-menu.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=e0ec0dd8396f6b5559e4c836f6a3ff05) ![Activity Log dialog for a Zendesk connection listing an imported ticket, a synced ticket, and a received webhook](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/integrations-activity-log.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=90fb6f077c26e62b77dcc6c182c371e2) The activity log is one timestamped line per event, in the shape Rulebase received it: tickets imported, tickets re-synced, webhooks that arrived. Use **Search logs...** to find a specific ticket ID, or **Pick date & time** to narrow to the window you care about. You do not need to read the JSON. An empty log, or a log whose last line is from three days ago, tells you the connection is not delivering, even when the ticket list looks the same either way. If you open a support ticket, include the timestamp of the last line. **Jira** adds **Configure** to this menu, which holds a single switch, *Import service requests as interactions*. Turn it on to bring Jira service requests into Rulebase as interactions that can be evaluated. **Unit21** adds **Edit**, for the credentials, and **Copy webhook URL**, which is the URL you paste into Unit21 so it can push alerts to Rulebase. ## Reconnect a source To reconnect, click the same **Add** button you used the first time. There is no separate reconnect button and no need to disconnect first: a stalled connection is almost always expired or revoked credentials, and signing in again refreshes them. Authorize the same account the connection already uses. For Zendesk, Slack, Jira, Salesforce, and Intercom, Rulebase recognizes the account you come back with (the subdomain, workspace, site, org, or app) and refreshes the credentials on the connection it already has. Nothing is duplicated and the connection keeps its place in the sync. Everywhere else, and any time you authorize a different account, you get a second connection alongside the first. If you meant to repair the original rather than add a second workspace, wait for the new connection to start syncing, then disconnect the stale one so nobody has to guess which row is live. ## Disconnect a source 1. Open the `...` menu on the connection and choose **Disconnect**. 2. Confirm in the dialog. ![Disconnect confirmation dialog reading Are you sure you want to disconnect this integration](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/integrations-disconnect-dialog.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=95af6308b78d245c04ac00585b4c1693) The dialog says the action cannot be undone, and for the connection itself that is true: the credentials are gone and the row disappears from the page. Your data stays. **What stops:** new tickets and conversations from that source, and any updates to the ones you already have. **What stays:** every ticket, transcript, evaluation, and QA score that already synced. They remain in Rulebase, still searchable, still counted in your reports and in each agent's performance history. Disconnecting Zendesk does not empty your ticket list or reset last quarter's scores. > [!TIP] > > Disconnecting is the right move when you are migrating help desks or you > connected the wrong account. Leave a connection you are merely not using in > place: it costs nothing, and reconnecting later starts a new connection rather > than resuming the old one. ## The API keys below The same page ends with an **API keys** section, which is a different job: keys are for pushing data into Rulebase from a tool we have no connector for, rather than for Rulebase pulling from a tool you use. [Set up API access](/guides/api-access) covers those. ## Related * [Zendesk](/guides/integrations/zendesk) * [Slack](/guides/integrations/slack) * [Set up API access](/guides/api-access) * [Configure notifications](/guides/settings/configure-notifications) * [Integration stopped syncing](/guides/troubleshooting/integration-stopped-syncing) # Mixmax Source: https://docs.rulebase.co/guides/integrations/mixmax Connect Mixmax to Rulebase to import Dialer sales calls and recordings. Connect Mixmax to Rulebase with a developer token so Dialer sales calls and recordings can sync into Rulebase for compliance monitoring. ## Prerequisites * You must have the Administrator role in Rulebase. * Create a **developer API token** in [**Mixmax Settings > Integrations**](https://app.mixmax.com/dashboard/settings/integrations). Mixmax shows the token once, so save it securely before leaving the page. Use a classic developer token (not a scoped `mxa_…` app credential). Scoped tokens are rejected by the Mixmax Calls API. * Prefer a **workspace admin** token so Rulebase can list calls for the whole workspace (`scope=workspace`). * Mixmax's Calls read API is limited-beta. If list or recording requests return `404`, ask Mixmax Support or `developer@mixmax.com` to allowlist your workspace for the Calls API (and Dialer Suite if you need a trial). * The workspace needs the Dialer **calling** feature for recording download. ## Store your developer token 1. Log in to your [Rulebase dashboard](https://app.rulebase.co). 2. Go to **Settings > Connections**. 3. Open **Mixmax** under **Voice and calls**. 4. Click **Add connection**. 5. Paste your Mixmax developer token. 6. Click **Save token**. Rulebase encrypts the token before storing it. The token is not shown again in the connection settings. After the token is saved, Rulebase discovers Dialer calls on a daily schedule, imports them as sales calls, and downloads recordings when available. ## Update or remove the token * To replace the token, add the Mixmax connection again and save the new token. * To remove it, open the connection menu and click **Disconnect**. # Notion Source: https://docs.rulebase.co/guides/integrations/notion Authorize Rulebase in Notion, share the pages it should read, and import them into your knowledge base as documents. Connect Notion to Rulebase to use Notion pages as knowledge base content. Connecting the workspace is only half the job: pages arrive in Rulebase after you run an import from **Knowledge**. ## Prerequisites * You must be able to authorize integrations in the Notion workspace. * You must have access to the pages Rulebase should read. * You must have the Administrator role in Rulebase. ## Installation ### Step 1: connect Notion 1. Log in to your [Rulebase dashboard](https://app.rulebase.co) 2. Go to **Settings** > **Integrations** 3. Find **Notion** in the list and click **Add** 4. Select the Notion workspace 5. Grant Rulebase access to the pages it should read Notion access is page by page. Rulebase can only see pages you explicitly share with the integration, and sharing a parent page shares the pages nested inside it. ### Step 2: import into Knowledge 1. Go to **Knowledge** 2. Click **Manage** to open **Manage knowledge base** 3. Open **Import** > **Notion** 4. Select the connected Notion account and click **Import** The **Notion** option only appears once a Notion connection exists. ## What lands in your knowledge base Each Notion page becomes one knowledge base document, with the page title, its content, and a link back to the original page. Documents can then be cited in evaluations and used in training and coaching workflows. ## Refresh behavior * Notion sources refresh once a day on the knowledge base sync schedule * A page is only reprocessed when it was edited since its last sync, so refreshes are cheap and do not create duplicates * Pages that are deleted, or that stop being shared with the Rulebase integration, are removed from the knowledge base on the next sync If a page you expected is missing after an import, the usual cause is sharing: open the page in Notion, check the connection is listed under its access settings, then run the import again. # Outreach Source: https://docs.rulebase.co/guides/integrations/outreach Authorize Rulebase in Outreach to sync sales calls with their user and prospect data, plus the webhook that delivers call updates. Connect Outreach to Rulebase to evaluate sales calls and related customer interactions. ## Prerequisites * You must be able to authorize apps in Outreach. * You must have access to the Outreach calls Rulebase should read. * You must have the Administrator role in Rulebase. ## Installation ### Step 1: connect Outreach 1. Log in to your [Rulebase dashboard](https://app.rulebase.co) 2. Go to **Settings** > **Integrations** 3. Find **Outreach** in the list and click **Add** 4. Sign in with your Outreach credentials 5. Authorize Rulebase to access Outreach ## What gets synced Rulebase syncs Outreach calls and related user and prospect data. Rulebase also creates the Outreach webhook it needs to receive call updates. ## Next steps * Confirm that the connected Outreach account can access the calls Rulebase should evaluate. * Review synced calls in Rulebase after the first sync completes. # Ringover Source: https://docs.rulebase.co/guides/integrations/ringover Connect Ringover to Rulebase with an API key. Connect Ringover to Rulebase with an API key from your Ringover account. ## Prerequisites * You must have a Ringover API key. * You must have the Administrator role in Rulebase. ## Connect Ringover 1. Log in to your [Rulebase dashboard](https://app.rulebase.co). 2. Go to **Settings** > **Connections**. 3. Find **Ringover** and click **Add connection**. 4. Enter your Ringover API key. 5. Click **Connect**. ## Update your connection To rotate the API key, create a replacement in Ringover and submit it from **Settings** > **Connections**. # Salesforce Source: https://docs.rulebase.co/guides/integrations/salesforce Authorize Rulebase in Salesforce to enrich customer and company records with Contact and Account data, and link them through Zendesk external IDs. Connect Salesforce to Rulebase to enrich customers and companies with Salesforce Contact and Account data. ## Prerequisites * You must have permission in Salesforce to authorize third-party apps. * You must have the Administrator role in Rulebase. ## Installation ### Step 1: connect Salesforce 1. Log in to your [Rulebase dashboard](https://app.rulebase.co) 2. Go to **Settings** > **Integrations** 3. Find **Salesforce** in the list and click **Add** 4. Sign in with your Salesforce credentials 5. Authorize Rulebase to access your Salesforce account ## Zendesk bridge If Zendesk is also connected, Rulebase can link Salesforce records to Rulebase customers and companies. To enable this, set the `external_id` field on your Zendesk users and organizations to the corresponding Salesforce Contact ID or Account ID. Rulebase detects the link and syncs enrichment data. ## Next steps Once connected, Rulebase starts syncing Contact and Account data. * Customer records are enriched with name, email, and phone from Salesforce Contacts * Company records are enriched with name from Salesforce Accounts * If you also use Zendesk, confirm your `external_id` values are set to valid Salesforce IDs to enable automatic linking # Scribe Source: https://docs.rulebase.co/guides/integrations/scribe Generate a Scribe API key, connect it in Rulebase, and import Scribe guides and knowledge pages into your knowledge base. Connect Scribe to Rulebase to use Scribe documents and knowledge pages as knowledge base content for evaluations and training. ## Prerequisites * Your Scribe organization must be on an **Enterprise Grid** or **Global** plan with Search & Retrieval API access. * You must be able to generate a Scribe API key with access to the teams whose documents Rulebase should read. * You must have the Administrator role in Rulebase. ## Installation ### Step 1: generate a Scribe API key 1. Open Scribe **Settings** → **Developer Access** 2. Generate a new API key 3. Copy the key immediately (it starts with `sc_` and is shown only once) API keys are scoped to one or more Scribe teams. Rulebase imports every document those teams can access. ### Step 2: connect Scribe in Rulebase 1. Log in to your [Rulebase dashboard](https://app.rulebase.co) 2. Go to **Settings** → **Integrations** 3. Find **Scribe** and click **Add** 4. Paste your Scribe API key and save ### Step 3: import into Knowledge 1. Go to **Knowledge** 2. Click **Manage** to open **Manage knowledge base** 3. Open **Import** → **Scribe** 4. Select the connected Scribe account and click **Import** Rulebase syncs all accessible Scribe documents (`scribe` guides and `page` knowledge pages), converts them into knowledge base documents, and downloads screenshots for SOP fidelity. ## What gets synced * Published Scribe documents and knowledge pages accessible to the API key * Step text for Scribe guides, including screenshots when available * Page body content from Scribe knowledge pages * Source links back to the original Scribe documents ## Refresh behavior * Importing creates a knowledge base source and starts an initial sync * Sources refresh on the regular knowledge base sync schedule * You can also refresh a source from **Manage knowledge base** * Documents are updated in place by Scribe document ID, so refreshes do not create uncontrolled duplicates * Documents removed from Scribe (or no longer accessible to the key) are discarded from the Rulebase knowledge base ## Next steps * Add the imported Scribe content to scorecards and training workflows * Confirm evaluations can cite the ingested Scribe documents # Slack Source: https://docs.rulebase.co/guides/integrations/slack Install the Rulebase app in your Slack workspace, then choose which Slack channels Rulebase alerts post to. Connect Slack to Rulebase so alerts and digests reach your team where they already work. The install covers the whole workspace; you choose the destination channels afterwards in Rulebase. ## Prerequisites * You must be allowed to install apps in your Slack workspace. * You must have the Administrator role in Rulebase. ## Installation ### Step 1: connect Slack 1. Log in to your [Rulebase dashboard](https://app.rulebase.co) 2. Go to **Settings** > **Integrations** 3. Find **Slack** in the list and click **Add** 4. Choose the Slack workspace in the popup 5. Review the requested permissions and click **Allow** Rulebase stores the workspace connection, which lets it post messages to channels, send direct messages, and read your channel list. No channel is chosen during the install. ### Step 2: choose where alerts post 1. Go to **Settings** > **Notifications** 2. Under **Real-time complaints alerts**, pick the channel complaint alerts should go to Digests such as the compliance and customer issues summaries are sent by email to the recipients you list, not to Slack. [Configure notifications](/guides/settings/configure-notifications) covers the whole page. Slack alerts announce to a channel. To send a particular kind of conversation to a specific person or queue and track it until someone deals with it, use a [routing rule](/guides/quality-assurance/create-routing-rules) instead. ## When a message does not arrive * **Invite the Rulebase app to the channel.** Rulebase can only post where the app is a member. In Slack, open the channel and use `/invite @Rulebase`. This is the most common cause of a rule that looks configured but stays quiet. * **Check the channel is in the list.** The picker is built from the public channels Rulebase can see in your workspace, refreshed daily, so a channel created minutes ago or a private channel may not appear yet. * **If the picker is completely empty**, reconnect Slack from **Settings** > **Connections**. Contact Rulebase support if the channel list remains empty. * **Confirm a complaint was detected.** The complaints alert posts only when Rulebase classifies a conversation as a complaint, so a quiet period produces no messages at all. * **Check the connection itself.** [Manage integrations](/guides/integrations/manage-integrations) covers reading connection state and reconnecting. # Unit21 Source: https://docs.rulebase.co/guides/integrations/unit21 Add a Unit21 API key so Rulebase syncs cases and alerts with their dispositions, investigator notes, and the rules that fired. Connect Unit21 to Rulebase to review your Unit21 investigations and use them as the population for compliance testing. ## Prerequisites * You must have a Unit21 API key with access to the cases and alerts you want reviewed. * You must have the Administrator role in Rulebase. ## Installation ### Step 1: connect Unit21 1. Log in to your [Rulebase dashboard](https://app.rulebase.co) 2. Go to **Settings** > **Integrations** 3. Find **Unit21** in the list and click **Add** 4. Enter your Unit21 API key 5. Click **Add** To rotate the key later, or to add the webhook signing secret, open the **...** menu on the connection and choose **Edit**. ## What gets synced Unit21 cases become Rulebase compliance cases and Unit21 alerts become compliance alerts. Each includes: * **Title, description, and status**, plus the source the case or alert came from * **Disposition**, including who dispositioned it, when, and the notes they left * **Assignee**, so work can be reviewed by investigator * **Investigator notes, actions, and narratives** recorded in Unit21 * **The rules that fired**, imported as compliance rules and linked to every case and alert that triggered them Rulebase syncs about every ten minutes and looks for two things: records created since the last run, and records dispositioned since the last run. The second pass keeps a case that was opened last week and closed this morning up to date. The first sync after connecting reaches back twenty-four hours, so a connection made today will not backfill your full case history. ## Webhooks Unit21 webhooks are optional. Copy the URL from **Copy webhook URL** in the **...** menu on the connection and add the signing secret through **Edit**. Rulebase verifies and records each event and then queues a sync for that connection, normally within about a minute. If a sync is already running, a follow-up sync runs as soon as it finishes. The regular ten-minute sync still runs, so a missed webhook does not lose data. To see what has arrived, open **View activity** on the connection. ## Use the synced records Synced cases and alerts are available as population record types when you populate a compliance test, where you can sample by date range and, for alerts, by alert type. See [Populate compliance tests](/guides/compliance/populate-tests) for that workflow. Which of the two a test draws on is fixed by the program's **Applies to** setting, and a test sheet keeps the columns it was created with, so decide that before you sample anything. [Set up a compliance testing program](/guides/compliance/set-up-a-testing-program) covers defining the program, its controls, and how its population is sampled. # Veriff Source: https://docs.rulebase.co/guides/integrations/veriff Connect Veriff to Rulebase with an API key, shared secret, and integration base URL. Connect Veriff to Rulebase with credentials from your Veriff workspace. ## Prerequisites * You need a Veriff API key, shared secret, and base URL from the same Veriff integration. * You need the Administrator role or permission to manage integrations in Rulebase. ## Find your Veriff credentials 1. Sign in to Veriff. 2. Go to **Workspace** > **All integrations**. 3. Open the integration you want Rulebase to use. 4. Open **API keys**. 5. Copy the **API key**, **Shared secret**, and **Base URL**. Keep the API key and shared secret private. Rulebase encrypts both values before storing them. ## Connect Veriff in Rulebase 1. Log in to your [Rulebase dashboard](https://app.rulebase.co). 2. Go to **Settings** > **Connections**. 3. Find **Veriff** under **Risk and compliance**. 4. Click **Add connection**. 5. Enter the **API key**, **Shared secret**, and **Base URL**. 6. Click **Connect**. The base URL must use HTTPS and cannot contain embedded credentials, query parameters, or a URL fragment. ## Update your credentials To rotate the credentials, create or retrieve the replacement values in Veriff, then submit them again from **Settings** > **Connections**. Rulebase replaces the stored Veriff credentials for your organization. # XCALLY Source: https://docs.rulebase.co/guides/integrations/xcally Work with Rulebase to set up XCALLY, send call recordings to the upload API, and confirm Rulebase is receiving them. Rulebase provisions the XCALLY connection for you, and recordings are pushed to Rulebase rather than pulled from XCALLY, so there is no **Add** button next to **XCALLY** in **Settings** > **Integrations**. Your side of the work is sending each call recording, with its metadata, to the Rulebase conversation upload API. ## Prerequisites * Your Rulebase account team must set up the XCALLY connection. * You need someone who can configure recording exports in XCALLY, whether that is your team or the partner who manages your XCALLY instance. * A Rulebase API key for the upload requests, which you can create in [Settings > Connections](https://app.rulebase.co/settings/connections). * You must have the Administrator role in Rulebase. ## Installation ### Step 1: ask Rulebase to set up XCALLY Contact your Rulebase account team. Setup goes faster if you already know: * Which XCALLY instance and which queues or call types should be sent to Rulebase * Whether recordings will be sent one at a time, or batched as archives * How your agents are identified in XCALLY, and whether that identifier is the same email address used on their Rulebase employee record * The timezone your XCALLY timestamps are recorded in Rulebase confirms when the connection is active. Nothing arrives until recordings start being sent. ### Step 2: send recordings Recordings go to the conversation upload API with the source `xcally`, either as a direct upload of a single recording or through the presigned URL workflow for archives. [Uploading conversations](/guides/uploads/conversations) has the endpoints, the metadata schema, and the archive layout. Each recording needs metadata alongside the audio: a unique call ID, the call type (`inbound`, `outbound`, or `dialer`), the agent's email, the calling and called numbers, and when the call was recorded. Recorded timestamps are read in your organization's timezone, so a mismatch there shifts every call in your reporting. ## What each recording becomes One recording becomes one conversation in Rulebase, holding a single call: * The call carries its direction, the customer's phone number, and the audio, which Rulebase transcribes for scoring and search. GSM recordings are converted before transcription. * The agent's email is matched to a Rulebase employee, which attributes the call for QA. An email that does not match an existing employee creates a new one. * The customer's number becomes a Rulebase customer. On inbound calls that is the caller; on outbound and dialer calls it is the number dialed. * Handling time is taken from the length of the audio, and the conversation is recorded as resolved at the time of the call. ## Confirm it is working Open **Settings** > **Integrations** and find **XCALLY**. A connection appears with **Last synced on** and a timestamp once Rulebase starts receiving uploads. For more detail, open the **...** menu on that connection and choose **View activity**, which lists the upload requests Rulebase has received, newest first, and can be filtered by date. To check specific calls, list your uploads through the API and read each upload's status. See [Confirm received uploads](/guides/uploads/confirm-received-uploads). # Zendesk Source: https://docs.rulebase.co/guides/integrations/zendesk Connect Zendesk with your subdomain and admin credentials so Rulebase syncs tickets and receives ticket updates through a webhook. Connect Zendesk to Rulebase to evaluate tickets and keep ticket activity available in Rulebase. ## Prerequisites * You must be a Zendesk admin to complete the installation. * You must have the Administrator role in Rulebase. ## Installation ### Step 1: connect Zendesk 1. Log in to your [Rulebase dashboard](https://app.rulebase.co) 2. Go to **Settings** > **Integrations** 3. Find **Zendesk** in the list and click **Add** 4. Enter your Zendesk subdomain, for example `my-company` for `my-company.zendesk.com` 5. Sign in with your Zendesk admin credentials 6. Authorize Rulebase to access your Zendesk account ## What gets synced Once connected, Rulebase starts syncing Zendesk tickets. Rulebase also creates the Zendesk webhook it needs to receive ticket updates. ## Next steps * New and updated tickets are available in Rulebase after sync. * If you use the Zendesk app, see [Zendesk App](/guides/apps/zendesk). * To check when Zendesk last synced, read its activity log, or disconnect it later, see [Manage integrations](/guides/integrations/manage-integrations). # Manage the knowledge library Source: https://docs.rulebase.co/guides/knowledge/manage-knowledge-library Write, import, organize, and refresh the knowledge base documents Rulebase searches when it evaluates a ticket against your policy. The knowledge library is where your written policy lives inside Rulebase: refund windows, verification steps, escalation paths, the tone guide nobody can ever find in Notion. Scorecard checks point at these documents, and when AI QA judges whether an agent gave the right answer, this is what it reads. ![Knowledge library list with seven documents and their sources: Rulebase, Web, Zendesk, Scribe, and Notion](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/knowledge-library-list.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=4689c9c3bd9fb61148e17ae532f5b5b2) ## Before you start * The library lives at [Knowledge > Library](https://app.rulebase.co/knowledge/library). * You need a role that can manage the knowledge base: `Administrator`, `Manager`, or `Team Leader` all include it. * Importing from Notion, Zendesk, Scribe, Confluence, or Intercom needs that tool connected first under [Settings > Connections](https://app.rulebase.co/settings/connections). Connecting alone does not move any content. Google Drive is different: share a folder or file with the Rulebase service account, then import the URL from Knowledge — there is no Connect step. ## Where documents come from Every row carries a **Source**, and it tells you who owns the content and whether Rulebase will overwrite it later: * **Rulebase** — written here, by you. Nothing else touches it. * **Web** — scraped from a link you supplied, such as a public help centre. * **Zendesk** — help centre articles, carrying their section, brand, and locale. * **Notion** — one document per shared page. See [Notion](/guides/integrations/notion). * **Scribe** — guides and knowledge pages, including step screenshots. See [Scribe](/guides/integrations/scribe). * **Confluence** and **Intercom** — pages and help centre articles from those tools. Confluence rides on the [Jira](/guides/integrations/jira) connection rather than having one of its own, so if the Confluence option is missing it is Jira you need to connect. * **Google Drive** — Docs, Sheets, PDFs, and other files under a shared folder or file URL. See [Google Drive](/guides/integrations/google-drive). Workflows can also read Drive at run time without importing into the library. ## Write a document in Rulebase 1. Click **New** and choose **Start from scratch**. Rulebase creates the document and opens the editor. 2. Give it a title that matches how someone would search for it: `Refund and cancellation policy`, not `Policy v3 final`. 3. Write the body. Headings matter more than they look: the evaluator retrieves passages, not whole files, so a document broken into `Refund windows`, `Cancellations`, and `What to tell the customer` returns better answers than one long block. 4. Click **Save**. Rulebase indexes the document for search shortly after you save. A document with a title and an empty body is never retrieved, so a stub you meant to fill in later is invisible to every check that points at it. ## Edit or replace what a document says Open a document from the list, click **Edit**, change the title or the body, and **Save**. **Cancel** discards. ![Knowledge base document open in read view with the Back and Edit controls above the content](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/knowledge-library-document.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=969592a6c7b7b6542cfd1495cc04c49f) > [!NOTE] > > Editing an imported document is temporary. The next sync from Notion, Zendesk, > Scribe, Confluence, Intercom, or Google Drive overwrites the title and body with > whatever the source tool says. Fix imported content in the tool it came from; > use Rulebase documents for policy Rulebase owns. ## Import from a connected tool Imports run from the **Manage knowledge base** dialog, which is also where you see everything currently feeding the library. 1. Click **New** > **Connect to a source**, or the connections icon next to the library search. 2. Open **Import** and choose the tool. For Notion, Zendesk, Scribe, Confluence, and Intercom, only tools you have connected appear — a missing option means the integration, not the import. **Google Drive** is always available: share the folder or file with the Rulebase service account, then paste its URL. 3. Follow the tool's dialog: usually selecting the connected account and clicking **Import**. For Google Drive, paste the shared folder or file URL and click **Import**. ![Manage knowledge base dialog with one synced source and the Import menu open on Web, Zendesk, Notion, and Scribe](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/knowledge-library-import-menu.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=649421a8273d4bac0d38b8b3cdd49fb0) Importing creates a source, starts a first sync, and drops the documents into the library as they arrive. **Web** is the odd one out: it asks for a link rather than an account, and Rulebase crawls from there. ![Import from the web dialog with a Link field and the Import button](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/knowledge-library-import-web-dialog.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=11b6912f8c4c8ad726da8086121663c4) To import a website directly, choose **New** > **Import from web**, paste its link, and click **Import**. ## Find things in a large library Search sits behind the magnifying glass next to the document count and matches on title. The filter icon beside it narrows the list four ways: | Filter | What it narrows by | | - | - | | **Source** | Where the document came from — Rulebase, Web, Zendesk, Notion, and so on | | **Section** | The help centre section an imported Zendesk article belongs to | | **Language** | The locale of a Zendesk article, for multilingual help centres | | **Brand** | The Zendesk brand the article belongs to | ![Knowledge library filter menu open with Source, Section, Language, and Brand](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/knowledge-library-filter-menu.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=6fff50266b8d0b5d98ce96b59a6918f8) Sections, languages, and brands come from Zendesk rather than being set in Rulebase, so those three filters are empty until a Zendesk help centre has been imported. ## Tidy up in bulk Select documents with the checkbox on the left of each row and a small action bar appears at the bottom of the list: * **Add tag** applies one tag to everything selected. Tags become part of the text Rulebase searches, so a tag like `refunds` on three scattered documents makes all three easier for the evaluator to find. They are not a column or a filter. * **Duplicate** copies a document, for when a new policy starts life as a variant of an old one. It is only available with a single document selected. * **Delete file** asks for confirmation, then permanently removes what you have selected and clears those document references from scorecard checks. ## Keep imported content fresh Sources refresh on their own once a day. Open **Manage knowledge base** to see the state of each one: the row shows `Last synced on` with a date, or `Syncing documents...` while a sync is running. The overflow menu on a source row gives you the two manual controls. **Sync** runs a refresh immediately, which is what you want after a batch of edits in Notion that should not wait for tomorrow. **Delete source** stops the refreshes; the documents it already brought in stay in the library, frozen at their last synced version. Refreshes update documents in place rather than duplicating them, skip anything the source tool says has not changed, and remove content that has disappeared from the source (an unshared Notion page, an archived Zendesk article) from the library. ## Related * [Link knowledge to scorecard checks](/guides/quality-assurance/link-knowledge-to-checks) * [Notion](/guides/integrations/notion) * [Scribe](/guides/integrations/scribe) * [Zendesk](/guides/integrations/zendesk) * [Google Drive](/guides/integrations/google-drive) * [Roles and permissions](/guides/roles-and-permissions) # MCP Source: https://docs.rulebase.co/guides/mcp Connect Claude Code, Claude Desktop, or Cursor to the Rulebase MCP server using the endpoint for your data region and OAuth sign-in. Rulebase exposes an MCP server for AI clients that support remote MCP. Use it to let tools like Claude Code, Claude Desktop, and Cursor read Rulebase data after you sign in to Rulebase. If you want to ask questions of the same data without connecting an outside client, [Instruct](/guides/instruct/use-instruct) is the chat built into Rulebase. *** ## Endpoint Use the endpoint for your Rulebase workspace's data region: | Data region | MCP endpoint | | - | - | | United States | `https://mcp.rulebase.co/mcp` | | European Union | `https://eu.mcp.rulebase.co/mcp` | Include the `/mcp` path. The base hostname alone is not an MCP server endpoint. The configuration examples below use the United States endpoint. For an EU workspace, replace it with `https://eu.mcp.rulebase.co/mcp`. *** ## Authentication Rulebase uses OAuth for MCP access. When your MCP client connects, it opens a browser sign-in flow. Sign in with your Rulebase account and approve access. Access is tied to your Rulebase organization. MCP tools only return data for that organization. *** ## Configure Claude Code Add the remote server to your Claude Code MCP configuration: ```json theme={null} { "mcpServers": { "rulebase": { "type": "http", "url": "https://mcp.rulebase.co/mcp" } } } ``` Or from the CLI: ```bash theme={null} claude mcp add --transport http rulebase https://mcp.rulebase.co/mcp ``` When Claude Code connects, it asks you to sign in and authorize access. Use `/mcp` to authenticate if needed. *** ## Configure Claude Desktop Claude Desktop uses **custom connectors** for remote MCP servers. 1. Open **Settings → Connectors** (or [claude.ai/customize/connectors](https://claude.ai/customize/connectors)). 2. Click **Add custom connector** → **Web**. 3. Enter the MCP endpoint for your workspace's data region. 4. Click **Add**, then **Connect** and sign in to Rulebase. 5. In a chat, use **+ → Connectors** and enable Rulebase. Rulebase works as a remote HTTP connector. *** ## Configure Cursor Add the remote server in Cursor MCP settings: ```json theme={null} { "mcpServers": { "rulebase": { "type": "http", "url": "https://mcp.rulebase.co/mcp" } } } ``` Cursor asks you to sign in when it needs access. *** # Add scorecard additional instructions Source: https://docs.rulebase.co/guides/quality-assurance/add-scorecard-additional-instructions Attach exceptions and clarifying guidance to a single scorecard check so AI QA scores it the way your team would. Most scorecard checks are a single sentence, and most of the time that's enough. The trouble starts with the exceptions your reviewers hold in their heads: the apology check that shouldn't fail when the customer apologized first, the verification step that doesn't apply to a returning caller. Additional instructions are where you write those exceptions down so AI QA applies them the same way every time. ## How instructions differ from the other check controls | Control | What it does | Use it when | | - | - | - | | Check description | The main statement the evaluator scores against | You're defining the behavior itself | | **Apply to** conditions | Limits which tickets the check runs on at all | The check is irrelevant to a whole class of tickets | | **Additional instructions** | Extra guidance applied while scoring the check | The check should run, but with a caveat or an exception | Conditions decide whether the check runs; additional instructions shape how it's judged when it does. If you find yourself writing "ignore this check when…", that's usually a condition, not an instruction. ## Before you start You need permission to manage evaluation scorecards, and you need a scorecard that already has a section, criterion, and at least one check; instructions attach to an existing check. Have the exception worded before you open the editor; it's easier to write clear guidance away from the form. ## Add instructions to a check 1. Go to [Knowledge > Scorecards](https://app.rulebase.co/knowledge/scorecards) and open the scorecard. 2. Navigate to the section and criterion holding the check you want to adjust. 3. On the check card, click **Add additional instructions**. A text area labeled **Additional instructions** opens beneath the check. 4. Type the guidance, for example `If the customer already apologized, do not fail for missing apology.` 5. Click **Save as draft** to keep working, or **Publish** to make the change live for future evaluations. ![Scorecard check card showing the Apply to condition, the check description, and an open Additional instructions field containing an exception rule](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-additional-instructions.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=b1a1fd806b736f8b1265561a540390b4) Instructions are scoped to the one check you added them to. A sibling check in the same criterion is unaffected. That lets you carve out a narrow exception without loosening the rest of the criterion. ## Writing instructions the evaluator can act on Instructions work best when they read like a rule a reviewer could follow, not a general principle: * **Name the trigger and the outcome.** "If the customer already apologized, do not fail for missing apology" gives both. "Be lenient about apologies" gives neither. * **Keep one exception per instruction.** Stacking three unrelated caveats into one block makes it hard to tell which one fired when you review a score later. * **Point at what's visible in the ticket.** The evaluator works from the conversation, so an instruction that depends on information outside the transcript won't be applied consistently. > [!TIP] > > After publishing, spot-check a handful of evaluations where the exception should > have applied. If the check still fails on those tickets, the instruction is > probably too abstract. Rewrite it as a concrete if/then and republish. ## Removing instructions When an exception no longer reflects your policy, click **Remove additional instructions** on the same check. This clears the text rather than hiding it, so copy anything you might want back before removing it. As with adding, the change takes effect for future evaluations once you publish. Evaluations that already ran keep the scores they were given. Instructions change how new evaluations are judged, not historical results. If you need old tickets rescored under the new guidance, re-evaluate them. ## Related * [Give feedback on AI QA evaluations](/guides/quality-assurance/give-feedback-on-ai-qa) * [Perform a manual evaluation](/guides/quality-assurance/perform-a-manual-evaluation) * [Create a routing rule](/guides/quality-assurance/create-routing-rules) # Configure evaluation eligibility Source: https://docs.rulebase.co/guides/quality-assurance/configure-evaluation-eligibility Decide which tickets automatic AI QA runs on using coverage rules, eligibility instructions, and the eligibility tester before you go live. Rulebase does not evaluate every ticket that closes. It runs gates first: is this a ticket you asked to cover, was there a real conversation to judge, has enough time passed. Eligibility settings control those gates. Set them wrong and you burn evaluations on "thanks!" tickets or score nothing. ![Eligibility settings card showing a coverage rule for source channel in email and chat at 20% of tickets, eligibility instructions, a 30-day eligibility window, and the linked tickets toggle](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/eligibility-settings-card.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=f2dde67e28effa44c5a96953a12b52c9) ## Before you start * Everything here lives under [Settings > Evaluations](https://app.rulebase.co/settings/evaluations), in the **Configuration** part of the settings sidebar. * You need an admin role. Without it the controls render but stay disabled. * Publish at least one scorecard first. Eligibility decides which tickets are considered; a scorecard decides whether any rubric applies. See [Create a scorecard](/guides/quality-assurance/create-a-scorecard). ## What each setting does The **Eligibility** card holds: * **Coverage rules**: how much of your ticket volume to sample, optionally narrowed to a channel, brand, team, or group. With no rules, every eligible ticket is evaluated. * **Eligibility instructions**: plain-language rules Rulebase applies to the conversation itself, to filter out tickets with nothing worth scoring. * **Eligibility window**: an age limit. Tickets that started before the window are left alone, which stops a backlog import from flooding QA. * **Wait for all linked tickets to close**: holds evaluation until every linked ticket in the thread is resolved, for teams that split one issue across several tickets. Below that, **Timing** holds the **Evaluation delay**, which decides how long after close Rulebase starts. ## Set your coverage rules Covering 100% from day one produces more evaluations than most teams can review. Start low, check volume after a week, and raise it. 1. Next to **Coverage rules**, click **Add rule**. 2. Under **Applies to**, pick what the rule keys on: **Source channel**, **Brand**, **Team**, or **Group**, then choose the values. 3. Add a second condition with **Add condition** if the rule should require both. 4. Under **Target**, choose how much to cover. 5. Click **Create rule**. ![Add coverage rule dialog with a source channel condition, a percentage target of 20%, and an estimate of about 12 tickets per month](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/eligibility-add-coverage-rule-dialog.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=ecfe58894a7f69afa05f642b3985400f) A target is either a percentage or a fixed count: | Target | What it means | Use it when | | - | - | - | | **Percentage** | A share of matching tickets, shown as an estimate of tickets per month based on the last three months | You want coverage to scale with volume | | **Fixed count** | A set number **per day**, **per week**, or **per month**, optionally split **per brand**, **per group**, **per team**, **per agent**, or **per scorecard** | You have a fixed review capacity, or you need every agent covered evenly | Rules stack: a ticket is sampled if any rule selects it. Click an existing rule to reopen it as **Edit sampling rule**, where you can change the target or delete the rule. > [!NOTE] > > With no coverage rules, every ticket that passes the other gates is evaluated, and > there is no sampling limit. That is a reasonable start for a low-volume team and > expensive for everybody else. ## Write your eligibility instructions Coverage rules work on ticket metadata; eligibility instructions work on the conversation, keeping AI QA off tickets where nobody would score anything: a customer who typed "hello?" and vanished, spam, a bot handoff with no human reply. Every organization starts with a default set, so this is usually an edit. Click **Edit** next to **Eligibility instructions** to see them. ![Edit eligibility instructions dialog showing the default rules requiring a real customer request and a meaningful two-way exchange with a human agent](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/eligibility-instructions-dialog.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=f1a8372949c3b1379a19cf7101943b17) The default asks for two things: the customer expressed a real request, and a human agent had a two-way exchange about it. Write additions in the same shape, as rules about the conversation. "Skip tickets where the only agent message is an automated acknowledgement" belongs here; "Only evaluate chat" belongs in a coverage rule. Click **Save changes** when done. The new wording applies to every ticket Rulebase considers from that point on. Tickets already skipped on an AI eligibility verdict are not re-checked, so request an evaluation by hand on any you still want scored. > [!NOTE] > > Eligibility instructions only gate **automatic** QA. Someone can still request an > evaluation by hand on a ticket the instructions filtered out, and that request > also bypasses your coverage rules. ## Test a ticket before you save The eligibility tester runs the same gates Rulebase applies when a ticket closes, against a real ticket. It uses whatever instructions are currently in the editor, including unsaved edits. Test a wording change here before committing it. 1. In the **Edit eligibility instructions** dialog, click **Test**. 2. Search by ticket ID or subject, then pick a ticket. 3. Read the result above the editor. A pass names the ticket, says what will happen, and shows each agent it would evaluate as a badge. ![Eligibility tester result showing ticket 48219 as eligible for QA with the summary that all agents on this ticket will be evaluated and a badge for the agent](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/eligibility-tester-eligible.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=a0ed6619e7bcc9899835161b163550ee) A failure names the first gate that blocked it, following the order Rulebase applies them in, so the top line is the thing to fix. ![Eligibility tester result showing ticket 48220 as not eligible for QA because no eligible agents were found on the conversation](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/eligibility-tester-not-eligible.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=f018c89bfbf3978d745591e36cca097a) When a result is eligible but automatic QA would not have run, the tester says so and lists the automatic-only blockers under **Auto QA**. Badges are grouped as **Rules**, **Auto QA**, and **Agents**; hovering one shows the full reason. > [!TIP] > > Test two tickets every time you change these settings: one you are sure should be > evaluated, and one you are sure should not. ## Delay the start, or limit how far back you go The **Evaluation delay** under **Timing** waits a set period after a ticket closes before evaluating. It exists because tickets reopen: an evaluation that ran at close would score an unfinished conversation. Click **Add delay**, then set the value and choose **minutes** or **hours**. ![Evaluation delay setting enabled with a 30-minute delay after close](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/eligibility-evaluation-delay.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=45d5c2e2a60ab249fba48da8c90537ff) The **Eligibility window** measures how old a ticket is. Set it to 30 days and Rulebase ignores tickets that started earlier, which is what you want the first time you connect a help desk carrying years of history. Click **Add window** to enable it, or clear it to evaluate regardless of age. ## Confirm your setup before go-live Check these in order: * A coverage rule exists and its estimate matches your review capacity. * The eligibility tester returns **Eligible for QA** for a ticket you would expect QA on, with the agents you expect listed as badges. * The same tester returns **Not eligible for QA** for a throwaway ticket. * The **Evaluation delay** is long enough that reopened tickets settle first, and short enough that reviewers are not waiting a day for scores. If a ticket you expected still comes back with nothing, [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) walks the remaining causes in the order Rulebase applies them. ## Related * [Create a scorecard](/guides/quality-assurance/create-a-scorecard) * [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) * [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster) * [Give feedback on AI QA](/guides/quality-assurance/give-feedback-on-ai-qa) * [How AI QA works](/guides/concepts/how-ai-qa-works) # Create a scorecard Source: https://docs.rulebase.co/guides/quality-assurance/create-a-scorecard Build a scorecard from sections, criteria, and checks, balance it to 100 points, scope it to the right tickets, and publish it so AI QA starts scoring. A scorecard is the rubric Rulebase scores tickets against: sections (resolution, accuracy, tone), the criteria under each, and the checks an evaluator makes before awarding points. Once published, AI QA scores every matching ticket against it, and human reviewers use the same structure. ![Scorecard editor for a published scorecard showing section navigation, a criterion with pass and fail scores, and the score preview totaling 100](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-editor-overview.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=21947b24ce6bab0646779bda80a9b815) ## Before you start * Scorecards live under [Knowledge > Scorecards](https://app.rulebase.co/knowledge/scorecards). * You need a role that can manage scorecards, such as `Administrator` or `Manager`. If **New** is greyed out, your role does not include it. * Decide your point weighting first. Top-level totals must add up to exactly 100. * Nothing is evaluated until you publish. A draft is invisible to AI QA and to reviewers. ## How a scorecard is put together A scorecard contains sections, sections contain criteria, and criteria contain checks: * **Scorecard**: the whole rubric, with a name, a description, and an optional scope. * **Section**: a themed group such as `Resolution`. Its points are the sum of its criteria; you never type a section score directly. * **Criteria**: the thing being judged, and the level that carries points. Each has score options: **Pass**, **Fail**, and optionally **Partial** or **Auto-fail**. * **Check**: the instruction an evaluator follows inside a criterion, plus the conditions that map an outcome onto a score. A criterion can hold several checks. Rulebase sums the top-level scores and refuses to publish anything other than 100\. Think of it as a budget you split between sections. ## Create the scorecard 1. Go to [Knowledge > Scorecards](https://app.rulebase.co/knowledge/scorecards). 2. Click **New**. 3. Give it a **Scorecard name** a reviewer would recognize, and use **Description** to record what it covers. 4. Click **Create**. ![New scorecard dialog with a scorecard name and a description describing which tickets it scores](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-new-scorecard-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=398da8eb9b489df3d4c0876cfcb9921f) Rulebase creates the scorecard as a draft and opens the editor. From the empty editor you can **Upload** an existing scorecard document (`.pdf`, `.csv`, `.xls`, or `.xlsx`) and let Rulebase draft the structure from it, or build it yourself with **New section**. ![Empty scorecard editor with the upload panel, the New section button, and an empty score preview](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-empty-editor.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=1f096ce76b0573e767844b8da99e8f45) ## Add your sections Add all your sections before writing any criteria, so the point budget is visible from the start. 1. Click **New section**. Rulebase adds it to the left-hand list with a placeholder name like `Section 1`. 2. Hover the new section, open its **...** menu, and choose **Edit**. 3. Set the **Name**, and use **Instructions** to tell the evaluator how to think about this theme. 4. Click **Save**. ![Edit section dialog with the name Resolution and instructions describing how to judge whether the customer left with their problem solved](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-section-details-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=a78da566ec8edb250106a63718ce8263) Section instructions are guidance for the whole section. They carry no points and apply to every criterion, so put a judgement call you would otherwise repeat in each check here. ## Write criteria and checks Open a section, then click **Create first criteria** (or **Add criteria** once the section has one) and fill in the card. 1. Type the **Criteria name** as the thing being judged, phrased so passing is good: `Resolved the customer's issue`. 2. Set the points. New criteria start at 10, so click the value next to **Pass**, set the **Score**, and press **Enter**. **Fail** stays at 0. 3. Add score options if pass/fail is too blunt. The **+** next to the scores offers **Partial score** for partial credit and **Auto-fail score**, which fails the whole evaluation when selected. 4. In **1st check**, describe what the evaluator has to look for. 5. Give the check its conditions. **Generate conditions** drafts them from the criterion and check text; **Add condition manually** lets you write your own. A condition pairs an observation with an outcome, reading `If ` … `mark as fail`, choosing from the criterion's non-passing scores. 6. Use **Add new check** when the criterion needs a second judgement, and **Add additional instructions** for guidance that belongs to this check alone. ![Criterion card showing the criteria name, pass and fail scores, a check description, a criteria condition mapping to fail, and the Generate conditions and Add new check controls](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-criteria-check-editor.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=b1e6acf54dac470bd14f0c72f2c592ce) The **Apply to** pill on a check reads **All tickets** until you narrow it. A check can be restricted to specific channels, to new or re-assigned tickets, to primary or duplicate tickets, to SLA breaches, or to inbound or outbound calls. > [!NOTE] > > Every check needs at least one condition before the scorecard can publish. If a > check has none, publishing stops with a message naming the criterion and the > check number. ## Balance the scores to 100 **Score preview** on the right keeps a running total. Section totals roll up from their criteria, and the **Total** at the bottom must read 100. ![Score preview panel listing three sections with their criteria scores and a total of 100](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-score-preview.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=1995d41bd53e99bb1aad5ac4cbfc7f60) If the total is off, publishing fails with `Total score of all top-level criteria must equal 100` and the current number. Weight sections by what you want agents to optimize for. ## Scope it to the right tickets A scorecard with no scope applies to every ticket Rulebase evaluates. That is fine for one company-wide rubric and wrong the moment you have two, so narrow at least one dimension. Click **Add scoping** in the header and pick from four groups: * **Applies to**: **Tickets**, **Work items**, or both. Both by default. * **People**: **Groups**, **Teams**, or **Agents**, for rubrics that only apply to part of the org. * **Ticket details**: **Channels** for email, chat, or calls, and **Ticket scope** for **External tickets** or **Internal tickets**. * **Calls**: **Call direction**, for inbound versus outbound. ![Add scoping dropdown showing the Applies to, People, Ticket details, and Calls groups](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-add-scoping-dropdown.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=74183fe55a44c993a3dffe358db451ee) Each choice becomes a pill next to the button; the **x** on a pill removes that part of the scope. ![Scorecard header scoping row with the Add scoping button and Emails and Chats pills applied](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-scoping-pills.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=b92615ab5aa8c8a98d9638a3a0f06f46) > [!TIP] > > Adding **Internal tickets** to a scorecard's ticket scope lets Rulebase evaluate > tickets where the customer never replied. Without a scorecard opted in to > internal tickets, those tickets are skipped, because there is no customer side to > judge. ## Publish it, or keep it as a draft **Publish** validates first: a missing condition or off-balance total surfaces as a message. Otherwise it makes the scorecard live and confirms with `Scorecard published successfully`, and from then on matching tickets are scored against it. **Save as draft** keeps everything you have written but leaves the scorecard out of evaluation. It works on a published scorecard too, to pull one out of circulation while you rework it. ![Save as draft confirmation dialog explaining that the scorecard will not be published and will remain in draft status](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-save-as-draft-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=be25ad12ba2e1523ef0e61ce648980aa) On the scorecards list, **Active** and **Drafts** are separate groups, and each card's overflow menu carries the state changes: **Publish** on a draft, **Move to drafts** on a published one, plus **Duplicate** and **Archive**. ![Scorecards list with one published scorecard under Active and one draft under Drafts](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-list-active-and-drafts.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=a03ddd7adf33f0a8172ac4d8465d7bf7) > [!TIP] > > To change a live rubric, **Duplicate** it, edit the copy, then publish the copy > and move the original to drafts. Editing in place changes what evaluators see > mid-cycle and makes before-and-after scores hard to compare. ## Confirm it before the first evaluation * **Score preview** reads exactly 100, and the section weights match what your team agreed. * Every check has at least one condition, and between them the conditions cover each way the criterion can be marked down. * The scoping pills describe the tickets you meant. An empty scoping row means every ticket. * The scorecard sits under **Active**. A draft sits under **Drafts** and never scores. Then prove it end to end. Open the eligibility tester in [Settings > Evaluations](https://app.rulebase.co/settings/evaluations), test a ticket this scorecard should cover, and confirm the result reports no scorecard problem. ## Related * [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility) * [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) * [Add scorecard additional instructions](/guides/quality-assurance/add-scorecard-additional-instructions) * [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster) * [How AI QA works](/guides/concepts/how-ai-qa-works) # Create and manage queues Source: https://docs.rulebase.co/guides/quality-assurance/create-and-manage-queues Build a queue of reviewers, understand how Rulebase picks who gets each ticket, and keep membership current as people move around. A queue is a named group of people that routing rules can hand work to. Instead of a rule naming Sam and Alex, it names `QA review`, and the queue decides which of its members gets the ticket. When Alex moves teams you edit one queue, not every rule that mentioned them. Queues are the destination most teams want for shared work: evaluations waiting on a reviewer, contested evaluations waiting on a decision, complaints that need a manager. ![Queues tab in routing settings listing four queues with their member counts](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-queues-list.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=e46cfa3cc1ddd461ad774def080a71b9) ## Before you start * You need access to manage organization settings. * Queues live at [Settings > Routing > Queues](https://app.rulebase.co/settings/routing/queues), on the tab beside **Rules**. * The people you want in the queue need to be members of your organization already. You add existing members to a queue; you cannot invite from here. ## Create a queue A queue is only a name at creation time. Members come next. 1. Open [Settings > Routing > Queues](https://app.rulebase.co/settings/routing/queues). 2. Click **Create queue**. 3. Name it after the work, not the people: `Evaluation validation` and `Complaints escalation` stay accurate when the roster changes. 4. Click **Create queue**. ![Create queue dialog with a name field filled in and the Create queue button](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-create-queue-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=a7971036141ce426b0bc6e2a03046786) ## Add and remove members Open a queue from the list to manage who is in it. The **Members** table is the queue: everyone listed is in the rotation, and nobody else is. ![QA review queue detail page with a Members table listing three members](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-queue-members.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=6a137752ef2ec5779d1fa2e9b5487ae3) 1. Click the queue name in the list. 2. Click **Add a member** and pick someone from the search list. 3. Repeat for each person. They appear in the table straight away. ![Add a member popover open over the queue members table showing the searchable list of organization members](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-queue-add-member.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=ccff6395841d5b67dc7fa5b131f4c3b0) To take someone out, open the row menu at the right of their name and choose **Remove**. It takes effect immediately and does not touch tickets already assigned to them, so check their open work before you remove someone who is leaving. ![Queue members table with a row menu open showing the Remove option](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-queue-member-menu.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=ca43500ea02280d37cbab95dbe2db993) > [!NOTE] > > The member picker lists everyone in your organization, including people already > in the queue. Adding a duplicate is harmless: Rulebase tells you the member is > already in the queue and nothing changes. ## How Rulebase picks who gets the ticket When a rule runs **Assign to queue**, two things happen: the ticket is stamped with the queue and marked unresolved, then the queue picks exactly one person and assigns the ticket to them. Nobody else in the queue is notified. Each queue has a **Distribution** setting: * **Even** (default) — rotate among eligible members so work spreads evenly. * **Custom** — set a percentage per member (must sum to 100). Rulebase still prefers members currently in working hours, then applies those weights among whoever is eligible. The pick is a rotation. Before applying Even or Custom weights, Rulebase narrows to members whose local time is inside working hours (8am to 5pm where they are) and only falls back to the rest of the queue when nobody is currently in hours. A queue that spans time zones therefore hands overnight work to whoever is awake. Tickets and back-office work items that route to the same queue share one rotation counter. If the ticket already has an assignee and is still unresolved, the assignment is skipped and the ticket stays with the person who has it. Rulebase does not take live work away from someone mid-review. ## Queue, role, or member | Action | Who it can pick | How it picks | Best for | | - | - | - | - | | **Assign to queue** | The queue's members | The queue's **Distribution** setting (even rotation or custom weights), preferring people in working hours | Shared review work with a stable rota | | **Assign to role** | Everyone holding that role | One at random | Broad coverage where the exact person doesn't matter | | **Assign to member** | The members you tick in the rule | One at random | A named pair or trio, defined per rule | The queue is the only one of the three that spreads work deliberately, and the only one you can change without editing a rule. Roles are the widest net: the pool is everyone with that role today, not the people who had it when you wrote the rule. ## Point a rule at the queue A queue with no rule pointing at it never receives anything. Once the members are in, create or edit a rule so its **Then** section reads **Assign to queue** and names this queue. The full walkthrough is in [Create a routing rule](/guides/quality-assurance/create-routing-rules). ## Rename or delete a queue The queue name has a pencil and a bin beside it on the detail page. Renaming is safe: rules point at the queue itself, so a rule reading **Assign to Contest review** follows the new name without being touched. Deleting is not reversible, and it leaves any rule that pointed at the queue with nowhere to send work. Those rules keep matching events, and keep stopping the rules below them, while assigning nobody. Before you delete a queue, check [Settings > Routing > Rules](https://app.rulebase.co/settings/routing/rules) for rows whose action names it, and repoint or remove them first. ## Related * [Create a routing rule](/guides/quality-assurance/create-routing-rules) * [Manage and reorder routing rules](/guides/quality-assurance/manage-routing-rules) * [Perform a manual evaluation](/guides/quality-assurance/perform-a-manual-evaluation) * [Invite members](/guides/access-management/invite-members) # Create a routing rule Source: https://docs.rulebase.co/guides/quality-assurance/create-routing-rules Assign tickets to a queue, role, or member automatically when an evaluation, approval, or complaint event fires. A routing rule watches for one event and does something when it happens. When AI QA finishes scoring a ticket and the score comes back low, the rule can drop that ticket into your review queue before anyone goes looking for it. When a trainer submits a manual evaluation for validation, the rule can hand it to a reviewer. Every rule reads as a sentence: **When** this event happens (optionally narrowed by conditions), **Then** do these things. Conditions are optional; actions are not. A rule with no action cannot be saved. ![Create rule dialog with a When section set to the Ticket is evaluated trigger narrowed by a QA score includes Low condition, and a Then section with an Assign to queue action pointing at the QA review queue](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/create-routing-rule-dialog.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=9a3996c0a16b109233ea9df81e4013f4) ## Before you start * You need access to manage organization settings. * Decide the trigger first. It determines which conditions are even available to you, so picking it changes what the rest of the dialog offers. * If several people will share the routed work, create the destination queue first under [Settings > Routing > Queues](https://app.rulebase.co/settings/routing/queues). Routing to a queue is usually better than naming individuals, because the queue survives people changing teams. ## Create a rule 1. Go to [Settings > Routing > Rules](https://app.rulebase.co/settings/routing/rules). 2. Click **Create rule**. 3. In **When**, pick the trigger event from the dropdown. 4. Optionally click **Add condition** and choose a field to narrow which tickets match. Add as many as you need; a ticket must satisfy all of them. 5. In **Then**, click **Add action** and choose what should happen. 6. Click **Create rule**. The rule takes effect on the next matching event. It does not reach back and route tickets that already went through the trigger before you saved. ## Choose the trigger A rule starts from one of these events: * **Ticket is evaluated** — AI QA has finished scoring a ticket. This is the general-purpose trigger for QA follow-up. * **Evaluation approval is requested** — someone submitted a manual evaluation that needs validation. Pair this with a review queue. * **Ticket changes status** — the ticket moved to a new status in your helpdesk. * **Evaluation is contested** — an agent disputed an evaluation result. * **Complaint is created** — a complaint record was raised. ![Create rule dialog with the Evaluation approval is requested trigger selected and an Assign to queue action sending the evaluation to a validation queue](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/create-routing-rule-approval-trigger.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=05d4658d3b4dfe501c40ce08a0b5e75c) ## Narrow with conditions Conditions filter the tickets a trigger applies to. The available fields depend on the trigger you chose, so the **Add condition** menu shows a different list for **Ticket is evaluated** than it does for **Evaluation is contested**. Across the five triggers the fields on offer are **Workspace**, **QA score**, **Agent QA score**, **Failed criteria**, **Group**, **Partner**, **Channel**, **Ticket status**, **Number of disputes by an agent**, **Contested evaluation source**, and **Complaint risk**. Switching the trigger after you have added conditions drops any that the new trigger does not support. Most conditions use an **includes** operator and let you tick several values at once. **QA score** offers Autozero, Low, Medium, and High, so a single condition can cover "Autozero or Low" and catch both tiers of poor result. Two fields are single-value comparisons instead: **Number of disputes by an agent** takes a number, and **Contested evaluation source** is either **AI** or **Manual**. > [!NOTE] > > A rule with no conditions matches every occurrence of its trigger. That's a > legitimate way to build a catch-all, but it should sit below your specific rules, > not above them. ## Pick the actions A rule runs every action you attach to it, so one rule can assign a ticket and send a notification at the same time. | Action | What Rulebase does | | - | - | | **Assign to queue** | Puts the ticket in the queue, which then assigns it to one of its members | | **Assign to role** | Picks one member holding that role, at random, and assigns the ticket to them | | **Assign to member** | Picks one of the members you selected, at random, and assigns the ticket to them | | **Resolve** | Marks the ticket resolved with an optional comment, if it is currently unresolved | | **Send a notification** | Sends an email to the addresses you list, or posts to a Slack or Teams channel | On **Assign to role** and **Assign to member**, selecting five members does not notify all five: Rulebase spreads the work across them one ticket at a time. If you want everyone to see the ticket, add a notification action alongside the assignment. A notification action is per-rule. Your organization's standing recipients and channels (the weekly customer issues digest, and the channel that gets complaint alerts) are set in [Configure notifications](/guides/settings/configure-notifications). The two overlap in one place. When a **Complaint is created** rule matches and sends a notification, it takes over for that complaint so you get one message rather than two. If no rule matches, Rulebase sends the alert to your default complaint channel instead. ## Worked example: route evaluations to validators One common reason teams build a routing rule is manual evaluation validation. 1. Create a queue named `Evaluation validation` under [Settings > Routing > Queues](https://app.rulebase.co/settings/routing/queues) and add your reviewers to it. 2. Create a rule with **When** set to **Evaluation approval is requested**. 3. Leave conditions empty if every submitted evaluation should be reviewed, or add a **Failed criteria** or **QA score** condition to narrow the route. 4. In **Then**, add **Assign to queue** and select the review queue. Submitters now use **Submit for review** and the ticket lands with a validator without anyone forwarding it. The submitter's manual evaluation already counts; the validator can publish corrections as the overriding evaluation. ## Order your rules and confirm they fire Rules have an explicit order on the Rules page and Rulebase evaluates them in that order, stopping at the first rule that matches, so put narrow rules above broad fallbacks. A catch-all sitting at the top will swallow the tickets your specific rules were meant to catch. One catch-all is already there. Most organizations start with a rule showing **Any** under Conditions and `-` under Actions, and although it looks inert it still claims low-scoring tickets and stops the rules beneath it from running. Drag your new rule above it. [Manage and reorder routing rules](/guides/quality-assurance/manage-routing-rules) covers ordering as a skill, and [Why didn't my routing rule fire?](/guides/troubleshooting/why-didnt-my-routing-rule-fire) has the full diagnosis when a rule that looks correct never runs. Once the rule is saved, confirm it works before relying on it: * Trigger the event on a real ticket and check that the ticket lands where you expect. * Confirm the assignee is someone you intended. If a rule assigns to a role, the pool is everyone with that role, which may be wider than you assumed. * Check that a ticket you expected to be excluded was in fact skipped. This is where a missing condition usually shows up. * If nothing happened, re-check the trigger. Matching the wrong event is more common than a bad condition. ## Related * [Manage and reorder routing rules](/guides/quality-assurance/manage-routing-rules) * [Create and manage queues](/guides/quality-assurance/create-and-manage-queues) * [Perform a manual evaluation](/guides/quality-assurance/perform-a-manual-evaluation) * [Configure notifications](/guides/settings/configure-notifications) * [Set SLA targets and business hours](/guides/settings/set-sla-targets-and-business-hours) * [Add scorecard additional instructions](/guides/quality-assurance/add-scorecard-additional-instructions) * [Invite members](/guides/access-management/invite-members) * [Roles and permissions](/guides/roles-and-permissions) # Edit scorecard structure Source: https://docs.rulebase.co/guides/quality-assurance/edit-scorecard-structure Reorder sections and criteria, add or remove checks, retarget Apply to, and rebalance points on a scorecard that is already scoring tickets. Scorecards change. A calibration reweights tone, a new policy needs its own criterion, or a check written for email keeps firing on chats. The safest way to change a live rubric is to edit a copy. ![Scorecard editor showing the section list, a criterion with pass and fail scores, a check with conditions, and the score preview totaling 100](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-editor-overview.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=21947b24ce6bab0646779bda80a9b815) ## Before you start * Scorecards live under [Knowledge > Scorecards](https://app.rulebase.co/knowledge/scorecards). * You need a role that can manage scorecards, such as `Administrator` or `Manager`. * Know whether the scorecard is **Active** or a **Draft**. A draft is invisible to AI QA and to reviewers, and safe to restructure freely. The next section is about the active case. * New to the four levels (scorecard, section, criteria, check)? The [glossary](/guides/concepts/glossary#scorecards-sections-criteria-and-checks) defines them, and [Create a scorecard](/guides/quality-assurance/create-a-scorecard) builds one from scratch. ## Work on a copy of a live scorecard Evaluations store the score a criterion was given, but read its *wording* live. Reword a check and the new sentence appears next to every past evaluation of that criterion. The numbers do not move, but a reviewer revisiting a disputed score sees text that was never applied to that ticket. Duplicating avoids this: the original keeps scoring, past evaluations keep their original wording, and you decide when the new version takes over. 1. Go to [Knowledge > Scorecards](https://app.rulebase.co/knowledge/scorecards). 2. Open the scorecard's overflow menu and choose **Duplicate**. 3. Rulebase opens a **Duplicate scorecard** dialog pre-filled with ` (copy)` and the original description. Name it something a reviewer can tell apart, such as `Support ticket quality - Q3`. 4. Click **Create**. The copy opens as a draft with every section, criterion, check, condition, and scoping pill carried over. The copied structure is not stored until you save it, so use **Save as draft** before closing the tab if you are not finishing in one sitting. 5. Make your changes, then **Publish** the copy. 6. Back on the list, use **Move to drafts** on the original so only one version is live. ![Scorecard card overflow menu open on a published scorecard with Move to drafts, Duplicate, and Archive](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/knowledge-scorecard-duplicate-menu.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=3c39f912ba9eb3efe24e616e21b0c17b) > [!NOTE] > > Small in-place edits are fine when nothing about the judgement changes: fixing a > typo, tightening a sentence that means the same thing, or adjusting a scorecard > nobody has been evaluated against yet. Anything that changes what a criterion is > asking for belongs on a copy. ## Reorder sections and criteria Order affects presentation only, but it is the sequence reviewers read when they score by hand. Reorder in **Score preview** on the right. Rows in the section list stay fixed. Hover a row, and a drag handle appears on its left; drag it to its new position. Dragging a section row moves the whole section with its criteria. ![Score preview with a drag handle revealed on a hovered criterion row, sections totaling 100](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/knowledge-scorecard-reorder-handle.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=9b8c3020be0831873475da484db5844c) Dragging works within one level: you can reorder sections against each other and criteria inside their own section, but not drag a criterion from `Resolution` into `Accuracy`. To move one across, recreate it in the target section and delete the original. Deleting also removes its history of wording. Checks reorder the same way, using the **Drag to reorder** handle on the check row, inactive when a criterion has only one check. ## Rename, edit, or remove a section The section list on the left is where a section's own details live. * **Rename it inline.** Double-click the section name, type, and press `Enter`. `Escape` cancels. * **Edit the name and instructions together.** Hover the section, open its overflow menu, and choose **Edit**. Section instructions apply to every criterion underneath. * **Delete it.** The same menu has **Delete**. It removes the section and every criterion inside it after you confirm, and their points leave the total. The option is hidden on the last remaining section. ![Section list with the overflow menu open on a section, showing Edit and Delete](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/knowledge-scorecard-section-menu.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=e010b62f270e9797d3d559af35b90fe6) Deleting has no undo short of leaving the editor without saving. If a section might come back, publish a copy without it instead. ## Add and remove checks Checks are the level you will touch most, because most rubric drift is a check that has stopped matching how your team judges tickets. 1. Open the section, then the criterion. 2. Click **Add new check** at the bottom of the criterion card. Rulebase numbers checks in order (`1st check`, `2nd check`) and drops the cursor into the new description. 3. Write what the evaluator has to look for, then give the check at least one entry under **Criteria conditions**, using **Generate conditions** or **Add condition manually**. 4. To remove a check, click the trash icon on its row. **Add new check** is disabled while any check in the criterion is still empty, and **Delete check** is disabled when the criterion is down to its last check. Publishing checks one more thing: a check with no **Criteria conditions** blocks it, naming the criterion and the check number. ## Retarget what a check applies to The **Apply to** pill decides whether a check runs on a ticket at all. It reads **All tickets** until you narrow it. Narrowing it is usually the fix when reviewers complain a check keeps failing on tickets it was never written for. Click the **+** next to the pill and pick a group: **Channel**, **Ticket hierarchy**, **Ticket state**, or **SLA breaches**. Each choice becomes its own pill, and the **x** on a pill removes that condition. ![Check row with the Apply to pill set to All tickets and the add-condition menu open on Channel, Ticket hierarchy, Ticket state, and SLA breaches](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/knowledge-scorecard-apply-to-menu.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=53d0f308708ff96b207c163e06df44fe) Conditions decide whether a check runs. If the check should still run but be judged more leniently in some cases, use [additional instructions](/guides/quality-assurance/add-scorecard-additional-instructions) instead. Scoping a whole criterion works differently: the **Criteria options** menu on the criterion header has **Scope**, where clearing **AI evaluations** takes the criterion out of AI scoring while leaving it in place for human reviewers. **Manual review** cannot be turned off. ## Rebalance the points Points live on criteria, and the top-level total must come back to exactly 100 before Rulebase will publish. **Score preview** keeps the running total and turns red the moment you are off. * **Change a criterion's value.** Click the number next to **Pass**, set it in the **Edit score** popover, and press `Enter`. Section totals recalculate from their criteria. * **Add partial credit.** The **+** in the **Scores** row offers **Partial score**, which starts at 5 and is editable like Pass, as long as it stays below the criterion's Pass value. A criterion can carry more than one partial option. * **Add an auto-fail.** The same menu offers **Auto-fail score**: selecting it on one criterion zeroes the entire evaluation, taking down every other criterion's points with it. One per criterion. Reserve it for a compliance breach that should void a ticket. * **Remove either of them.** Partial and auto-fail options carry an **x**. **Pass** and **Fail** do not, and `Fail` stays at 0. ![Scores row for a criterion with Pass 25 and Fail 0, and the add-score menu open on Partial score and Auto-fail score](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/knowledge-scorecard-score-options-menu.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=c4c1ba96e8b775c99cdd3c505de6cce6) Outside the points arithmetic, the **Add** button beside **Criteria name** offers **Score cap**, a rule reading `If this criteria is not pass, cap overall score at N`. It caps the whole scorecard's total and leaves the points on other criteria alone. Use it for a failure that should hold a ticket's total down no matter how well everything else went. One cap per criterion. > [!TIP] > > Rebalance in pairs. When you take 5 points off tone, put them somewhere in the > same edit, and the total never leaves 100 long enough to lose track. ## Publish, and what changes for whom **Publish** validates first (the total, and a condition on every check), then makes the version live. From that moment, tickets that match the scorecard's scope are evaluated against the new structure. Evaluations that already ran keep their scores; new evaluations use the new structure. To judge older tickets by the new rubric, re-evaluate them, because publishing does not reach backwards. **Save as draft** on a published scorecard keeps everything you have written and takes the scorecard out of evaluation. Use it when you started restructuring in place and would rather finish somewhere safer. ## Check before you publish * **Score preview** reads 100, and the section weights match what was agreed. * Every check still has at least one entry under **Criteria conditions**, including the ones you edited. * The **Apply to** pills say what you meant. A check you narrowed to `Chats` stops running on email entirely. * If this is a duplicate, the original is under **Drafts** and only one version is **Active**. Two live scorecards covering the same tickets both evaluate. ## Related * [Create a scorecard](/guides/quality-assurance/create-a-scorecard) * [Link knowledge to scorecard checks](/guides/quality-assurance/link-knowledge-to-checks) * [Add scorecard additional instructions](/guides/quality-assurance/add-scorecard-additional-instructions) * [Rulebase glossary](/guides/concepts/glossary) * [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility) # Export QA evaluation data Source: https://docs.rulebase.co/guides/quality-assurance/export-qa-evaluations Export filtered QA evaluations with separate criterion scores and written feedback. Export filtered QA evaluations to an Excel workbook from the Evaluations list. You can choose the columns, preview matching rows, and keep criterion scores separate from written feedback. ## Export evaluations 1. Go to **Evaluations**. 2. Apply the date, scorecard, agent, team, or other evaluation filters you need. 3. Click **Export**. 4. Choose the columns to include. 5. Review the export preview. 6. Click **Export**. The export uses the filters currently applied to the Evaluations list. Your column selection becomes the default the next time the export dialog opens. This is a one-off download. When someone needs the same workbook on a repeating basis, or your analysts want QA data in a warehouse, create a schedule instead. See [Schedule recurring exports](/guides/insights/schedule-exports). ## Export criterion scores and feedback Use the two criterion options for different kinds of analysis: | Option | Exported value | | - | - | | **Criteria scores** | A score column for each criterion in the evaluation's scorecard | | **Criteria feedback** | A separate feedback column for each criterion in the evaluation's scorecard | Select both options when you need numeric results and the explanation behind each result. Select only **Criteria feedback** when the workbook is for qualitative review. Feedback is exported as readable text. Message references become labels such as `[message 1]`, knowledge references use the document title when available, and timestamps use the workspace timezone. ![Export evaluations with Criteria scores and Criteria feedback selected separately](https://mintcdn.com/rulebase/LmDNkLxqrTRcrtJn/images/evaluation-export-columns.png?fit=max\&auto=format\&n=LmDNkLxqrTRcrtJn\&q=85\&s=a0cbe293c6f3d607817b6e908f9f6314) ## Interpret the workbook The workbook groups criterion columns by scorecard. An evaluation appears under the scorecard used to produce its result. Empty or unavailable values appear as `NA`. Keep the export filters with the workbook when sharing it so reviewers know which evaluations were included. # Filter by team, partner, brand, and channel Source: https://docs.rulebase.co/guides/quality-assurance/filter-interactions What each filter dimension selects, which ones come from your agent roster versus your help desk, and how to narrow a QA list without excluding tickets you meant to include. Several filter dimensions look interchangeable and are not. Team and Group both sound like "the agent's team". Source and Workspace both sound like "the integration". Partner sits under a heading called Ticket while describing a property of the agent. This page defines what each one selects. [Find tickets and evaluations](/guides/quality-assurance/find-tickets-and-evaluations) covers applying and saving filters. ## Where the filters live Both QA lists put filters in the same place: the pill row at the top of the list, with a **+** at the end and a filter icon in the toolbar. The menu that opens has a search box at the top, so typing `brand` is faster than hunting through submenus. ![The interactions filter menu with the People submenu open, showing Agents, Team, Group, Customer, and Partner](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/reviewer-interactions-filter-people.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=35f4e5474a312d18cffd883576d60bfc) Filters combine the way you would hope: two values inside one dimension are an either/or, and two different dimensions both have to be true. Selecting the Billing and Onboarding teams gives you both teams' tickets; adding the Chat channel narrows that to chats handled by either team. ## Team, Partner, Brand, and Channel ### Team A Rulebase concept. You build teams on the agent roster and assign agents to them, so a ticket matches a team filter when **at least one agent who worked it belongs to that team**. A ticket handled by two people from different teams therefore matches both. The catch is that the link between a ticket and a team is only recorded for agents marked eligible for QA evaluation. If someone touched the ticket but is not on the roster as evaluable, their team will not pull that ticket into a team filter. ### Partner Also a Rulebase concept, and also carried by the agent rather than the ticket: each agent belongs to at most one partner, and a ticket matches when one of its agents does. This is the dimension for outsourced or BPO work, and the way to compare an outsourcer against your in-house team: a partner is a property of who did the work, not of where the ticket came from. Unlike Team, the partner link is recorded for every agent on the ticket regardless of QA eligibility. ### Brand A property of the ticket, and specific to Zendesk. **Brand** only appears in the filter menu when your organization has a Zendesk connection, and it selects the Zendesk brand the ticket arrived under. Use it when one help desk serves several customer-facing brands and the standards or scorecards differ between them. ### Channel The medium the conversation happened on: **Call**, **Email**, or **Chat**. A conversation can carry more than one (a chat that turned into a call has both), and the filter matches on any of them. Selecting Call returns every conversation that contained a call, not only the ones that were purely calls. > [!NOTE] > > **Group** means different things on the two lists. On the Tickets list it means > the group the ticket sits in on your help desk, a Zendesk or Intercom group > synced across with the ticket. On the Evaluations list, the same label means the group of the > agent being scored. Both trace back to your help desk rather than to Rulebase, but > one is a property of the ticket and the other a property of the person, so the same > filter answers a different question on each list. Team, by contrast, is always a > Rulebase concept, and it lines up with Group only if you set it up to. ## Source and Workspace **Source** is the integration type: Zendesk, Intercom, Freshdesk. Filter by Source when you want everything from one product. **Workspace** is a specific connection. It only appears when your organization has more than one customer interaction integration connected, and it is what you need when you have two Zendesk accounts and want one of them. It also offers **No workspace**, for interactions that are not tied to a connection at all. ## Filtering the Evaluations list The Evaluations list has its own filter menu, organized around the fact that a row here is one agent's score rather than one ticket. ![The Evaluations filter menu grouped into agent filters, Ticket, Evaluation, and Date](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/reviewer-evaluations-filter-menu.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=3760dfae05e166281719019ce45377be) **Agent**, **Team**, and **Group** sit at the top because they describe the person being scored. The **Ticket** group holds **Channel type**, **Partner**, **Brand**, **Assignee**, and **Review status**. Partner is filed here despite being an agent property. The **Evaluation** group is the part with no equivalent on the Tickets list: * **Evaluator** narrows to a named reviewer, or to AI. **Evaluation source** is the blunter version of the same idea, **AI** or **Human**. * **Score** and **Revision score** filter on the number, with the revision score being the value after a contest changed it. * **Status** is **Approved** or **Pending**, and it is about the approval state of the evaluation rather than the review state of the ticket. The three date filters differ: **Evaluation date** is when the score was produced, **Revision date** is when it was last changed, and **Ticket started date** is when the customer first made contact. Reporting on last month's QA usually means Evaluation date, not Ticket started date. > [!TIP] > > Filter combinations you rebuild every week belong in a saved view. Set the filters > up once and click **Save**, and everyone in your organization is looking at the > same slice. Otherwise two people comparing numbers are usually comparing two > different date ranges. ## When a filter returns less than you expect Common causes: * **The date filter you did not set.** Both lists arrive with a date range already applied, and on Tickets that range is on the *evaluated* date rather than when the ticket came in. A ticket that arrived in the window but was scored outside it will not be there. * **A ticket that was never evaluated.** The built-in Tickets view only shows tickets that have a QA evaluation, so an unevaluated one is invisible no matter what else you select. [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) explains the reasons. * **Team where you meant Partner, or Group where you meant Team.** Re-read the definitions above; a wrong-but-valid filter returns a plausible-looking list rather than an error. ## Related * [Find tickets and evaluations](/guides/quality-assurance/find-tickets-and-evaluations) * [Create and manage teams](/guides/access-management/create-and-manage-teams), for how the Team dimension gets populated * [Manage the agent roster](/guides/quality-assurance/manage-agent-roster), for agent eligibility and partner assignment * [Export QA evaluation data](/guides/quality-assurance/export-qa-evaluations), for taking a filtered list out of Rulebase * [Rulebase glossary](/guides/concepts/glossary) # Find tickets and evaluations Source: https://docs.rulebase.co/guides/quality-assurance/find-tickets-and-evaluations Get to the reviews waiting on you, move between the ticket list and the evaluation list, and save the filtered views you open every morning. Rulebase keeps QA results in two lists: **Tickets** is one row per conversation, **Evaluations** is one row per agent score. ![The Tickets view grouped into Low, Medium, and High score sections, each row showing a QA score, the agent, and the ticket summary](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/reviewer-tickets-list.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=7f9cbef26a7981a6626f697a14f5a230) ## Before you start * Both lists show completed evaluations, so an organization that has not scored anything yet sees empty sections. [How AI QA works](/guides/concepts/how-ai-qa-works) covers what has to be true before a ticket gets a score. * Every organization starts with a **Tickets** view in the sidebar under **Quality assurance**. An **Evaluations** view is one you add yourself; see below. * Nothing on this page changes a score, so it is safe to click around while you learn the filters. ## Start with what is assigned to you Before you go looking through a list, check whether the work has already been handed to you. When a routing rule or a colleague assigns you a ticket, it lands on **Home**, and the number beside **Home** in the sidebar is the count of tasks assigned to you. The **Workbench** tab shows an **Assigned to you** card next to the composer, with any calibration batches first and your assigned tickets below them. Click a row to open it, or click **Begin reviews** to work the whole set as a queue rather than one at a time. ![Home Workbench with an Assigned to you card listing three assigned tickets and a Begin reviews link](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/reviewer-home-assigned.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=5e772ad3997e68b796917309be3ad542) > [!TIP] > > The count next to **Home** also includes calibration batches you have been added > to, not just individual tickets. If the number is higher than the ticket list > explains, an evaluation batch is waiting for you. ## Tickets and Evaluations answer different questions Both lists group by score band and both show a coloured score chip. A ticket handled by three agents produces one row in **Tickets** and three rows in **Evaluations**. | | Tickets | Evaluations | | - | - | - | | One row is | a ticket | one agent's score on one ticket | | The score shown | the ticket's QA score | that agent's score | | Default date range | evaluated in the last 7 days | evaluated in the last 30 days | | Reach for it when | you are working a review queue conversation by conversation | you are comparing agents, or checking one evaluator's output | | Clicking a row | opens the ticket's QA workspace | opens the same workspace with that evaluation selected | The two lists ship with different default date ranges, so neither contains the other. A ticket scored three weeks ago is in **Evaluations** and not in **Tickets** until you widen the date filter. > [!NOTE] > > **Low scores** is the red section at the top, because a low score is the > high-risk one. Sections are ordered by how much attention the tickets need, not > by score. ## Add an Evaluations view Views are the saved lists in your sidebar, and you add one from the section header rather than from a settings page. 1. Hover the **Quality assurance** heading in the sidebar and click the **+** (**New view**). 2. Pick **Evaluations** in the **New view** dialog. 3. Rulebase creates the view and opens it in edit mode with a suggested name, so renaming it is the first thing you can do. ![The New view dialog offering Tickets, Evaluations, Complaints, Interactions, and Evaluation contests](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/reviewer-new-view-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=c86c3b6759bb2a3026044dff5b5cadd4) The same dialog is where the other list types come from. **Complaints** narrows to interactions Rulebase flagged as complaints, **Interactions** drops the QA filter and shows everything that arrived from your help desk, and **Evaluation contests** collects the scores agents have disputed (see [Work contested tickets](/guides/quality-assurance/work-contested-tickets)). A **Complaints** view lists tickets, not complaint cases. The casework (one record per complaint, with its category tags, timeline, and status) lives in Compliance instead, and [Work a complaint case](/guides/compliance/work-a-complaint-case) is the guide for it. Every view is shared with your whole organization and appears under **Quality assurance** in the sidebar. The **Options** menu (**⋯**) in the view's toolbar holds **Edit**, **Duplicate**, and **Delete**. ![An Evaluations view listing agent scores grouped by score band, with an Evaluated Last 30 days filter pill](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/reviewer-evaluations-view.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=d512f0a8190181cb72d60a3adab9c2ef) ## Narrow a list without losing it Filters live in two places on a view, and they do the same thing. The pills along the top row show what is currently applied; the **+** at the end of that row and the filter icon in the toolbar both open the filter menu. Add a filter and the list updates immediately, but nothing is saved yet. Changing filters on a view is a temporary exploration. As soon as your filters differ from the ones the view was saved with, **Clear** and **Save** appear at the right-hand end of the pill row: **Save** makes the change permanent, **Clear** puts the view back to its saved filters. Do neither and the view reverts on its own the next time you open it. The **×** on an individual pill only removes that one filter. If a filter combination is one you want permanently and the view belongs to someone else, duplicate it from the **⋯** menu first and change the copy. [Filter by team, partner, brand, and channel](/guides/quality-assurance/filter-interactions) covers what each filter dimension selects, including the ones that only appear for certain integrations. ## Change how the list reads The sliders icon in the toolbar opens **Displays**, which controls two things independent of any filter. **Grouping** decides the section headings. On Tickets you can group by **Risk level** (the default Low / Medium / High sections) or by **Review status**; on Evaluations the choices are **Score level** or **Type**, where Type splits AI reviews from manual ones. Both offer **No grouping**, which gives you one flat list and is usually what you want once a filter has already narrowed things down. **List options** controls what appears on each row. Turn off the ones your team does not use and the summary text gets more room. Display choices save to the view on their own, unlike filters. Search sits beside them in the same toolbar and matches the ticket number, the subject, and the summary Rulebase generated, not the message text inside the conversation. Search for a phrase a customer used and you will usually come up empty; search for the ticket number and you land on it immediately. Once a list is narrowed the way you want it, **Export** in the top right sends the same rows out as a file. [Export QA evaluation data](/guides/quality-assurance/export-qa-evaluations) covers what the columns contain. ## Related * [Open a ticket's QA workspace](/guides/quality-assurance/ticket-qa-workspace) * [Filter by team, partner, brand, and channel](/guides/quality-assurance/filter-interactions) * [Perform a manual evaluation](/guides/quality-assurance/perform-a-manual-evaluation) * [Rulebase glossary](/guides/concepts/glossary), for the difference between tickets, interactions, and work items * [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated), when a ticket you expected is missing from both lists # Give feedback on AI QA evaluations Source: https://docs.rulebase.co/guides/quality-assurance/give-feedback-on-ai-qa Thumbs up or thumbs down a single AI-scored criterion, explain what the AI got wrong, and have it re-score that criterion. When an AI evaluation gets one criterion wrong, you usually don't want to redo the whole scorecard. Criterion feedback is the small correction: mark a single criterion right or wrong, say why in a sentence or two, and optionally have Rulebase score that one criterion again with your note in hand. It's also how the AI learns your team's standards, because a thumbs down becomes a written correction that later evaluations of similar tickets can draw on. ![AI QA scorecard results with thumbs up and thumbs down feedback icons next to each criterion's score](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/qa-feedback-criterion-thumbs.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=ca44b9486462c66e5f3e9f6c873e746d) ## Before you start * You need permission to give evaluation feedback and train the model. Without it the thumbs icons don't appear at all. * Open a ticket that already has a completed AI QA evaluation. Feedback attaches to a criterion result, so there has to be a result to attach to. * Know what specifically went wrong before you click. Thumbs down won't submit without a note. ## Leave feedback on a criterion Feedback attaches to one criterion at a time. There's no control for rating a whole scorecard at once, so pick out the criterion that was scored wrong. 1. Open a ticket with an AI QA evaluation and find the criterion in the review. 2. Click the thumbs up or thumbs down icon next to its score. 3. In the **Leave feedback** dialog, write your note. 4. On thumbs down, tick **Re-evaluate this criterion after submitting** if you want Rulebase to score it again now. 5. Click **Submit**. ![Leave feedback dialog with the How can we improve this evaluation prompt, a note, and Re-evaluate this criterion after submitting selected](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/qa-feedback-leave-feedback-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=4ccd53d6f28ab5b136cb05d7702393cf) The dialog changes with the direction of your feedback. Thumbs down asks **How can we improve this evaluation?** and blocks submission until you write something. Thumbs up asks **Any additional comments? (optional)** and accepts an empty box, so a bare thumbs up is a fine way to record that a result looks right. ## Thumbs up and thumbs down do different jobs The two directions look symmetrical in the UI, but only one of them changes anything: | Behaviour | Thumbs up | Thumbs down | | - | - | - | | Note required | No | Yes | | Can trigger a re-score | No | Yes, via the checkbox | | Feeds later evaluations | No | Yes | | Use it for | Confirming a result you agree with | Correcting a result that's wrong | Thumbs up records that a human checked this criterion and agreed. It leaves the score alone and has no effect on how future tickets are scored. Everything else on this page describes thumbs down. > [!TIP] > > Write the note as guidance rather than a verdict. "Wrong" gives the AI nothing to > act on; "the customer asked twice about the refund window and the agent only > answered the second time, which should fail this criterion" gives it the > situation, the evidence, and the expected outcome. Name the policy or the specific thing the agent should have said; that is what > the note gets matched on later. ## Re-evaluate a criterion Ticking **Re-evaluate this criterion after submitting** asks Rulebase to score that one criterion again, this time with your note, the original reasoning, and the previous score as context. No other criterion is re-scored, though the evaluation's overall score updates to account for the new result. The result arrives shortly after you submit rather than instantly. You'll see the criterion move through **Applying feedback...** and then settle on **Feedback applied** once the new score is in. Reach for it when the AI missed context you can point out in a sentence or two. Skip it when the criterion is a judgement call you'd rather record than overturn: the note is still saved and still informs later evaluations without changing this ticket's score. > [!NOTE] > > If a re-evaluation fails after Rulebase has exhausted its retries, your feedback > is removed along with it so you can submit again. If a note you left disappears, > that's why. Re-submitting it is the fix. You don't have to decide at submission time. Feedback and re-evaluation are separate steps, so you can leave the note now and trigger the re-score later from the badge described below. ## What you see after submitting Once feedback exists on a criterion, the thumbs icons are replaced by a badge reading **Feedback submitted**, or **Feedback applied** if a re-evaluation completed. That badge is the entry point for everything you might want to do afterwards. Click it to open a popover showing the note, who wrote it, and when, along with three actions: * **Re-evaluate criterion** — run the re-score now, for feedback you submitted without ticking the checkbox. Thumbs down only. * **Edit feedback** — reopen the dialog and revise the note. Editing regenerates the guidance Rulebase derived from it, so a correction here propagates. * **Delete feedback** — remove the note entirely, including from the guidance used by later evaluations. The criterion returns to plain thumbs icons. Agents never see any of this. On their own view of an evaluation the feedback controls and badges are hidden, so a note you leave is not a message to the agent. If the agent needs to hear it, that belongs in a [coaching session](/guides/coaching/run-a-session). ## How feedback shapes later evaluations When you thumbs down an AI QA criterion, Rulebase expands your note into a written correction that records the situation on the ticket, what the AI originally concluded, your feedback in your own words, and concrete guidance for scoring a similar case. That correction is stored in your organization's feedback library, scoped to the scorecard and criterion it came from, and later evaluations retrieve it when they hit a comparable situation. It's private to your organization. Specific notes are worth more than general ones: retrieval works by matching situations, and a note with no situation in it has nothing to match on. Breadth helps too: correcting the same criterion across different tickets covers more ground than repeating one correction on near-duplicate tickets. ## Verify your feedback landed Before moving on to the next ticket, confirm: * The criterion shows a **Feedback submitted** or **Feedback applied** badge rather than plain thumbs icons. * If you asked for a re-evaluation, the criterion's score reflects the new result and the badge reads **Feedback applied**. A badge still reading **Applying feedback...** after a while means the job hasn't finished. * Opening the badge shows your note as you wrote it, with your name against it. * You left the feedback on the criterion you meant to. Feedback sits on one criterion, and moving it means deleting and re-adding. ## Related * [Perform a manual evaluation](/guides/quality-assurance/perform-a-manual-evaluation), for a full manual override * [Add scorecard additional instructions](/guides/quality-assurance/add-scorecard-additional-instructions), for changing how a criterion is scored everywhere rather than on one ticket * [Run a coaching session](/guides/coaching/run-a-session) * [Roles and permissions](/guides/roles-and-permissions) * [How AI QA works](/guides/concepts/how-ai-qa-works), for where criterion feedback sits in the evaluation pipeline # Link knowledge to scorecard checks Source: https://docs.rulebase.co/guides/quality-assurance/link-knowledge-to-checks Mention knowledge base documents inside a scorecard check so AI QA judges the ticket against your written policy and cites what it used. A check like `Gave correct policy information` is only as good as the policy the evaluator has in front of it. Left on its own, AI QA judges correctness from the conversation and its own general knowledge, which is fine for tone and hopeless for your refund window. Mentioning a knowledge base document inside the check tells the evaluator where the answer is written down, and the evaluation comes back citing the passage it used. ## Before you start * You need a role that can manage scorecards, such as `Administrator` or `Manager`. * The policy has to be in the knowledge library already. If it is not, [manage the knowledge library](/guides/knowledge/manage-knowledge-library) covers writing it in Rulebase or importing it from Notion, Zendesk, Scribe, Confluence, or Intercom. * Mentions live in **check descriptions**, so the criterion needs at least one check. Criteria names are plain text and cannot hold a mention. * Change a live scorecard on a copy; see [Edit scorecard structure](/guides/quality-assurance/edit-scorecard-structure). ## Mention a document in a check 1. Open the scorecard from [Knowledge > Scorecards](https://app.rulebase.co/knowledge/scorecards) and navigate to the section and criterion. 2. Click into the check description and type `@` where the reference belongs. 3. A menu opens, grouped by type. Scroll to **Knowledge base**, or keep typing to search by title. 4. Pick the document. It drops in as a pill carrying the document's title. 5. Finish the sentence around it, then **Publish**, or **Save as draft** if the scorecard is not ready to go live. ![Check description with the @ mention menu open on the Knowledge base group, listing Entire knowledge base, Brand's knowledge base, and individual documents](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/knowledge-check-mention-menu.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=3a4a8e406e99390781125ffa9ca2dae3) The result reads as a normal sentence with the document embedded in it. The mention is a reference the evaluator can follow, not a setting. ![Check description reading "Check that the agent resolved the issue the customer asked about, judged against @Refund and cancellation policy"](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/knowledge-check-mention-inserted.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=99a9350eba87369af67cafded7dd40da) To remove a reference later, put the cursor after the pill and delete it like any other text. ## The three kinds of knowledge reference The **Knowledge base** group offers two catch-all entries above your documents, and they behave differently from naming a document outright: * **Entire knowledge base** — search everything. Right when the check covers a broad standard ("communication guidelines", "escalation playbooks") that is spread across several documents. * **Brand's knowledge base** — narrow to the help centre content belonging to the ticket's brand, for organizations running more than one brand out of one Zendesk. * **A specific document or section** — search only that document, or that Zendesk help centre section. This is the sharpest option and the one to reach for when a check maps onto a single policy page. Typing after the `@` searches documents and sections by title, so on a large library that is faster than scrolling the menu. ## Write the check so the mention has a job A pill dropped into a sentence with no instruction around it tells the evaluator very little. Give the reference a purpose: * **Say what to look up, not just where.** `Check that the agent applied the correct verification level per @User verification policy` beats `@User verification policy`. * **Name one source per judgement.** Three documents in one check invites the evaluator to average them. If the check really spans three policies, it is probably two checks. * **Keep the check's own conditions doing their job.** The mention supplies the facts. The [conditions](/guides/concepts/glossary#scorecards-sections-criteria-and-checks) still decide what a failure looks like, and [additional instructions](/guides/quality-assurance/add-scorecard-additional-instructions) still carry the exceptions. ## What happens during evaluation When AI QA reaches a criterion whose checks reference knowledge, it searches that knowledge before it scores, using a question built from the customer's actual situation rather than the check's wording. Naming a document confines the search to it; **Entire knowledge base** and **Brand's knowledge base** leave the search wide and let the brand and language of the ticket steer it. The passages that came back are attached to the result. Open the evaluation and the criterion carries a **Knowledge base** row of document pills: the documents the score was reasoned from. > [!NOTE] > > A referenced document only helps if it has content. A stub with a title and an > empty body is not searchable, and neither is a document whose import failed. > Open it in the library and read it before you point a check at it. ## Confirm it is working Publish, wait for the next batch of matching tickets to be evaluated, then open one and look at the criterion you changed: * The criterion shows a **Knowledge base** row with at least one document pill. * The pill is the document you named, not something else the search preferred. * The written summary reasons from the policy rather than restating the check. If no citation appears, the usual causes are the mention pointing at an empty document, the check not applying to that ticket because of its **Apply to** conditions, or the scorecard edit never having been published. ## Related * [Manage the knowledge library](/guides/knowledge/manage-knowledge-library) * [Edit scorecard structure](/guides/quality-assurance/edit-scorecard-structure) * [Add scorecard additional instructions](/guides/quality-assurance/add-scorecard-additional-instructions) * [How AI QA works](/guides/concepts/how-ai-qa-works) * [Give feedback on AI QA evaluations](/guides/quality-assurance/give-feedback-on-ai-qa) # Manage the QA agent roster Source: https://docs.rulebase.co/guides/quality-assurance/manage-agent-roster Add new team members to be evaluated, import or export the roster, move agents between teams, and remove people who have left. The agent roster is the list of support reps Rulebase runs QA and performance reports on. Anyone missing from it goes unevaluated; anyone left on it after they leave keeps appearing in team performance views and exports. Teams usually edit it after a hiring round, a reorg, or a QA report that returns an unfamiliar name. ![Settings People Agents page with the agent count, Import and Add buttons, and a table with Agent, Team, and Membership columns](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-roster-page.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=116fe5ad5d8cfdc83f3354b0ab9539d7) > [!NOTE] > > Rulebase uses "agent" for two things. This page is the **QA roster** at > [Settings > People > Agents](https://app.rulebase.co/settings/people/employees): > the human support reps you evaluate. The AI agents you build live under > [Home > Agents](https://app.rulebase.co/home/agents), and nothing here affects > them. The [glossary](/guides/concepts/glossary) defines both terms. ## Before you start * Work from [Settings > People > Agents](https://app.rulebase.co/settings/people/employees). The header reads **Agents**, subtitle "People you run QA and performance reports on." * For any roster-wide change, download the existing roster first and edit that file. It already has the right headers and the identifiers Rulebase matches on. * Being on the roster is not the same as having a Rulebase login. The **Membership** column tracks the login separately, and evaluation does not depend on it. ## What the table tells you * **Agent**: email (the primary identifier) with the name underneath. * **Team**: their team, or `-` if none. Teams drive grouping in QA reports; an agent with no team drops out of team-level views. * **Membership**: whether they can log in. Values are **Not invited**, **Invite pending**, or **Invited**. Invitations are optional; a **Not invited** agent is still evaluated. The agent count sits above the table. The filter icon next to it narrows by **Team** or **Workspace** (where **Sales** and **Customer Service** agents are separated). ## Export the current roster The export lives inside the import dialog. 1. Click **Import**. 2. Click **Download existing agents**. ![Import agents dialog with the Upload CSV drop zone, Download existing agents link, and Remove agents not in sheet checkbox](https://mintcdn.com/rulebase/LmDNkLxqrTRcrtJn/images/agent-import-csv.png?fit=max\&auto=format\&n=LmDNkLxqrTRcrtJn\&q=85\&s=f3a078fcb32e519bf56226ff84bc1b9f) You get a CSV named after your organization and download date, with every current agent. Columns are **Name**, **Email**, **Employee ID**, and **Team**, plus one per custom agent field your organization defines; organizations that evaluate sales calls also get a **Role** column. Edit values, but leave headers alone. Rulebase matches columns by header name, and a renamed header is treated as an unrecognized custom field. ## Import roster changes An import is an upsert: Rulebase matches by email first, then employee ID, updates whoever it finds, and creates the rest. Anyone it touches lands on the active roster, so an import also brings back someone you removed earlier. 1. Click **Import**. 2. Drop your file on **Upload CSV**, or click it and pick the file. 3. Select **Remove agents not in sheet** when this file is the complete roster. Leave it off to only add and update the rows you supply. 4. If you selected it, read the **Preview** that appears. It groups pending changes into **Agents to add**, **Agents to remove**, and **Agents to update**, with the changed fields named next to each updated agent. 5. Click **Import**. ![Import agents dialog after uploading a CSV with Remove agents not in sheet selected, showing the Preview grouped into add, remove, and update](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-roster-import-preview.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=785da56ca3fbdb3c49286e312885b369) What each column does on the way in: | Column | Effect | | - | - | | **Email** | The main match key. A row with an unfamiliar email creates a new agent | | **Employee ID** | Fallback match key when the email has changed. Must be unique within your organization | | **Name** | Updates the display name shown throughout QA and reports | | **Team** | Assigns the agent to that team, creating the team if it does not exist yet | | **Role** | Recognizes `Sales` and `Customer Service`. Other values are ignored | > [!NOTE] > > The **Preview** only appears with **Remove agents not in sheet** selected. A plain > import shows no preview and adds or updates the supplied rows without removing > anyone. Imports are all or nothing. If any row fails validation (a malformed email, or an employee ID already used by someone else), Rulebase rolls the whole file back and lists the failures as `Row 4: …` in the dialog. Fix those rows and upload again. ## Add one or two agents without a CSV For a single new hire, **Add** beats editing a spreadsheet. The **Employees** field searches people already in Rulebase who are not on the roster, including anyone who arrived through a ticket integration but was never added for QA. If nobody matches, type a full email address and the dropdown offers **Add** with that address, creating the agent outright. ![Add agents dialog with an email typed into the Employees field and an Add option offering to create that address as a new agent](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-roster-add-agents-dialog.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=7d7fb59f980f2aacec7d52bc8ebb377b) Select as many people as you need, then click **Add**. To fill in a name, team, or custom field afterwards, click the agent's row to open **Edit agent**. ## Move agents between teams 1. Hover a row and click its checkbox, then select the rest. Hold **Shift** for a range. 2. Click **Change team** in the floating action bar. 3. Search for the destination team and select it. ![Agents table floating action bar showing 3 selected with Change team, Invite, and Remove actions, and the team search popover open](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-roster-change-team.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=f53363158d4e190863e04102cab5e83b) Every selected agent moves at once. An agent belongs to one team at a time, so this replaces their current team; **No team** clears the assignment. The bar also holds **Invite**, which sends a login invitation to selected agents still marked **Not invited**. ## Remove agents from the roster 1. Select the agents to remove. 2. Click **Remove** in the floating action bar. 3. Read the confirmation, then click **Remove agents**. ![Remove selected agents confirmation dialog explaining that historical data stays intact and removed agents will not appear in future reports](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-roster-remove-confirm.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=df19aac1bb9d4cd28760c40d8554453a) Removal takes people off the active roster; nothing is deleted. Past evaluations, scores, and report history stay put. Going forward, they stop being picked up for new QA evaluations and drop out of performance reports. For a single person, the row's overflow menu has **Remove agent**. If someone returns, add them back with **Add** or in your next import; they rejoin with their old history intact. ## Confirm the roster is right After a bulk change: * The agent count matches the number of people you expect to be evaluated. * Nobody sits on `-` in the **Team** column unless you meant them to. * Filtering by each team returns the people their lead would expect. Performance views read from this roster. A team page shows only the agents the roster puts on that team. In [Use team and agent performance](/guides/insights/team-and-agent-performance), a name missing there is almost always a name missing here. If a specific person's evaluations still look wrong, the cause is usually a second record under a different email. Open the overflow menu on their row and use **Merge** to combine the two. ## Related * [Create and manage teams](/guides/access-management/create-and-manage-teams) * [Use team and agent performance](/guides/insights/team-and-agent-performance) * [Export QA evaluation data](/guides/quality-assurance/export-qa-evaluations) * [Invite members](/guides/access-management/invite-members) * [Create and manage roles](/guides/access-management/create-and-manage-roles) # Manage and reorder routing rules Source: https://docs.rulebase.co/guides/quality-assurance/manage-routing-rules Change the order routing rules run in, edit a rule that is routing to the wrong place, and delete one safely. Rules accumulate. Someone sets up evaluation validation, someone else adds a complaints escalation, and six months later nobody is sure which rule handles a low-scoring ticket. The Rules page answers that, but only if you read it as what it is: a priority list, top to bottom, where **only the first matching rule runs**. Ordering is the main thing you manage on this page. ![Routing rules list showing five ordered rules with their conditions and assign-to-queue actions, and a rule at the bottom reading Any with no action](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-rules-list-ordered.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=5b98f813458c89f710cc268c7c20cfde) ## Before you start * You need access to manage organization settings. * Open [Settings > Routing > Rules](https://app.rulebase.co/settings/routing/rules). Everything in this guide happens on that page. * Reordering needs at least two rules. With one rule there is no drag handle. ## Read the list as a running order Each row prints the trigger event, then its conditions in brackets, then what happens. `Ticket is evaluated (QA score includes Autozero, Low)` means the rule wakes up when AI QA finishes scoring and only acts when the score landed in those two bands. `(all conversations)` in the brackets means the rule has no conditions and takes every occurrence of its trigger. When an event fires, Rulebase starts at row one, ignores rules watching a different event, and stops at the first rule whose conditions all hold. The rules below it never see that ticket. Nothing in the interface records the near-miss. ## Move a rule 1. Hover the row you want to move. A drag handle appears at the left edge. 2. Drag the row to its new position. 3. Let go. The new order saves immediately; there is no separate save step. ![Routing rules list with the cursor hovering a rule row, showing the drag handle that appears at the left edge](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-rules-drag-handle.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=204aba7076a9d13c5f6775d2d0812983) > [!NOTE] > > Rules cannot be paused. The row menu offers **Edit** and **Delete**, and there is > no enabled column, so position is the only non-destructive control. Parking > a rule at the bottom of the list is the closest thing to switching it off while > keeping it. ## Decide where each rule belongs Ordering is a judgement call, but a few patterns hold for almost every setup: * **Narrowest conditions at the top.** A rule for `Complaint risk includes High` should sit above one that takes every complaint, or the general rule absorbs the urgent ones. * **Escalations above their base case.** A repeat-dispute rule (`Number of disputes by an agent equals 2`) has to outrank the first-contest rule, otherwise every contest is treated as a first. * **Catch-alls last.** Any rule showing `(all conversations)` is a fallback by definition. It belongs at the end of its trigger's cluster, not the start. * **The rule you didn't create belongs at the bottom.** Most organizations have a rule reading **Any** under Conditions and `-` under Actions. It watches every event, it matches low-scoring tickets, and it still stops the rules underneath it even though it appears to do nothing. If your routing only fails on tickets that scored badly, drag your rules above it and read [Why didn't my routing rule fire?](/guides/troubleshooting/why-didnt-my-routing-rule-fire) for the full diagnosis. Rules for different triggers can sit in any order relative to each other, since a rule only ever competes with rules that can see the same event. Grouping them by trigger anyway makes the list much easier to reason about a year later. ## Edit a rule Click anywhere on a row to open it, or use the row menu and choose **Edit**. The dialog is the one you used to create the rule: the trigger, the conditions, and the actions are all editable. ![Routing rules list with a row menu open showing the Edit and Delete options](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-rule-row-menu.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=c41676a91590e9e00c924ff450756b3a) 1. Open the rule and change what you need. 2. Remove a condition or action with the **×** beside it, or add one with **Add condition** and **Add action**. 3. Click **Save changes**. ![Edit Rule dialog with a Ticket is evaluated trigger, a QA score condition, and an Assign to queue action pointing at the QA review queue](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-rule-edit-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=210e955e758e8bbe7cf0ba34363074c8) Editing does not move the rule. A rule that now has broader conditions may need to move down the list as well, and that is easy to forget in the moment. Changes apply to the next matching event only; tickets that already went through the trigger stay where they are. ## Delete a rule Deleting is permanent and there is no undo, so it is worth a moment of thought about the rules below the one you are removing. They have been sitting in its shadow and will start receiving traffic the moment it goes. 1. Open the row menu and choose **Delete**. 2. Check the rule named in the confirmation against the row you meant to click. 3. Click **Delete**. ![Delete confirmation dialog naming the routing rule by its condition, Complaint risk includes High](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-delete-rule-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=39b2f9cf00342d5fab0d43cd1a7b5b57) Treat that name as a hint rather than proof. The confirmation quotes the rule's first condition, and it labels some condition types incorrectly: a QA score or count condition can come out as the wrong band, or as `None`. A rule with no conditions has nothing to quote at all and is called `Unknown rule`. Whenever the name does not clearly match the row you meant, close the dialog and open the rule to check. > [!TIP] > > If you are removing a rule because it fires too often rather than because it is > wrong, edit its conditions or move it down instead. A deleted rule cannot be recovered, and the next person will rebuild it from > scratch. ## Confirm the new order does what you meant Order changes apply going forward, so the ticket that prompted the change still needs handling by hand. Raise the trigger again on a real ticket and check three things: the ticket lands with the queue or person you expected, a ticket you meant to exclude was left alone, and nothing you moved down has gone quiet entirely. If a rule stopped working after a reorder, the rule now above it is almost always the reason. ## Related * [Create a routing rule](/guides/quality-assurance/create-routing-rules) * [Why didn't my routing rule fire?](/guides/troubleshooting/why-didnt-my-routing-rule-fire) * [Create and manage queues](/guides/quality-assurance/create-and-manage-queues) * [Perform a manual evaluation](/guides/quality-assurance/perform-a-manual-evaluation) # Perform a manual evaluation Source: https://docs.rulebase.co/guides/quality-assurance/perform-a-manual-evaluation Score a ticket by hand against a scorecard: choose where to start from, mark each check, write feedback the agent can act on, and submit. A manual evaluation is your score on a ticket, recorded against the same scorecard the AI uses. Use it when the AI got something wrong, when the ticket was never evaluated, or when the process calls for a human score on a sample regardless. If your role cannot provide model-training feedback, the button at the end says **Submit for review**. Your evaluation still becomes the active manual result immediately, and Rulebase also sends it through your organization's validation workflow. ## Open the editor On the ticket's QA workspace, the review tabs sit at the top. If nobody has scored the ticket by hand yet, there is no **Manual evaluation** tab. Click the **+** beside the tabs and choose **Manual evaluation**, which both adds the tab and opens the editor in one step. ![The plus button beside the AI review tab, open, showing the Manual evaluation option](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/reviewer-manual-evaluation-menu.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=d42d07c9fca923d891d9371666bba6d5) Where a manual evaluation already exists or is awaiting validation, **Manual evaluation** is already a tab and you can go straight to it. The editor replaces the review panel. The header reads **New evaluation by** your name, with **Cancel** on the left and the submit button on the right. The ticket panel stays where it was, so the transcript is still beside you while you score. ![The manual evaluation editor open beside the conversation, showing the header, Evaluation details, the agent tab, and the first criterion](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/reviewer-manual-evaluation-editor.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=eb1d8c3266b5492172f7bfe2b45b7deb) ## Decide what you are starting from Under **Evaluation details** you choose the scorecard, and next to it sits **Prefill scores**, which sets the starting score for every check. ![The Prefill scores menu offering Pre-fill with 1, Pre-fill with AI score, and Start from 0](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/reviewer-prefill-scores-menu.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=f7d924d6a56b715735b918ef16288f05) | Option | What it does | Use it when | | - | - | - | | **Pre-fill with 1** | Sets every criterion to its highest score and marks every check passed | The ticket was handled well and you expect to mark down one or two things | | **Pre-fill with AI score** | Copies the AI's result for this agent, score by score | You mostly agree with the AI and want to correct a specific criterion | | **Start from 0** | Sets every criterion to its lowest score, and fails each check that has a reason available, with that reason preselected | The ticket went badly, or your process requires scoring up from nothing | Rulebase also picks a starting point for you when the editor opens: your own earlier score if you have one, otherwise the AI's, otherwise full marks. > [!NOTE] > > **Pre-fill with 1** does not set anything to 1. It selects the highest score option > on every criterion and marks every check passed, whatever those options are worth > on your scorecard. Read it as "start from a pass". ## Score the criteria Each criterion on the scorecard becomes a section, and how you score it depends on how the scorecard was built. Criteria with checks are scored one check at a time; criteria without checks are scored directly, and where the AI already scored that criterion its result sits beside yours for comparison. [Create a scorecard](/guides/quality-assurance/create-a-scorecard) explains the difference from the author's side. ![A criterion with its checks grouped by outcome, and one check open showing Your score, Feedback option, and Your feedback](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/reviewer-manual-check-card.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=7db16fcb59a7542ad78f7e9778362d39) For a check-based criterion, the checks are grouped by outcome (**Failed**, **Partial**, **Passed**, **Not Applicable**) with a count on each group so you can see the shape of your evaluation without scrolling it. Open a check and you get three fields: * **Your score** — passed, failed, partial, or not applicable. Partial only appears where the scorecard defines a partial option. * **Feedback option** — appears once you mark a check failed or partial. These are the reusable reasons the scorecard author wrote, and choosing one keeps your reasons consistent with everyone else's and countable in reports. * **Your feedback** — free text, and the part the agent reads. Type `@` to reference a document or a specific message in the conversation, and use **Attach** for a screenshot. The score at the top recalculates as you go, so you can watch the criterion cost land as you mark things down. A criterion marked as an auto-fail drops the whole scorecard to zero the moment you select it, and a score cap configured on the scorecard will hold the total below what your individual marks add up to. > [!TIP] > > Write the feedback for the agent, not for the record. "Asked for the email but not > a second identifier before changing the address" tells them what to do next time. > "Verification failed" does not, and it is what the score already says. ## Multi-agent tickets Where more than one agent handled the ticket, each gets their own tab and their own score for their part of the ticket. **Add agent** brings in someone who touched the conversation but was not scored automatically, and the **×** on a tab drops an agent from this evaluation. You do not have to score everyone. Rulebase only requires that at least one agent has been evaluated before you can submit. ## Submit **Submit evaluation** publishes the score. Rulebase saves your evaluation, closes the editor, and the ticket's headline score updates to reflect the human result. The **Manual evaluation** tab now sits permanently beside **AI review**, and opening it shows your scores side by side with the AI's for the same scorecard. If your role can only submit feedback for approval, the button reads **Submit for review** instead and your work waits for a reviewer to publish it. The reviewer cannot publish until every failed or partial check has a feedback option chosen, so filling those in as you go saves a round trip. **Cancel** discards the evaluation, and Rulebase asks first if you have unsaved changes. ## Related * [Open a ticket's QA workspace](/guides/quality-assurance/ticket-qa-workspace) * [Create a scorecard](/guides/quality-assurance/create-a-scorecard), for how the criteria, checks, and feedback options you are choosing between were defined * [Give feedback on AI QA](/guides/quality-assurance/give-feedback-on-ai-qa), when one criterion is wrong and a full manual evaluation is more than you need * [Why is my score different?](/guides/troubleshooting/why-is-my-score-different) # Run a calibration Source: https://docs.rulebase.co/guides/quality-assurance/run-a-calibration Sample a set of tickets, put several reviewers on each one, and read the agreement numbers that come back. Two reviewers can apply the same scorecard to the same ticket and land eight points apart. The gap is a disagreement about what the rubric means, and it stays invisible until two people score the same conversation. A calibration assigns reviewers the same tickets and reports the spread. The output is a set of deltas: how far each reviewer sat from the AI, and how far they sat from each other. Those deltas are the session agenda. > [!NOTE] > > Calibrations are switched on under > [Settings > Features](https://app.rulebase.co/settings/features), in the > **Quality assurance** group. If you cannot find them, that toggle is off, or > quality assurance itself is. Open calibrations from **Home > Tasks** or from a > direct link. ## Before you start * Have a published scorecard, and tickets already evaluated in the period you want to sample. Sampling draws on tickets **evaluated** inside the span you choose, not tickets merely created in it. A quiet fortnight of QA yields a thin sample. * Line up at least two reviewers who are members of the organization. One reviewer is legitimate for spot-checking your AI, but reviewer-to-reviewer agreement needs two. * Decide the question first. "Are we consistent on refunds?" leads to a very different sample from "is our scoring too harsh at the bottom of the range?" ## Build the sample 1. Go to **Home > Tasks** and find the **Calibrations** card. Click the **+** button in its header, or use **New > Calibration** in the tasks list. 2. If your organization evaluates back-office work, you are asked what to calibrate first: **Tickets** or **Work items**. Pick one and continue. When work item QA is off, the question is skipped. 3. Choose how the tickets are chosen. **Randomise** samples from a filtered pool; **Specific tickets** lets you name ticket IDs yourself, which is the right choice when you want everyone arguing about the same known-hard cases. 4. Name it. Leave it blank and you get today's date plus "Calibration", which is fine for a recurring session and useless in a list of twenty. 5. Set the **Due date**. It defaults to a week out and it is what reviewers are chased against. 6. Set the **Ticket span**, the date range the sample is drawn from. 7. Under **Filter tickets by**, narrow the pool by team, channel, or **Agent QA score level**, and by brand if your help desk has brands. Sampling only Low-band tickets asks whether your reviewers agree about failure, which is where rubrics usually diverge. 8. Add your **Reviewers**. Everyone you add gets their own assignments. 9. Choose the sampling method. **Fixed** takes a **Ticket split**, the number of tickets given to each reviewer. **Percentage** takes a share of the eligible pool instead. 10. Click **Create calibration**. ![New calibration dialog with a name, a due date a week out, a ticket span of the last 30 days, a Score level Low filter, three reviewers, Fixed sampling, and a ticket split of five](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/calibration-new-calibration-dialog.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=093cd64bbf09bc5bc2de3451ca5a0cf0) > [!TIP] > > For the deltas to mean anything, reviewers have to score the same tickets. Open > **Advanced settings** and switch on **Require multiple evaluations per ticket**, > then set **Evaluations per ticket** to how many reviewers should independently > score each one. It cannot exceed the number of reviewers you added. **Rotate > evaluator pairs** varies who overlaps with whom across the sample, so you do not > learn only about one pair. ## What reviewers do Reviewers reach their work from **Home > Tasks**, or from **Start review** on the calibration itself. Each ticket opens the normal evaluation workspace with a counter ("3 of 8 tickets") and their own completion percentage in the header. They score the scorecard as usual, click **Submit evaluation**, and move to the next one with the navigation arrows. Nothing about the calibration changes how an evaluation is produced. Each reviewer's submission is a real manual evaluation on that ticket: [Perform a manual evaluation](/guides/quality-assurance/perform-a-manual-evaluation) covers the scoring itself, and calibration scores show up in reporting like any other manual score. ## Read the results Open the calibration and you land on the **Calibration** tab: the assignment count, the span, an overall percentage complete, the filters you chose echoed back, and a card per reviewer with their own progress. Chase from here: a calibration where one reviewer has done nothing produces no reviewer-to-reviewer numbers at all. ![Calibration tab showing 12 assignments, a date range, 83% complete, a Medium score level filter, and three reviewers at 75%, 100%, and 75% complete](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/calibration-overview-tab.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=bf8ead98388f94142a08536b94f77a6a) The **Evaluations** tab has one row per ticket, with: * **AI QA score** — what automatic QA said. * **A column per reviewer**, holding that person's score for the ticket. * **AI-to-evaluators delta** — the average absolute gap between the AI score and each human score. Lower is closer agreement. * **Evaluator-to-evaluator delta** — the average absolute gap between every pair of human scores. This column only appears once two or more reviewers have work in the calibration. * **Status** — whether that ticket's reviews are done. ![Evaluations tab with a row per ticket, the AI score beside each reviewer's score, AI-to-evaluators and evaluator-to-evaluator delta columns, review status, and a summary strip of reviewers, tickets, assignments, and average scores](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/calibration-evaluations-deltas.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=337163d53b26ef13883d1a02bbc9560e) Both delta columns sort. Sorting descending on evaluator-to-evaluator delta puts the tickets your team scored most differently at the top. Along the bottom, a summary strip counts reviewers, tickets, and assignments, and shows **Avg AI Score**, **Avg Human Score**, and completion rate. A persistent gap between those two averages means your AI is calibrated differently from your people overall; a large per-ticket delta means your people disagree case by case. ## Take one ticket apart Click a row and you get the ticket beside a comparison panel: **Alignment**, **Deltas** as a count of misaligned checks out of the total, the **Evaluators** involved, and the **Agents** on the ticket. Work check by check to turn "we were twelve points apart" into "we disagree about whether verifying the account counts when the customer volunteered it". Mark individual checks as correct or incorrect as you go. Run the meeting from this view, and record whatever you settle somewhere durable: usually a sharpened criterion description in the scorecard, or an organization-wide rule in [Set score bands and the contest window](/guides/settings/set-score-bands-and-contest-window). ## Adjust a calibration in flight The Evaluations tab has a search box and a **Filter** by review status for finding your way around a large sample. You can add more tickets to a running calibration, and select rows to export them or remove them. Removal only applies to tickets nobody has scored yet, which keeps completed evaluations from disappearing out of your numbers. ## Related * [Perform a manual evaluation](/guides/quality-assurance/perform-a-manual-evaluation) * [Set score bands and the contest window](/guides/settings/set-score-bands-and-contest-window) * [Create a scorecard](/guides/quality-assurance/create-a-scorecard) * [Glossary](/guides/concepts/glossary) # Open a ticket's QA workspace Source: https://docs.rulebase.co/guides/quality-assurance/ticket-qa-workspace Read the AI score, its per-criterion reasoning, the conversation, and the ticket's history from the single screen where a reviewer does the work. Opening a ticket from any QA list puts you on the same screen: the evaluation on the left, the ticket itself on the right, and a thin top bar that moves you between tickets. Everything a reviewer needs to agree or disagree with a score is on it. ![The ticket QA workspace, with the AI review and criterion reasoning on the left and Details, Timeline, and Conversation on the right](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/reviewer-qa-workspace.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=54b852e87fcb83a917e437101d8cd2b9) ## The layout The **top bar** holds the back arrow, the review tabs, and **Previous** / **Next**. Those two buttons walk the list you arrived from, in the order it was in, so a filtered list works as a queue: filter down to what you want to review, open the first one, and work forwards. The **left panel** is the evaluation: a headline score, then one tab per agent on the ticket, then a card for each criterion on the scorecard. The **right panel** is the ticket: **Details**, **Timeline**, and **Conversation**, each of which collapses. On a narrow window it hides entirely rather than shrinking, and a **Conversation details** button appears in the top bar to bring it back. ## Reading the score Under the heading **Quality score**, a coloured tile carries the number and a **Summary** beside it gives Rulebase's account of what happened on the ticket in a few sentences. Red, amber, and green come from your organization's risk thresholds rather than a fixed cutoff, so 44 can be red in one organization and amber in another. Click the small information icon on the tile to break the number down. The tooltip lists every evaluator and scorecard that contributed, which is the fastest way to tell whether the headline is one AI score, an average across two agents, or a manual score sitting on top of an AI one. > [!NOTE] > > A ticket handled by more than one agent gets one tab per agent under the score > tile, and each agent is scored separately. The tab you are looking at is one > agent's evaluation, not the ticket's. Hover a tab to reveal a **⋮** menu with > **Add to coaching**, which sends this ticket into that agent's coaching queue. Under the tabs, a row reads `Priya Nair's score • Support QA` and repeats the score for that specific agent and scorecard. If the same agent was scored against more than one scorecard, the scorecard name becomes a dropdown. * **Auto-Zero** — a criterion marked as an auto-fail was failed, so the whole scorecard result is zero regardless of what else went well. * A score badge with a tooltip — a score cap was applied, and the tooltip says why. ## Reading the reasoning Each criterion is a card: the question from the scorecard, the outcome as a pair of badges (`Fail` and `0 / 30`), and beneath it the AI's explanation of how it reached that outcome against this specific conversation. ![A criterion card showing the question, a Fail badge with the score, and the AI's written reasoning](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/reviewer-ai-criterion-reasoning.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=61a112551235d23706dc11d2e2309620) Review the explanation, not just the score. If it cites a moment that did not happen, or misses one that did, that is a scoring error, and the thumbs up and thumbs down on each card are how you say so. [Give feedback on AI QA](/guides/quality-assurance/give-feedback-on-ai-qa) covers what happens next. A **View explanation** control appears on cards where there is more underneath: the justification the AI selected, or an answer drawn from your knowledge base with the sources it used. Not every card has one, and its absence just means the reasoning above is the whole story. > [!TIP] > > Scan the criterion cards before you read the transcript. You will read the > conversation differently once you know which two moments are in dispute, and it > takes half the time. ## The ticket panel **Details** carries the facts you need to judge context or to go and look at the original: the customer, ticket status, the assignee, when it started, and the ticket ID as a link out to the source help desk. **View more** expands the rest: when the ticket was evaluated, when Rulebase imported it, and any custom fields that came across from your help desk. ![The Details panel showing customer, status, tags, ticket ID, assignee, and start date](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/reviewer-ticket-details-panel.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=e59aac27ba7de5f696824ddf28a5669a) The **Tags** row is editable: the **+** applies an existing Rulebase tag or creates one on the spot. The **Tags** filter is only offered on an **Interactions** view, not on the QA **Tickets** view, so tag now and filter there. **Timeline** is the review history rather than the ticket history: who this ticket was assigned to and when, and any contests raised against its scores. **Conversation** is the exchange itself, customer messages and agent replies in order. ## When there is no AI score Not every ticket has one, and the workspace tells you which case you are in rather than showing a blank panel. **Evaluation skipped** comes with the specific reason, often that the ticket is not closed yet, or that a linked ticket in the thread is still open. **Evaluation failed** means it ran and errored. **Evaluation scheduled** means it is queued. **No QA evaluations** means nothing was ever attempted. Each of those states offers a button to move it along: **Try again** after a failure or a skip, **Run now** on a scheduled evaluation, and **Request evaluation** where nothing has run. [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) covers the reasons in depth. None of it stops you scoring the ticket yourself. The **+** beside the review tabs adds a **Manual evaluation** tab and starts one, whether or not the AI got there first. See [Perform a manual evaluation](/guides/quality-assurance/perform-a-manual-evaluation). ## Related * [Find tickets and evaluations](/guides/quality-assurance/find-tickets-and-evaluations) * [Perform a manual evaluation](/guides/quality-assurance/perform-a-manual-evaluation) * [Give feedback on AI QA](/guides/quality-assurance/give-feedback-on-ai-qa) * [How AI QA works](/guides/concepts/how-ai-qa-works), for what produced the score before you opened it * [Why is my score different?](/guides/troubleshooting/why-is-my-score-different), when the number here disagrees with a report # Work contested tickets Source: https://docs.rulebase.co/guides/quality-assurance/work-contested-tickets Find evaluations an agent has disputed, read their objection against the original score, and accept or reject each contested criterion. When an agent disagrees with a score, they contest it from their own view of the evaluation, naming the criterion and saying why. That creates a contested evaluation: a small, self-contained review job with the original result on one side and the agent's argument on the other. Your job is to decide, criterion by criterion, who is right. Contests have their own view in the Inbox rather than a section of their own, and if you have a routing rule for **Evaluation is contested** they arrive already assigned to a queue or a person. ![Contested tickets inbox with three groups, Assigned to you, Up for review, and Completed, each holding one contest](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-contested-inbox.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=54763dd199f41719b81076c15a0d5397) ## Before you start * Open [Inbox > Contested tickets](https://app.rulebase.co/inbox/contested-tickets). * Agents raise contests themselves, and only on an evaluation with at least one failed criterion. If your organization sets a dispute window under [Settings > Evaluations](https://app.rulebase.co/settings/evaluations), contests raised after that many days are refused. * An agent can have only one open contest per ticket at a time, so you will not find two competing objections on the same evaluation. ## Find the contest you should be working The page groups contests by who is holding them: * **Assigned to you** — unresolved contests on tickets you are the assignee of. Start here. * **Up for review** — unresolved contests assigned to someone else, or to nobody. Scan this when you are covering for a colleague or clearing a backlog. * **Completed** — contests that have had a decision. The badge on the right reads **Accepted** or **Rejected**. Each row shows the QA score that was contested, the agent who raised it, a summary of the ticket, and an assignee avatar you can click to hand the contest to someone else without opening it. When the list is long, **Filters** narrows it by **Assignee**, **Agent**, **Contest level**, **Outcome**, **Contested evaluation** (whether the disputed score came from AI or a person), and **Contest date**. Contest level counts how many times that agent has contested that ticket, so filtering to `2nd level` surfaces the disputes that have already been through one decision. ## Open the objection A contested ticket carries an **Evaluation contested** banner across the top, and the evaluation itself shows how many criteria are in dispute. 1. Click the contest in the inbox to open the ticket. 2. Under the agent's score, find the line counting the contested evaluations, such as **1 evaluation contested** when a single criterion is in dispute. 3. Click **Review evaluation**. ![Ticket view with an Evaluation contested banner at the top and a Review evaluation button beside the contested count](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-contest-ticket-banner.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=77d5f5bfc640a53acd5a3daebbc3033d) ## Read both sides before deciding The **Contested evaluations** dialog puts the conversation, the original result, and the agent's argument side by side, so the transcript is one pane away while you weigh the claim. * **Left** — ticket details and the conversation itself. This is where you verify the claim rather than take it at face value. * **Middle, Contested evaluation** — the criterion as it was scored, its score out of the maximum, and the reason recorded at the time. Where the evaluation used checks, **Check results** expands to show each one. * **Right, Contests** — what the agent is disputing. **Disagree with score** and **Disagree with feedback** are separate claims and a contest can carry both, followed by their comment and any files they attached. ![Contested evaluations dialog showing the conversation details, the contested criterion with its score and reason, and the agent's objection with Accept and Reject buttons](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-contest-review-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=208bdec1e539fc911fa4e65530ee8d7d) ## Decide each criterion Every disputed criterion needs its own decision, and **Submit feedback** stays disabled until they all have one. 1. Click **Accept** or **Reject** on the criterion. 2. If you accepted, set the **New Score**. A criterion built on score options gives you a dropdown, sometimes with a feedback option to pick alongside it. A criterion scored out of a maximum gives you a number field. 3. Write the **Feedback**. It is required either way, and the agent reads it, so say what you looked at rather than just restating the outcome. 4. Attach a file if the evidence is easier to show than describe. 5. Click **Submit feedback**. ![Contested evaluations dialog with Accept chosen, a new score of 4 entered, and reviewer feedback written in the Feedback field](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/routing-contest-decision.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=2e8b04b64387f38786702a42a4f853dd) Rejecting takes only feedback: there is no score to set, because the original stands. > [!TIP] > > Answer the specific claim. If the agent says the information came from a billing > email rather than from them, the useful reply says whether you found that email, > not that the score was correct. A contest is usually the only written exchange an > agent gets about a score they think is wrong. ## What submitting does Rulebase writes a new manual evaluation in your name, copied from the contested one, with your decision recorded against each criterion you ruled on. That evaluation becomes the ticket's current result, replacing the contested one, as the warning at the top of the dialog says. The contest itself moves to **Completed**. Its badge reads **Accepted** only when you accepted every disputed criterion; reject any one of them and the whole contest is recorded as **Rejected**. On the ticket, the review prompt is replaced by an **Evaluation contested** line with a **View results** button, so the agent and anyone reviewing later can read your decision and feedback against each criterion. Resolving a contest does not resolve the ticket. If the ticket was assigned to you for review, it stays assigned and unresolved until you close it out the usual way. ## Related * [Give feedback on AI QA evaluations](/guides/quality-assurance/give-feedback-on-ai-qa) * [Create a routing rule](/guides/quality-assurance/create-routing-rules) * [Create and manage queues](/guides/quality-assurance/create-and-manage-queues) * [Manage and reorder routing rules](/guides/quality-assurance/manage-routing-rules) # Roles and permissions Source: https://docs.rulebase.co/guides/roles-and-permissions Reference for the seven system roles Rulebase ships with and the permissions each one grants. Every member holds exactly one role, a bundle of permissions. This is the reference for the roles Rulebase ships with and what each can do. To build or change a role, see [Create and manage roles](/guides/access-management/create-and-manage-roles). ## How roles work * **One role per member.** Roles don't stack. To widen someone's access, switch their role or edit the role. Editing a role changes it for everyone who holds it. * **Roles set what someone can do.** What they see is a separate setting, the membership's **Scope**. Scope defaults to the whole organization and can be narrowed to groups, teams, or partners; the picker appears once partners are configured. See [Invite members](/guides/access-management/invite-members). * **Seven system roles** ship with every organization, identical everywhere and not editable. Anything else is a custom role you build. ## System roles | Role | Built for | | - | - | | **Administrator** | Owning the organization. Every permission on this page, including settings, integrations, members, and AI agents. | | **Agent** | A front-line agent's view of their own work. No permissions. Works differently from the rest (see [The Agent role](#the-agent-role)). | | **Compliance Analyst** | Reviewing conversations for compliance. Performs evaluations, provides model feedback, and reads reports. | | **Manager** | Running QA org-wide: scorecards, knowledge base, teams, calibrations, evaluations. | | **Member** | Read-only visibility into reports, analytics, and agent performance. | | **Quality Analyst** | Day-to-day evaluating. Performs evaluations, provides model feedback, and reads reports. | | **Team Leader** | Leading a team. Same as Manager, minus scorecards. | **Quality Analyst** and **Compliance Analyst** differ by one permission (Quality Analysts can open AI evaluations on every conversation). **Manager** and **Team Leader** differ by one too (only Manager manages scorecards). ## Permissions by role This mirrors the **Permissions** checklist in the role dialog: same groups, order, and wording. **Administrator** holds everything (no column); **Agent** holds nothing. A dashed row is Administrator-only. | Permission | Quality Analyst | Compliance Analyst | Manager | Team Leader | Member | | - | - | - | - | - | - | | **People** | | | | | | | Invite new members | — | — | — | — | — | | Manage members, assign roles, and create or edit roles | — | — | — | — | — | | Manage teams and their members | — | — | Yes | Yes | — | | Manage the agent roster (add, remove, and edit agents) | Yes | Yes | Yes | Yes | — | | **Evaluations & calibrations** | | | | | | | Evaluate conversations and request AI evaluations | Yes | Yes | Yes | Yes | — | | Mark conversations Done and change review status | Yes | Yes | Yes | Yes | — | | Submit manual evaluations for approval only | Yes | Yes | Yes | Yes | — | | Approve submitted evaluations, publish directly, and give feedback that trains the AI | Yes | Yes | Yes | Yes | — | | Delete evaluations from conversations and performance metrics | — | — | — | — | — | | Assign conversations for review, manage tags, and edit assignment rules | Yes | Yes | Yes | Yes | — | | View calibrations | — | — | Yes | Yes | — | | **Scorecards & knowledge** | | | | | | | Manage scorecards, compliance rules, and complaint categories | — | — | Yes | — | — | | Manage knowledge base sections, sources, and documents | — | — | Yes | Yes | — | | **Insights** | | | | | | | View organization-wide reports and analytics | Yes | Yes | Yes | Yes | Yes | | View the agent roster | Yes | Yes | Yes | Yes | Yes | | View AI evaluations on every conversation | Yes | — | — | — | — | | Set performance targets for teams, agents, and brands | Yes | Yes | Yes | Yes | — | | **Customers** | | | | | | | View customers and companies | Yes | Yes | Yes | Yes | Yes | | **Settings & integrations** | | | | | | | Manage integrations (Zendesk, Intercom, etc.) and API keys | — | — | — | — | — | | Manage organization settings: sampling rules, risk levels, partners, and notifications | — | — | — | — | — | | **Workflows** | | | | | | | View workflows, their runs, and approvals | — | — | — | — | — | | Run workflows and chat with them | — | — | — | — | — | | Create, edit, publish, and delete workflows | — | — | — | — | — | | **Instruct** | | | | | | | View, send, and share Instruct chats and artifacts | Yes | Yes | Yes | Yes | Yes | The table does not show: * Anyone with **Evaluate conversations and request AI evaluations** can also submit a manual evaluation. **Submit manual evaluations for approval only** is the weaker alternative for reviewers who should never publish. Without **Approve submitted evaluations, publish directly, and give feedback that trains the AI**, a member's manual evaluations enter the approval queue instead of publishing; that same permission is what lets an approver clear the queue. * **Settings & integrations** and **Workflows** are Administrator-only in every system role. To let someone else configure integrations or build AI workflows, use a custom role. * Conversations have no permission of their own. Every member can open the conversations inside their membership **Scope**; permissions only gate what they can do to them. * The **Workflows** permission group is the AI workflows you build under [Home > Agents](https://app.rulebase.co/home/agents). It has nothing to do with the **Agent** role or the agents on your roster. ## The Agent role **Agent** has no permissions by design. Agents don't use the main workspace. Signing in lands them on their own performance portal, where they can: * read their own performance overview and weekly reports * open evaluations of their own tickets * contest an evaluation within your organization's dispute window * read the coaching sessions their coach has logged for them The portal is scoped to their own records **in code**, so no permission checkbox widens it. For a hard data boundary, such as an outsourced reviewer who must never load another partner's conversations, use a scoped membership. Scope is enforced on every conversation query. See [Set up partners](/guides/access-management/set-up-partners). A few Agent-specific quirks: the **Scope** picker is disabled for Agent (and for Administrator); neither can be cloned as the basis for a custom role; and a new member given the Agent role is added to the [QA roster](/guides/quality-assurance/manage-agent-roster) as evaluable. Their tickets start flowing into QA automatically. ## Custom roles When none of the seven fits, build a custom role from the same permissions. Common cases are a validator who reviews evaluations but can't touch member settings, or a knowledge owner who maintains scorecards and nothing else. Most custom roles are a system role plus or minus one permission, so cloning **Quality Analyst** or **Manager** and toggling a checkbox is usually faster than building from scratch. Viewing roles needs no special access; creating, editing, cloning, or deleting one needs **Manage members, assign roles, and create or edit roles**. ## Related * [Create and manage roles](/guides/access-management/create-and-manage-roles): create, edit, clone, assign * [Invite members](/guides/access-management/invite-members): set role and scope at invitation * [Assign roles from your identity provider](/guides/identity-provider-role-assignment): map IdP groups to roles * [Create and manage teams](/guides/access-management/create-and-manage-teams) # Configure notifications Source: https://docs.rulebase.co/guides/settings/configure-notifications Set who receives the weekly customer issues digest by email and which Slack channel gets complaint alerts, and know what to check when a message never arrives. [Settings > Notifications](https://app.rulebase.co/settings/notifications) holds your organization's default recipients and channels: a weekly email digest of customer issues, and instant Slack alerts when a complaint is detected. ![Settings > Notifications page showing the Agent QA reviews, Compliance digest, Customer issues digest, and Real-time complaints alerts groups](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/notifications-settings-page.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=a32fb1b22354006cf4db8c090551ba65) Every section on the page saves as you edit it. There is no **Save** button and no confirmation: clear a recipient list and you have turned that notification off for everyone. > [!NOTE] > > Anyone who can open Settings can change the recipients and the alert channel, so > treat this as shared configuration and check it after onboarding a new team. ## The weekly customer issues digest The digest is a written summary of the week's customer issues (complaints, service and product issues, and how your scorecards moved) assembled by Rulebase after the week closes and emailed to the people you list. The recipient list is the control. Add addresses under **Customer issues digest** and the digest goes to those people; leave the list empty and it goes to nobody. Type or paste addresses separated by commas, then click outside the box to save. ![Customer issues digest section with Weekly, Mondays, 9:00 AM and Notify via Email selected, and two example recipients listed](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/notifications-customer-issues-digest.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=9e82f3969e33d9ead00f508b68eba712) The pickers alongside it offer **Weekly** and **Notify via Email**. Each digest covers the previous complete week in your organization's timezone and arrives after that week closes. > [!TIP] > > Two or three people is the right size for this list; it is a management > summary. If individual agents need their own numbers, use performance > reporting. ## Complaint alerts in Slack Under **Real-time complaints alerts**, pick the channel that should receive a message when Rulebase detects a complaint. ![Real-time complaints alerts section with a Slack channel picker open, showing a channel search box and four public channels](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/notifications-slack-channel-picker.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=e7c6a68368d16e6868d58a76c13b327b) The picker lists the public channels Rulebase can see in your workspace, refreshed daily. Pick one and it saves immediately. ![Real-time complaints alerts with the customer-complaints channel selected](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/notifications-complaints-channel.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=3f530194366b2bee131c2d91c9b3ac78) Not every complaint produces a message. Rulebase posts when the complaint is **high priority** and the ticket is **still open**, and it posts once per complaint, not once per update. A complaint found on a ticket your team already resolved does not raise an alarm about work that is finished. Reading the tickets detection attached, logging your investigation on the timeline, and closing the case are covered in [Work a complaint case](/guides/compliance/work-a-complaint-case). ### When the alert does not arrive Work through these in order, because the first one is the cause most of the time: 1. **Invite Rulebase to the channel.** Rulebase can only post where the app is a member. In Slack, open the channel and run `/invite @Rulebase`. A configured channel that has never received a message is almost always this. 2. **Check the channel is in the picker.** The list is built from public channels and refreshed daily, so a channel created an hour ago, or a private one, may not be there yet. 3. **If the picker is completely empty**, the channel list has never synced for your workspace. Reinstall Slack from [Settings > Connections](https://app.rulebase.co/settings/connections), and contact Rulebase support if it remains empty. 4. **Confirm a complaint was detected.** Open the complaints list for the period and check the priority. A week of medium-priority complaints produces no alerts, correctly. 5. **Check whether a routing rule has taken over.** If anyone has built a routing rule that notifies on complaint creation, the channel above stops being used; see the warning below. Check this before you start reinstalling anything. See [Slack](/guides/integrations/slack) for the install itself. ## When a rule needs to do more than the defaults The two settings above are organization-wide: one digest list, one complaint channel. When you need something conditional (this queue's escalations to that channel, complaints above a threshold to a specific group), build it as a routing rule instead, under [Settings > Routing](https://app.rulebase.co/settings/routing). Routing rules have a **Notify** action with Slack and email targets, and they run against the conditions you set on the rule. If you have any routing rule that notifies on complaint creation, that rule takes over and the **Real-time complaints alerts** channel above is skipped, so you get one message rather than two. [Create routing rules](/guides/quality-assurance/create-routing-rules) covers the setup. > [!WARNING] > > The takeover is all or nothing. One complaint rule switches the channel above off > for **every** complaint, whether or not the rule's conditions match, and the > channel keeps showing as configured here either way. A rule scoped to one team > therefore leaves every complaint outside that team with no alert from either > place. > > If you want a rule for a subset, pair it with a second rule that has no > conditions and points at the channel above, so the rest are still covered. The **Alerts** section at the bottom of the page, with its **Add rule** button, holds condition-based rules of its own: a Slack channel or a list of email addresses, plus conditions on risk levels, customer sentiment, or a specific scorecard criterion. If you have a rule here that has never produced a message and the Slack checks above come back clean, raise it with support rather than rebuilding it. Build the alert you need as a routing rule. ## Related * [Slack](/guides/integrations/slack) * [Create routing rules](/guides/quality-assurance/create-routing-rules) * [Work a complaint case](/guides/compliance/work-a-complaint-case) * [Manage integrations](/guides/integrations/manage-integrations) # Set score bands and the contest window Source: https://docs.rulebase.co/guides/settings/set-score-bands-and-contest-window Define the Low, Medium, and High score ranges the rest of Rulebase reads from, add organization-wide evaluation instructions, and decide how long agents have to dispute a score. An evaluation produces a number, but almost nothing in Rulebase acts on the number directly. Routing rules match on **Low**, **Medium**, or **High**; calibration sampling can ask for one band and ignore the rest; charts group results into the same three buckets. Score bands are where you decide what those words mean. The **Review** card underneath sets how long an agent has to dispute a score before the option disappears. ![Scoring and Review cards in evaluation settings, showing score bands of 0-50, 51-80, and 81-100, evaluation instructions, and a 14-day contest window](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scoring-review-settings-scoring-and-review.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=8d54ad1a98ee9939d3eb445f728fc887) ## Before you start * Both cards live under [Settings > Evaluations](https://app.rulebase.co/settings/evaluations), below **Eligibility** and **Timing** in the **Configuration** part of the settings sidebar. * You need an admin role. Without it the fields render but stay disabled. * Look at where your scores currently sit before you move a threshold; team and agent scores live under [Insights](https://app.rulebase.co/insights/performance?view=teams). Otherwise a whole team can end up in one colour. ## Decide where the bands sit Rulebase ships with 0-50 Low, 51-80 Medium, 81-100 High. Those defaults treat a score in the fifties as a real problem, which suits teams whose scorecards fail tickets readily. If most of your criteria are pass/fail and a single miss costs 25 points, the same thresholds will file competent work as Medium and your routing will send far more tickets for review than you intended. You set two numbers and Rulebase derives the third. 1. Under **Scoring**, click the three coloured badges next to **Score bands**. The editor expands underneath. 2. Set the **Low** maximum. Every score from 0 up to and including this number is Low. 3. Set the **Medium** maximum. Medium starts one point above the Low ceiling. 4. Click outside the field. There is no save button; the change is stored when the field loses focus, and **High** redraws as everything above your Medium ceiling. The editor keeps the bands from crossing: the Low ceiling can never reach the Medium ceiling, and the Medium ceiling stops at 99, so a perfect score is always High. > [!NOTE] > > Each evaluation records its band at the moment it is scored. Filters and > calibration sampling read that recorded band, so moving a threshold changes > which band new evaluations land in without re-filing the ones you already have. > Set the bands before you turn on automatic QA, and treat later changes as a > before-and-after in your reporting rather than a retroactive correction. ## Where the bands are used **Routing rules.** The **QA score** and **Agent QA score** conditions offer exactly Low, Medium, High, and Autozero. A rule that sends low-scoring tickets to a reviewer is really a rule about your Low ceiling, so raising that ceiling quietly widens the rule's reach. See [Create routing rules](/guides/quality-assurance/create-routing-rules). **Calibration sampling.** When you build a calibration you can filter the pool by **Agent QA score level**, which is how you run a session on the tickets your scorecard was harshest about. [Run a calibration](/guides/quality-assurance/run-a-calibration) covers the rest of that flow. **How scores are presented.** A ticket's QA workspace colours its score by band, and dashboard charts that break QA results down into High, Medium, and Low draw their groupings from the same two numbers. A generous Low ceiling makes a lot of competent work look like a problem in every chart you show the team. The colour on a ticket is worked out from your current thresholds, while filters and calibration sampling use the band recorded when the evaluation was scored. Right after you move a threshold, a ticket can look Medium on screen and still answer to a Low filter. ## Write instructions that apply to every evaluation The **Evaluation instructions** row sits under the same **Scoring** card. It is broader than the per-check instructions on a scorecard: whatever you write here is handed to the AI evaluator on every evaluation it runs, whatever scorecard applies, including the re-scores triggered by criterion feedback. It does not change the scorecard's structure or points; it changes how the criteria are interpreted. 1. Next to **Evaluation instructions**, click **Edit**. 2. Write your rules in the editor. Markdown works, and one rule per paragraph is easier to revise later than a single dense block. 3. Click **Save changes**. ![Edit evaluation instructions dialog with four organization-wide scoring rules about agent knowledge at the time of reply, borderline checks, reply length, and language switching](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scoring-review-evaluation-instructions-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=e4719ced1acf43acee760c8dbaf5a11e) The instructions worth putting here are the judgement calls you would otherwise repeat in every scorecard: how to treat a policy that changed after the ticket closed, whether a short reply can still be complete, what to do when a check could reasonably go either way. Leave anything that belongs to one rubric on that rubric, with [Add scorecard additional instructions](/guides/quality-assurance/add-scorecard-additional-instructions). > [!TIP] > > Write these as decisions rather than aspirations. "Award the points and note the > reason when a check could go either way" changes scores. "Be fair" does not. ## Open the contest window An agent who disagrees with a score can dispute it, and the **Contest window** decides for how long. The clock starts at the evaluation date, not the ticket's close date, which matters when your evaluation delay is long or a backlog was scored weeks after the fact. 1. Under **Review**, click **Add window**. 2. Set the number of **days**. Anything from 1 to 365 is accepted, and 30 is the starting value. 3. Click outside the field to save. To remove the deadline entirely, click the **×** to the left of the field. That does not block disputes; it means a dispute is accepted no matter how old the evaluation is. Most teams land somewhere between 7 and 30 days. Short windows keep disputes close to the conversation, while a window shorter than your review cadence means agents lose the right to dispute before anyone has discussed their scores with them. ## Cap contests per ticket Under the contest window sits **Max contests per ticket** — how many times the same agent can dispute the same ticket. Every organization starts at **3**. Set any value from 1 to 100; blur the field to save. Once an agent hits the cap, the contest button stays disabled and the API rejects further contests for that ticket, even after earlier disputes are resolved. ## What an agent can and cannot contest The window is one of five gates: * An agent can only contest **their own** evaluation, and only the one currently in effect for that ticket. A superseded evaluation is read-only. * The evaluation must have at least one failed check. * One open dispute per agent per ticket. The next one waits until the first is resolved. * The agent has not reached the max contests per ticket for that interaction. * The evaluation must be inside the contest window. Past the deadline, the button stays visible but disabled, and the tooltip names the window length so the agent can see what they missed. Disputes that land arrive as work for your QA team rather than as a status change on the ticket, and [Work contested tickets](/guides/quality-assurance/work-contested-tickets) picks the story up from there. ## Related * [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility) * [Work contested tickets](/guides/quality-assurance/work-contested-tickets) * [Create routing rules](/guides/quality-assurance/create-routing-rules) * [Run a calibration](/guides/quality-assurance/run-a-calibration) * [Glossary](/guides/concepts/glossary) # Set SLA targets and business hours Source: https://docs.rulebase.co/guides/settings/set-sla-targets-and-business-hours Build an SLA policy with response and resolution targets, then define the work schedule and holidays those timers run against. An SLA policy answers two questions: how fast should we reply, and when does the clock run. The first is a set of targets per ticket priority. The second is a work schedule: Rulebase measures targets in business hours, so until you tell it which hours those are, a four-hour first-response target set at 5pm on Friday is breached by 9pm on Friday. Policies, schedules, and the holiday calendars those schedules skip all sit on one page. ![SLA policies settings page with one policy, a default work schedule of Mon-Fri 9:00 AM to 5:00 PM EDT, and a US public holidays calendar](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/sla-settings-policies-schedules-holidays.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=c57d88750cdee27b635c8cbbbaf7c457) ## Before you start * The page is [Settings > SLA policies](https://app.rulebase.co/settings/sla-policies), under **Configuration** in the settings sidebar. No feature toggle gates it. * You need an admin role. * Connect a help desk first. SLA outcomes are calculated from synced tickets, so a policy built before any integration exists has nothing to measure. * Decide which channels the policy covers and what you promise at each ticket priority before you start clicking. Both are editable later, but saving a policy rewrites its conditions and targets wholesale, so it is a page you want to fill in deliberately rather than incrementally. ## Create the policy 1. Click **Create policy**. 2. Give it a **Name** your team will recognise in a chart legend; policy names appear as a filter on SLA dashboard charts. 3. Add a **Description** if the name needs qualifying. 4. Click **Create policy**, then open the policy from the list. > [!NOTE] > > Ignore **Set as default policy** in this dialog. A policy is matched to a ticket > by its conditions, and the default flag is not a reliable substitute for them, so > give every policy at least one condition. A policy with no conditions is not > applied to anything. ## Say which tickets it covers Conditions are what make a policy apply, and there is one field to key on: **Channel**. 1. On the policy, click **Add condition**. 2. Leave **Type** as **Channel**. 3. Choose the **Operator**. Use **includes** whenever you want more than one channel; **is** and **equals** compare against a single value and match nothing when two are selected. 4. Under **Value**, pick from **Call**, **Email**, and **Chat**. Several policies can apply to the same ticket if their conditions all match, which is how a stricter chat policy and a general email policy coexist. It is also how you accidentally double-count breaches, so keep the channel sets disjoint unless you mean to overlap them. ## Set a target for each priority A target is one metric measured across the four ticket priorities. Three metrics are available and each can be used once per policy: * **First response time** — from the ticket arriving to the first agent reply. * **Next response time** — the gap on each subsequent customer message. * **Resolution time** — from arrival to the ticket being resolved. 1. Click **Add target**. 2. Pick the **Target metric**. Metrics already on this policy are not offered again, and **Add target** greys out once all three are in use. 3. For each of **Urgent**, **High**, **Normal**, and **Low**, enter a number and choose **Minutes**, **Hours**, or **Seconds**. 4. Click **Add target**, and repeat for the next metric. ![Add new target dialog for first response time with 30 minutes for urgent, 1 hour for high, 4 hours for normal, and 8 hours for low](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/sla-add-target-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=9455c9e4ce82bfdbb2607387fe01b647) Leave a priority at 0 and no target is created for it, the right move when you have no commitment at that priority. Filling all four is usually easier to explain to the team than a policy with gaps. ## Stop the clock while you are waiting on the customer **Pause Statuses** takes the status names from your help desk. While a ticket sits in one of them, the timer stops and the deadline shifts by however long the pause lasted, so a customer who takes three days to reply does not cost you the breach. Type the status exactly as your help desk spells it (for example `pending`, `on-hold`, `waiting for customer`) and click **Add**. Matching ignores case but not spelling, and a status name that does not exist in your help desk simply never pauses anything. ![Pause Statuses with a pending status added, and a Targets table listing first response time and resolution time across urgent, high, normal, and low priorities](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/sla-policy-targets-and-pause-statuses.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=6e8d1a32e1c2b81970a32c5492e22b59) > [!NOTE] > > **Save changes** rebuilds the policy's conditions and targets from what is on > screen. Finish the whole policy in one pass and save once. Do not leave a blank > condition row in place when you save, and re-open the policy afterwards to > confirm the targets are still listed. ## Teach Rulebase your working week Targets created here are measured in business hours. With no schedule to consult, Rulebase falls back to continuous elapsed time, so an untouched account looks like it is breaching overnight and at weekends. Fix that once, at the bottom of the same page. Start with the holidays, because a schedule can link a calendar the moment you create it. 1. Next to **Holiday calendars**, click **Add calendar**. 2. Name it, choose the **Country**, and narrow it with **Subdivision** if a region observes different days. Leave the subdivision on **National holidays** for the country-wide set. 3. Check the year preview on the right, then click **Create calendar**. ![Add holiday calendar dialog for US public holidays with a preview listing New Year's Day, Martin Luther King Jr. Day, and other national holidays](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/sla-add-holiday-calendar-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=03d7b75e97d143b1ce8efd42b490fcf8) Then the schedule: 1. Next to **Schedules**, click **Add schedule**. 2. Name it and set the **Timezone**. Every start and end time you enter is read in this timezone, so pick the one your team actually works in rather than the organization's reporting timezone. 3. Switch on the days you work and set the hours. Monday to Friday, 9:00 AM to 5:00 PM is the starting point. 4. Choose the **Holiday calendar** you just created. 5. Click **Create schedule**. ![Create work schedule dialog with Monday to Friday enabled from 9:00 AM to 5:00 PM in America/New York and a linked US public holidays calendar](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/sla-create-work-schedule-dialog.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=19f124a3f1f583db248c7e1bada0e657) Finally, connect the schedule to your timers: open the row's **⋯** menu and choose **Make default**. A **Default** badge appears next to the name. Schedules that are not the default, and carry no conditions, are never consulted. > [!TIP] > > If your coverage differs by region or brand, create the extra schedules with > conditions and keep one plain schedule as the default. Rulebase prefers a > matching conditional schedule and falls back to the default. ## Choose the week your reports start on The calendar icon beside **Create policy** opens **SLA reporting window**, where **Week starts on** sets the week boundary for SLA reports and the weekly SLA dashboard charts. It defaults to Sunday. Set it to whatever your operations week uses; otherwise a Monday-to-Sunday team reads every weekly chart off by a day. ## Confirm it before you rely on it Check these in order: * The policy list shows your policy, and opening it still shows the conditions and targets you saved. * The **Schedules** table shows a **Default** badge, with the hours and timezone you expect. * The holiday calendar's count looks right for the country you chose, and the schedule row lists it under **Holidays**. * A ticket that arrived outside working hours has a deadline that starts at the next working morning rather than an hour after it landed. Outcomes surface on dashboard charts built from the **SLA breaches**, **SLA adherence**, and **SLA weekly outcomes** data sources, filterable by policy. [Manage dashboard charts](/guides/insights/manage-dashboard-charts) covers building those. ## Related * [Manage dashboard charts](/guides/insights/manage-dashboard-charts) * [Set score bands and the contest window](/guides/settings/set-score-bands-and-contest-window) * [Glossary](/guides/concepts/glossary) # SSO setup Source: https://docs.rulebase.co/guides/sso-setup Give your IT team administrator access and get them into the Admin portal, where Single Sign-On is configured for your organization. Set up Single Sign-On (SSO) for your Rulebase organization. ![Settings members page](https://mintcdn.com/rulebase/InjNtZCAyHTTZZPL/images/settings-members.png?fit=max\&auto=format\&n=InjNtZCAyHTTZZPL\&q=85\&s=9c5a724e7f75552540c05fc14906e1af) ## For administrators As an administrator in your Rulebase account, you need to invite your IT team as administrators to enable SSO setup. 1. Navigate to your organization's members settings: * For EU region: [https://eu.app.rulebase.co/settings/members](https://eu.app.rulebase.co/settings/members) * For US region: [https://app.rulebase.co/settings/members](https://app.rulebase.co/settings/members) 2. Invite your IT team members as administrators. Once your IT team has administrator access, they can proceed with the SSO configuration. ## For IT administrators After being invited as an administrator, follow these steps to set up SSO: 1. Navigate to your organization's members settings: * For EU region: [https://eu.app.rulebase.co/settings/members](https://eu.app.rulebase.co/settings/members) * For US region: [https://app.rulebase.co/settings/members](https://app.rulebase.co/settings/members) 2. Click the **Admin portal** button located at the top right of the page. 3. Follow the setup instructions in the Admin portal to configure SSO for your organization. To map identity provider groups to Rulebase roles, see [Assign roles from your identity provider](/guides/identity-provider-role-assignment). ## Need help? If you encounter any issues during SSO setup, please reach out to [support@rulebase.co](mailto:support@rulebase.co) for assistance. # An agent is missing from reports Source: https://docs.rulebase.co/guides/troubleshooting/agent-missing-from-reports Find out why someone doesn't appear in QA or performance views: roster membership, team assignment, whether they were evaluated at all, or a second record under another email. Someone's lead says they cannot find them in performance. "Missing" covers four different situations. | What you are seeing | Where to start | | - | - | | Their name is nowhere in Rulebase | The roster | | They exist, but not under their team | Their team assignment | | They are listed with no score, or fewer tickets than you expect | Their evaluations | | Their numbers are split, or the same person appears twice | A duplicate record | | The team page reads zero, or no teams appear at all | The view, not the person | Work through them in order; each rules out the next, even if you have a hunch. ## Start on the roster The agent roster at [Settings > People > Agents](https://app.rulebase.co/settings/people/employees) is the list of people Rulebase runs QA and performance on, and everything downstream reads from it. If someone is not on this page, they will not appear in reports, and no amount of looking in Insights will change that. ![Settings People Agents page showing the agent count, Import and Add buttons, and a table of agents with Agent, Team, and Membership columns](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-roster-page.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=116fe5ad5d8cfdc83f3354b0ab9539d7) Search by email, since it is the identifier Rulebase matches on and names are often shortened or spelled differently. Clear any filters first. The filter icon next to the agent count narrows by **Team** and **Workspace**, and a **Workspace** filter left on **Sales** hides every customer service agent on the roster. If they are genuinely not listed, one of two things happened: they were never added (common for someone who arrived through a help desk integration), or they were removed from the active roster deliberately. Either way the fix is the same: **Add** them, or include them in your next roster import. > [!NOTE] > > The roster and a Rulebase login are separate things. The **Membership** column > tracks the login separately, and an agent showing **Not invited** is evaluated > normally. Do not chase an invitation to fix a reporting gap. Adding someone back does not create a fresh person. Their previous evaluations, scores, and history are still attached and reappear with them. ## Team views only show agents who have a team The roster gives this away in a single column. An agent whose **Team** column reads `-` belongs to no team, and team performance is organized entirely by team: [Insights > Performance](https://app.rulebase.co/insights/performance?view=teams) drills from a team into its agents, so someone with no team has no page to be found on. Their tickets are still evaluated and their evaluations still exist. There is just no route to them through team views. ![Agents table floating action bar showing 3 selected with Change team, Invite, and Remove actions, and the team search popover open above it](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/agent-roster-change-team.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=f53363158d4e190863e04102cab5e83b) Select them on the roster, click **Change team**, and pick the team. An agent belongs to one team at a time. Someone appearing under a team their lead did not expect usually moved teams in a CSV import, where the **Team** column reassigns silently. > [!TIP] > > After any bulk roster change, scan the **Team** column for rows sitting on `-`, > and check the team filter for names you do not recognize. An import that misspells > a team name silently creates the misspelled team, parking a handful of agents > somewhere nobody is looking. ## On the roster, in a team, but no score The agent is visible now, but the numbers are empty or lower than you expect. Performance figures are built from evaluations, and an evaluation only counts towards an agent's score when all of the following are true: * It finished processing. An evaluation still in progress contributes nothing. * It is the current one. When an evaluation is superseded by a later one (a manual evaluation replacing an AI result, for instance), only the replacement counts. * Its ticket was evaluated inside the date range you are looking at. Performance is dated by **when the evaluation happened**, which can differ from when the ticket arrived. A backfill of last quarter's tickets scored today lands in today's date range, so a week that looks empty for one agent may just be a week where nothing of theirs was scored yet. Widen the range before concluding anything. If the count is zero across every range, their tickets were probably never evaluated at all. That is a separate diagnosis with its own causes: eligibility settings, scorecard scoping, and roster timing among them. Work through [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) next. ## One person, two records Here the agent is split across two records. When someone's tickets arrive under a second email address (a help desk account created after a name change, or a personal address on one integration and a work address on another), Rulebase treats those as two people. Each holds part of the history, so both look wrong: one has the tickets you remember and the other has none, and neither total matches reality. Search the roster by surname, since a full-email search finds only one of the two records, and scan the **Agent** column for their name. Two rows for one person usually means the record is duplicated. Confirm both rows belong to the same person, by matching email or employee ID, before merging; merging reassigns interactions and evaluations and the wrong merge corrupts history. To combine them, open the overflow menu on the row you want to retire and choose **Merge**. Pick the record to keep under **Merge into**, and everything linked to the record you opened (interactions, evaluations, and the rest) is reassigned to it. The checkbox underneath, **Delete … after merging** with the retired record's name in it, throws the emptied record away. Leaving it unticked is the safer choice on your first merge. The search inside **Merge into** is not limited to the roster, so it can find a record removed from the roster earlier that is holding history you want back. ## When the problem is the view Sometimes the agent record is fine and the view is the problem. You can tell without checking the roster again. **No teams appear anywhere.** If [Insights > Performance](https://app.rulebase.co/insights/performance?view=teams) reads "No teams found" while [Settings > People > Teams](https://app.rulebase.co/settings/people/teams) clearly lists teams, stop looking at your data. That combination means your own membership cannot resolve which teams it is allowed to see, and the page shows an empty result without telling you so. Nothing you change on the roster will help. Ask a colleague to open the same page: if they see the teams, the problem is your account and Rulebase support can correct it. **A team total reads zero above agents with real numbers.** A team page showing 0 resolved tickets or 0% review coverage, while the agents table underneath shows tickets and scores for the same period, is a contradiction. The team-level figure needs each ticket linked to a team; the agent-level figure does not, so the two can disagree on the same work. Trust the agent rows, which count from the evaluations themselves, and treat the team summary as unreliable until it agrees with them. We are fixing this. > [!NOTE] > > Neither of these is caused by anything you did, and neither is fixed by moving > people between teams. Re-importing a roster to chase a zero that turns out to be > this will reassign agents you did not mean to touch, so rule it out before you > start editing. ## After you fix it Roster changes take effect going forward, and the reports catch up on their own. Add someone back or move them into a team and their existing evaluations attach to them again. Nothing needs recalculating by hand, because the numbers derive from evaluations that were already there. Before you close the ticket, confirm: * The agent count above the roster matches the number of people you expect to be evaluated. * Their team's page under [Insights > Performance](https://app.rulebase.co/insights/performance?view=teams) lists them, and their own detail page shows the tickets you expected. If a person's tickets were never evaluated in the first place, fixing the roster only helps from now on. Past tickets stay unevaluated unless someone re-runs them. ## Related * [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster) * [Create and manage teams](/guides/access-management/create-and-manage-teams) * [Team and agent performance](/guides/insights/team-and-agent-performance) * [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) * [Export QA evaluation data](/guides/quality-assurance/export-qa-evaluations) # My integration stopped syncing Source: https://docs.rulebase.co/guides/troubleshooting/integration-stopped-syncing Diagnose a help desk connection that stopped bringing tickets in: confirm the gap, read the connection's last sync and activity log, reconnect, and verify tickets are arriving. The first sign is usually indirect. Someone notices QA has gone quiet, or a dashboard flattens out, and the conclusion is that the help desk connection has died. Sometimes that is exactly right. Often the tickets are arriving fine and something further down the line stopped, so the fastest route to an answer is to establish which of those two you are dealing with before you reconnect anything. ## Are tickets missing, or just evaluations? These look identical from a dashboard and have nothing in common underneath. Open the interactions list and filter by **Workspace**, which narrows it to a single connection, then find the most recent ticket that came through it: * **Recent tickets, no recent evaluations.** Syncing is fine. This is an evaluation problem; start with [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) instead. * **Nothing since a particular day.** Something stopped, and that date is your most useful piece of evidence. Note it before going further; it usually lines up with a change on the help desk side. Pick one ticket you know exists in the help desk and was created after that date, then search for it in Rulebase. A ticket that is absent confirms a sync problem rather than a display or filter problem. ## What the Connections page tells you Go to [Settings > Connections](https://app.rulebase.co/settings/connections). Each connected account appears as a row under its provider, labelled with the account it points at (your Zendesk subdomain, your Intercom app, your Jira site), which matters when you have more than one connection to the same provider and only one of them is stalled. The row carries one piece of diagnostic information: **Last synced on**, with a date. A date that sits close to your last known ticket is a strong signal you have found the right connection. > [!NOTE] > > Not every connection shows **Last synced on**, and its absence is not itself a > fault. Some connections receive data without recording a sync timestamp, so treat > the date as useful when present rather than as a health indicator when missing. Be aware of what the page does **not** do: there is no error badge, no red state, and no warning when credentials stop working. A connection that has been failing for a week looks exactly like a healthy one apart from its date. That is why the ticket-level check above is worth doing first. ## Read the activity log Open the row's overflow menu and choose **View activity**. The **Activity Log** dialog lists what the connection has received, newest first, as a timestamp followed by the raw payload. It has a search box and a date range picker, so you can jump to the day things stopped and see whether anything arrived at all. Before you interpret it: * The log records data pushed to Rulebase by the provider. Connections that work by Rulebase pulling on a schedule will not have entries here, so **No activity logs yet** does not mean the connection is broken. * Entries stop at a fixed number per connection, with the oldest discarded, so a quiet connection can still show a full log from weeks ago. The boundary is decisive. If entries run daily and then stop dead on the date your tickets stopped, the delivery path stopped on that date. Check the help desk's webhook or app configuration first, and if that side looks right, contact Rulebase support. ## Reconnect the account Reconnecting is the same **Add** button you used the first time, in the provider's row. There is no separate reconnect action, and you do not need to disconnect first. You need permission to manage integrations to use it; without that, the provider's **Add** button is disabled. Authorize the same account you connected originally. Rulebase matches what you authorize against the connection it already has (by Zendesk subdomain, Intercom app, Jira site, Salesforce org, or Slack workspace) and refreshes that connection's credentials in place. Your history, settings, and sync position survive. Authorize a different account and you get a second connection alongside the first rather than a replacement, which is worth avoiding: two connections to overlapping data is harder to unpick than the original problem. > [!TIP] > > Connections carry the access of whoever authorized them, so reconnect as an > account with durable access rather than whoever happens to be at the keyboard. > When a connection breaks shortly after someone leaves, the person who originally > authorized it is usually the reason. Because the connection is refreshed rather than replaced, it keeps its place in the sync. Tickets from the gap arrive over the following minutes rather than being lost, so give it a little time before deciding the reconnect did not work. ## Disconnecting is not a repair step **Disconnect** in the same overflow menu is a common instinct and rarely the right move. It asks whether you are sure and tells you the action cannot be undone, which is accurate about the connection itself. Your data is not deleted. Tickets, evaluations, and reports that came in through that connection stay exactly where they are. What you lose is the connection's configuration and its place in the sync, so a reconnect afterwards starts fresh rather than resuming. Reach for it when you are deliberately retiring a connection or replacing it with a different account, not as a way of restarting one that stalled. > [!NOTE] > > If you connected Jira and its service requests never appeared as tickets, this is > not a sync failure. Open the row's **Configure** action and check **Import > service requests as interactions**, which is off unless someone turned it on. ## Confirm tickets are arriving again Give it a few minutes after reconnecting, then check in this order: 1. The connection's **Last synced on** date has moved to today. 2. The interactions list, filtered to that **Workspace**, shows tickets created after the date things stopped. 3. The specific ticket you looked up at the start is now present. 4. Something raised in the help desk right now appears in Rulebase shortly afterwards. This is the only check that proves the connection is live rather than catching up on a backlog. Tickets that arrive from the gap go through the same path as any other new ticket, so eligible ones are evaluated without anyone re-running them and the quiet period in your QA numbers fills in on its own. If tickets are arriving but evaluations still are not appearing, the sync is fixed and what remains is a different problem. ## Related * [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) * [Connect Zendesk](/guides/integrations/zendesk) * [Connect Intercom](/guides/integrations/intercom) * [Roles and permissions](/guides/roles-and-permissions) # Why didn't my routing rule fire? Source: https://docs.rulebase.co/guides/troubleshooting/why-didnt-my-routing-rule-fire Work out why a ticket wasn't assigned: rule order, the exact trigger event, conditions that cannot match, and actions that ran but changed nothing. Routing fails quietly. A ticket that should have landed in your review queue sits untouched, with nothing in the interface to say a rule was considered and passed over. **Only one rule runs per event.** Start on [Settings > Routing > Rules](https://app.rulebase.co/settings/routing/rules); the answer is usually visible on that page before you open anything. ## Read the rules list the way Rulebase reads it **Conditions** prints the whole trigger as a sentence: the event name, then the conditions in brackets joined with `and`, or `(all conversations)` when the rule has none. **Actions** prints what will happen, or `-` when the rule has nothing to do. Row order on this page is execution order. When an event fires, Rulebase walks the list from the top, skips any rule watching a different event, and stops at the first rule whose conditions all hold. Hovering a row shows a drag handle on the left for reordering, which appears once you have more than one rule. Scan for these before opening your own rule: * A rule **above** yours that would also match this ticket. It runs, yours does not. * A rule whose Conditions cell reads **Any**. It has no trigger set, so it is a candidate for *every* event. * A rule whose Actions cell reads `-`. It still counts as a match and still stops the rules below. ## One rule per event, and the top one wins Matching is not cumulative. If two rules could both handle a ticket, only the higher one does, and the lower one behaves exactly as though it did not exist. Nothing in the interface logs that this happened. > [!NOTE] > > Most workspaces contain a rule nobody created. It reads **Any** under Conditions > with nothing in the brackets, and **-** under Actions, and it matches any event on > a ticket that scored below 80%. It stops the rules under it while appearing to do > nothing. If your rules only fail on low-scoring tickets, this is almost certainly > why. Drag your own rules above it. Whatever is absorbing the event, the fix is the same shape: narrow rules at the top, catch-alls at the bottom. A rule with no conditions matches every occurrence of its trigger, so it belongs at the end of the list. ## The trigger has to be the exact event A rule only ever sees the one event it names, set in the rule's **When** section. Open your rule, read what **When** is set to, and check it against the list below. Each trigger fires at a different moment. ![Create rule dialog with the Evaluation approval is requested trigger selected and an Assign to queue action sending the evaluation to a validation queue](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/create-routing-rule-approval-trigger.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=05d4658d3b4dfe501c40ce08a0b5e75c) | Trigger | Fires when | | - | - | | **Ticket is evaluated** | AI QA finishes scoring a ticket | | **Evaluation approval is requested** | Someone submits a manual evaluation for validation | | **Ticket changes status** | The ticket moves to a new status in your help desk | | **Evaluation is contested** | An agent disputes an evaluation result | | **Complaint is created** | A complaint record is raised | Submitting a manual evaluation for review raises **Evaluation approval is requested**. A rule built on **Ticket is evaluated** will never see it, however well its conditions fit. A rule also cannot fire on an event that never happened. If AI QA never scored the ticket, no evaluated event was raised, and no rule tuning will route it. That is a [different problem](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) with a different fix. ## Conditions must all hold, and some cannot hold at all Conditions are joined with `and`, so every one has to be satisfied and each condition you add is another chance for the rule to be skipped. Common silent misses: * **Blank data fails a condition, including a negative one.** A ticket with no channel recorded does not match `Channel includes Email`, and it does not match `Channel does not include Call` either. The same holds for **Group**, **Partner**, **Workspace**, and **Ticket status**: no value on the ticket means no match, in either direction. * **Failed criteria means scored zero.** The condition matches criteria that scored 0 on the evaluation. A criterion that came back partial does not count as failed here, even though the ticket looks like a failure to a reader. * **Customer sentiment never matches.** The field cannot be added from the rule editor under any of the five triggers. If an existing rule contains it, remove it and use a QA score condition instead. > [!TIP] > > When a rule with several conditions is not firing, strip it back to just the > trigger. Save it, confirm it fires, then add conditions back one at a time. Much > faster than reasoning about which combination excluded the ticket. ## The rule fired, but nothing appeared to happen Sometimes the rule matched and the action ran without changing anything. From outside this looks identical to a rule that never fired, but the causes differ, and every one still consumes the event, so the rules below stay skipped. | Action | Quietly does nothing when | | - | - | | **Assign to queue**, **Assign to role**, **Assign to member** | The ticket already has an assignee and is still unresolved. Existing work is not reassigned out from under someone | | **Assign to role** | Nobody in your organization currently holds that role | | **Resolve** | The ticket is not in the unresolved state | | **Send a notification** by email | The trigger is anything other than **Complaint is created**. Email notifications are only supported for complaints | | **Send a notification** to Slack | There is no Slack connection behind the channel you picked | Assignment also picks *one* person at random. If a rule assigns to a role or a set of members, selecting five people spreads tickets across them one at a time rather than notifying all five, so "my rule didn't fire" sometimes turns out to be "it fired and picked someone else". Add a notification action alongside the assignment when everyone needs to see the ticket. ## There is no on/off switch A rule's row menu offers **Edit** and **Delete**, and nothing else. Rules cannot be paused, and there is no enabled column. That makes ordering your only non-destructive lever. To take a rule out of play without losing it, drag it to the bottom of the list, or edit it so its conditions no longer match. To take it out permanently, delete it, and check what was sitting below it first, because those rules will start firing on traffic the deleted rule used to absorb. ## Confirm the fix on the next event Rules apply going forward only. Fixing the order or loosening a condition does not reach back and route tickets that already went through the trigger, so the ticket in front of you needs assigning by hand. To prove the fix, raise the trigger again on a real ticket and check three things: the ticket lands where you expected, the assignee is someone you intended, and a ticket you meant to exclude was in fact left alone. If the ticket still isn't routed, go back to the top of the list. A rule above yours matching first remains the most likely answer. ## Related * [Create a routing rule](/guides/quality-assurance/create-routing-rules) * [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) * [Perform a manual evaluation](/guides/quality-assurance/perform-a-manual-evaluation) * [Roles and permissions](/guides/roles-and-permissions) # Why is this score different from what I expected? Source: https://docs.rulebase.co/guides/troubleshooting/why-is-my-score-different Trace an unexpected QA score from the total down to the individual check, and work out whether a cap, a condition, a scoping rule, or an edited scorecard produced it. An evaluation came back at 62 and you were expecting something in the eighties. Or a criterion your reviewer would have passed came back as a fail. An unexpected score is a number assembled from many smaller decisions. Take it apart in the order Rulebase built it: total, then criterion, then check. If the number that surprised you is a team or agent average, start with [Use team and agent performance](/guides/insights/team-and-agent-performance) instead. The usual culprits there are the period, the roster, and how few reviews the average was built from. ## First, check whether the total is the sum of its parts Most of the time the overall score is just the criteria added together. A few features can override that, and each leaves a visible mark. Start with the info icon beside the ticket's QA score. The breakdown behind it lists every evaluator's score grouped by scorecard, and it is where these show up. | What happened | What you see | Where it comes from | | - | - | - | | **Auto-fail** | `0` with **- Autofail** beside it, even though most criteria passed, and an **Auto-Zero** badge on the evaluation | A criterion where the evaluator chose an **Auto-fail score** option. Auto-fail zeroes the whole evaluation | | **Score cap** | **- Capped** beside the score, with the reason printed underneath | A **Caps** rule on one criterion, reading `If this criteria is fail, cap overall score at 60` | | **Dependency rule override** | A normal-looking total, but a check marked failed with a **Dependency rule override** badge | A check that inherits its outcome from another criterion's result | A capped result looks like a miscalculation until you read the reason line, which spells out both numbers, for example `Score capped at 60 because "Verified the customer" failed. Uncapped score: 88.` If the cap is doing more damage than intended, it lives on the criterion it names, in the **Caps** row of the scorecard editor. ## Then work down to the check that decided it Once the total makes sense, open the criterion you disagree with. A scorecard built from checks puts a count under each criterion result (**4 total checks**) next to a **View checks** button. Expanding it sorts them into four tabs: **Failed checks**, **Partial checks**, **Passed checks**, and **Not Applicable checks**, each carrying its own number. **A criterion scores as low as its worst check.** Rulebase does not average them. A criterion with four checks where three passed and one failed takes the failed check's score, so a criterion worth 10 points can come back at 0 on a single judgement. The inverse is also true: when no check comes back failed or partial, the criterion is awarded a full pass. If the criterion score looks disproportionate to the summary, the **Failed checks** tab usually holds the answer, and the count on each tab tells you how much of the criterion actually ran. ## The condition sets the score A check does not carry a score of its own. When a check fails or partially passes, the evaluator picks one of the **Criteria conditions** sitting under it, and every condition ends in an outcome (`mark as fail`, or partial or auto-fail where the criterion offers them). That outcome sets the number. ![Criterion card showing the criteria name, pass and fail scores, a check description, a criteria condition mapping to fail, and the Generate conditions and Add new check controls](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-criteria-check-editor.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=b1e6acf54dac470bd14f0c72f2c592ce) If every condition on a check maps to **fail**, the check has no way to express "mostly right", and near-misses land on zero. If you expected partial credit and did not get it, look for a partial score option on the criterion and a condition attached to it. The condition text also tells you what the evaluator thought it saw, often more diagnostic than the summary paragraph. ## A check marked Not Applicable did not cost anything **Apply to** narrows a check to particular contexts: specific channels, new or re-assigned tickets, primary or duplicate tickets, SLA breaches, inbound or outbound calls. Scoped-out checks still appear, under **Not Applicable checks**. The evaluator sees the check's conditions and decides whether they hold; scoping does not filter the check out before scoring. The evaluator also marks a check not applicable on its own judgement when the required action was never possible: a call that only reached voicemail cannot have a consent disclosure, and an unanswered message cannot have a back-and-forth. > [!NOTE] > > A not-applicable check contributes nothing to the score. A criterion whose checks > are *all* not applicable is awarded full marks. If a criterion you expected to be > skipped is quietly handing out points, that is why. ## Additional instructions change how a check is judged **Apply to** decides whether a check runs; additional instructions change how it is judged when it does. The exception lives on the check, in its instructions field. ![Scorecard check card showing the Apply to condition, the check description, and an open Additional instructions field containing an exception rule](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-additional-instructions.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=b1a1fd806b736f8b1265561a540390b4) Open the check in the scorecard editor and read the **Additional instructions** field alongside the description. An instruction like `If the customer already apologized, do not fail for missing apology` will make the check behave in a way the description alone never suggests. When a whole section scores oddly, look one level up. A section's **Instructions**, set from its **...** menu under **Edit**, reach every criterion inside it, so a single sentence there can shift several scores at once. ## The rubric may have moved since the ticket was scored An evaluation keeps the score it was given. The check and criterion text shown next to that score is read live from the scorecard as it stands today. Edit a published scorecard and older evaluations start looking self-contradictory: current wording beside a historical result. Also confirm which scorecard you are reading. A ticket that matches more than one published scorecard is scored against each of them separately, and the evaluation header turns into a scorecard picker when that happens. If the criteria you were expecting are absent entirely, check the picker before concluding the result is wrong. You may be looking at the other rubric. > [!TIP] > > If you are about to change a live rubric, **Duplicate** the scorecard, edit the > copy, > publish it, and move the original to drafts. Old evaluations then keep displaying > the wording they were judged against. ## When the score is wrong Everything above explains a score. When the explanation does not hold up and the result is wrong, correct it on the criterion. Thumbs down the criterion, write what the evaluator missed, and tick **Re-evaluate this criterion after submitting** to have it scored again with your note in hand. The overall score updates with the new result, and the correction is retained for later evaluations of similar tickets. If the same criterion keeps coming back wrong across different tickets, the fix belongs in the scorecard instead: usually a condition that is too broad, or an exception that should have been written as an additional instruction. ## Related * [Give feedback on AI QA evaluations](/guides/quality-assurance/give-feedback-on-ai-qa) * [Create a scorecard](/guides/quality-assurance/create-a-scorecard) * [Add scorecard additional instructions](/guides/quality-assurance/add-scorecard-additional-instructions) * [Use team and agent performance](/guides/insights/team-and-agent-performance) * [Why wasn't my ticket evaluated?](/guides/troubleshooting/why-wasnt-my-ticket-evaluated) # Why wasn't my ticket evaluated? Source: https://docs.rulebase.co/guides/troubleshooting/why-wasnt-my-ticket-evaluated Diagnose a missing QA evaluation with the eligibility tester, then work through scorecard scope, agent roster, coverage rules, and timing in the order Rulebase applies them. A ticket closed, you opened it expecting a score, and there is nothing there. This is almost never a bug: Rulebase applies a series of gates before it evaluates, and one said no. It will tell you which. The causes below are in the order the product checks them. ## What the ticket already tells you Open the ticket and look at its **Review** tab. It shows one of these states: * **Evaluation skipped**: Rulebase considered the ticket and decided against it. The text underneath is the reason. This is the most likely state. * **Evaluation scheduled**: queued and waiting, usually on your evaluation delay. **Run now** starts it immediately. * **Evaluation failed**: something went wrong while running. **Try again** is safe. * **No QA evaluations**: no evaluation was ever attempted, which points at a gate that runs before a request is created. Keep reading. ## Get the full picture with the eligibility tester The Review tab shows the first reason; the eligibility tester shows every gate at once, which matters when two things are wrong. 1. Go to [Settings > Evaluations](https://app.rulebase.co/settings/evaluations). 2. Next to **Eligibility instructions**, click **Edit**, then click **Test**. 3. Search for the ticket by ID or subject, and select it. ![Eligibility tester result showing ticket 48220 as not eligible for QA because no eligible agents were found on the conversation](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/eligibility-tester-not-eligible.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=f018c89bfbf3978d745591e36cca097a) The headline tells you whether the ticket is eligible at all. The line underneath is the first blocking reason. Remaining problems appear as badges grouped into **Rules**, **Auto QA**, and **Agents**; hovering a badge shows the full explanation. Some gates block everything; others block only **automatic** QA, so the ticket can still be evaluated on request. When the tester says the ticket is eligible but automatic QA would not have run, it lists those automatic-only blockers under **Auto QA**, and you can request an evaluation by hand without changing any settings. > [!NOTE] > > If the ticket does not appear in the tester's search at all, it has not reached > Rulebase yet. That is an ingestion question. Check your help desk connection under > [Settings > Connections](https://app.rulebase.co/settings/connections). ## Most likely: no scorecard applies to the ticket This is the top cause on new accounts, and the message names it plainly: no published scorecard applies to this conversation. Two things cause it. **The scorecard is still a draft.** Only published scorecards evaluate anything; a draft sits under **Drafts** on [Knowledge > Scorecards](https://app.rulebase.co/knowledge/scorecards) looking finished. ![Scorecards list with one published scorecard under Active and one draft under Drafts](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-list-active-and-drafts.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=a03ddd7adf33f0a8172ac4d8465d7bf7) **Its scope excludes the ticket.** Open the scorecard and read the pills next to **Add scoping**. Every pill narrows what the scorecard covers: one scoped to **Emails** and **Chats** will never touch a phone call, and one scoped to a single team ignores everybody else. ![Scorecard header scoping row with the Add scoping button and Emails and Chats pills applied](https://mintcdn.com/rulebase/KflItiUNDiZr-ZIR/images/scorecard-scoping-pills.png?fit=max\&auto=format\&n=KflItiUNDiZr-ZIR\&q=85\&s=b92615ab5aa8c8a98d9638a3a0f06f46) One special case: if the customer never replied, Rulebase skips the ticket unless a published scorecard opts in to internal tickets. Add **Internal tickets** to a scorecard's **Ticket scope** to score one-sided tickets. ## Then check the agent roster An agent on the ticket counts only when all of these are true: * They are on the QA roster at [Settings > People > Agents](https://app.rulebase.co/settings/people/employees). Someone who arrived through a help desk integration but was never added is not evaluated. * They are a human agent. Automated replies never produce an evaluation. * They wrote at least 100 characters of customer-facing messages. A one-line "sorted, thanks!" handoff is not enough material to score. * A matching scorecard applies to them. Scorecards scoped to a team or group only cover agents in it. When no agent clears all four, the tester reports no eligible agents were found. Fix it by adding the person to the roster, or widening the scorecard's people scope. [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster) covers both. ## Then check whether coverage rules selected it Coverage rules are sampling. If you cover 20% of tickets, the other 80% are skipped on purpose, and the reason says the conversation was not selected by your coverage rules. Nothing is broken. ![Eligibility settings card showing a coverage rule for source channel in email and chat at 20% of tickets, eligibility instructions, a 30-day eligibility window, and the linked tickets toggle](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/eligibility-settings-card.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=f2dde67e28effa44c5a96953a12b52c9) A subtler version: the ticket can be selected while none of its agents are, which happens with rules that cover a fixed number of tickets per agent. The reason then names the agents. Both are deliberate settings decisions. If a specific ticket needs a score anyway, request the evaluation by hand: manual requests ignore coverage rules entirely. ## Or it is still waiting Several gates are about timing. A ticket that trips one may be evaluated later without you doing anything. * The ticket is not closed. Automatic QA runs after a ticket is resolved. * The evaluation delay has not elapsed. The Review tab shows **Evaluation scheduled**, and the eligibility tester names the time it is scheduled for. * An evaluation is already queued or in progress, which blocks a second request on the same ticket. * A linked ticket in the thread is still open, and you have **Wait for all linked tickets to close** turned on. ## Or there was nothing to score The last group of gates is about the conversation itself. Your eligibility instructions drive these, and a skip here is usually correct. Either an agent never sent a customer-facing message, so there is no work to judge; or the AI eligibility check read the transcript and found no real request and no meaningful exchange, which is what the default instructions ask it to filter out. If tickets you care about keep landing here, edit the instructions. See [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility). ## Every check, and what to do about it The tester labels each gate. Use this as the lookup table when a badge is not self-explanatory: | Check | Why it blocked | Where to fix it | | - | - | - | | **Ticket closed** | The ticket is still open | Nothing to do. Automatic QA runs after resolution | | **Agent responded** | No agent sent a customer-facing message | Nothing to fix. There is no agent work to score | | **Evaluation credits** | The organization ran out of evaluation credits | Nothing on the ticket. Tickets are reconsidered once credits are restored, and current usage sits under **Credits** at the bottom of the sidebar | | **Evaluation window** | The ticket started before your eligibility window | Widen or clear **Eligibility window** in evaluation settings | | **Already evaluated** | A completed AI evaluation already exists | Open the existing evaluation, or request a fresh one | | **No blocking evaluation request** | Another evaluation is queued or running | Wait for it to finish | | **Linked tickets closed** | A linked ticket in the thread is open | Close the linked ticket, or turn off the linked-tickets setting | | **Customer responded (or scorecard accepts internal)** | The customer never replied and no scorecard accepts internal tickets | Add **Internal tickets** to a scorecard's ticket scope | | **Scorecard match** | No published scorecard covers this ticket | Publish a scorecard, or widen its scoping | | **Eligible agents** | Nobody on the ticket qualified | Add the agent to the roster, or widen the scorecard's people scope | | **Coverage rules** | Sampling did not select this ticket | Raise coverage, or request the evaluation by hand | | **Agents match coverage rules** | Sampling selected the ticket but not these agents | Adjust the rule's target or how it splits per agent | | **AI eligibility** | The transcript did not meet your eligibility instructions | Edit **Eligibility instructions**, then re-test | ## Once you have fixed the cause Some fixes catch up on their own. When you publish a scorecard, add an agent to the roster, change coverage rules, close a linked ticket, or top up credits, Rulebase re-checks tickets skipped for those reasons about once a day. That sweep only looks at tickets that closed inside the current sampling period (the current day unless your coverage rules use weekly or monthly targets), so recent tickets recover automatically and older ones do not. For the ticket you were investigating, do not wait. Open it and use the button on the **Review** tab: **Try again** on a skipped or failed evaluation, **Request evaluation** where none was attempted. Manual requests skip coverage rules and the AI eligibility filter, so they work while you are still tuning those settings. For the tickets that come next, confirm the fix the same way you found the problem. Run the eligibility tester on a ticket that should qualify and check the result reads **Eligible for QA** with the agents you expect. ![Eligibility tester result showing ticket 48219 as eligible for QA with the summary that all agents on this ticket will be evaluated and a badge for the agent](https://mintcdn.com/rulebase/nRmHnyRJv-otnvt6/images/eligibility-tester-eligible.png?fit=max\&auto=format\&n=nRmHnyRJv-otnvt6\&q=85\&s=a0ed6619e7bcc9899835161b163550ee) ## Related * [Why is this score different from what I expected?](/guides/troubleshooting/why-is-my-score-different), when the evaluation ran but the number looks wrong * [My integration stopped syncing](/guides/troubleshooting/integration-stopped-syncing), when the ticket never reached Rulebase at all * [An agent is missing from reports](/guides/troubleshooting/agent-missing-from-reports) * [Configure evaluation eligibility](/guides/quality-assurance/configure-evaluation-eligibility) * [Create a scorecard](/guides/quality-assurance/create-a-scorecard) * [Manage the QA agent roster](/guides/quality-assurance/manage-agent-roster) * [Give feedback on AI QA](/guides/quality-assurance/give-feedback-on-ai-qa) * [How AI QA works](/guides/concepts/how-ai-qa-works), for the pipeline these gates sit in and the setting behind each one # Confirm received uploads Source: https://docs.rulebase.co/guides/uploads/confirm-received-uploads Check that Rulebase received your conversation uploads by listing them through the API, filtering by source or date, and reading each upload status. Use this guide to list conversation uploads and confirm Rulebase received them. Requests use `https://api.rulebase.co` and require a Rulebase API key in the `Authorization: Bearer ` header. For the upload write path, see [Uploading conversations](/guides/uploads/conversations). ## List conversation uploads Send a `GET /v1/conversation_uploads` request. Filter by the same `source` value you used when uploading. ```bash theme={null} curl "https://api.rulebase.co/v1/conversation_uploads?source=xcally&limit=50" \ -H "Authorization: Bearer YOUR_API_KEY" ``` A successful response returns the newest uploads first: ```json theme={null} { "data": [ { "id": "conversation_upload_abc123", "status": "completed", "source": "xcally", "external_id": "abc123", "created_at": "2025-07-02T12:00:00.000Z", "updated_at": "2025-07-02T12:05:00.000Z" } ], "meta": { "page": { "next": null, "limit": 50 } } } ``` For the full REST spec see [List conversation uploads](/api-reference/conversations/list-conversation-uploads). ## Supported sources Use the same `source` keys as the conversation upload API: | Source key | Description | | - | - | | `outreach_kaia` | Outreach Kaia conversation exports | | `xcally` | XCALLY voice recordings | ## Query parameters | Parameter | Required | Description | | - | - | - | | `source` | No | Upload source key (for example `xcally`) | | `external_id` | No | Exact match on upload metadata `unique_id` when present | | `limit` | No | Page size. Default `50`, maximum `100` | | `cursor` | No | Opaque cursor from `meta.page.next` for the next page | | `updated_after` | No | Inclusive lower bound on `updated_at` (ISO 8601) | | `updated_before` | No | Inclusive upper bound on `updated_at` (ISO 8601) | ## Response fields | Field | Description | | - | - | | `id` | Rulebase upload id | | `status` | Upload status: `pending`, `processing`, `completed`, `failed`, or `expired` | | `source` | Upload source key | | `external_id` | Value from upload metadata `unique_id` when present, otherwise `null` | | `created_at` | When the upload was created | | `updated_at` | When the upload was last updated | `external_id` is available for sources that send `unique_id` in upload metadata (for example XCALLY). It may be `null` for uploads that only carry identifiers inside the archive. ## Pagination By default the response returns up to 50 of the most recent uploads, ordered by `created_at` descending. When more results exist, `meta.page.next` contains a cursor. Pass that value as `cursor` to fetch the next page. # Uploading conversations Source: https://docs.rulebase.co/guides/uploads/conversations Send recorded calls to Rulebase with the presigned URL workflow or a direct upload, and package each archive the way its source expects. This guide describes how to programmatically upload recorded conversations so that Rulebase can ingest and review them. ## Upload methods Rulebase supports two methods for uploading conversations: 1. **Presigned URL workflow** (recommended for large files) - Two-step process using presigned URLs 2. **Direct upload** (simpler for smaller files) - Single request with multipart form data *** ## Method 1: presigned URL workflow The presigned URL workflow consists of **two HTTP requests** and is recommended for larger files: 1. Generate a presigned upload URL (HTTP `POST`). 2. Upload the ZIP archive to the returned URL (HTTP `PUT`). ### 1 · Generate a presigned URL Send a `POST /v1/conversation_uploads/presign` request, passing the **filename** and **source** that describe the ZIP archive you intend to upload. ```bash theme={null} curl -X POST https://api.rulebase.co/v1/conversation_uploads/presign \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "upload": { "filename": "your-filename.zip", "source": "[source]" } }' ``` Successful responses contain the upload **id** and a time-limited **url** to which you must upload the archive: ```json theme={null} { "upload": { "id": "unique_upload_id", "url": "https://presigned-url-to-upload" } } ``` For the full REST spec see [Generate a presigned URL for conversation upload](/api-reference/conversations/generate-a-presigned-url-for-conversation-upload). ### 2 · Upload the archive Upload the ZIP archive with an HTTP `PUT` request to the `url` from step 1. The request **must** use `Content-Type: application/octet-stream`. ```bash theme={null} curl -X PUT "https://presigned-url-to-upload" \ -H "Content-Type: application/octet-stream" \ --data-binary "@your-filename.zip" ``` ⚠️ The presigned URL expires **30 minutes** after it is issued. *** ## Method 2: direct upload For simpler integration and smaller files, you can upload conversations directly using a single `POST` request with multipart form data. ### Upload a conversation directly Send a `POST /v1/conversation_uploads` request with the source, metadata, and call recording as multipart form data. ```bash theme={null} curl -X POST https://api.rulebase.co/v1/conversation_uploads \ -H "Authorization: Bearer YOUR_API_KEY" \ -F "upload[source]=xcally" \ -F "upload[metadata][unique_id]=abc123" \ -F "upload[metadata][type]=inbound" \ -F "upload[metadata][agent]=agent@example.com" \ -F "upload[metadata][caller]=+2348012345678" \ -F "upload[metadata][called]=+2349098765432" \ -F "upload[metadata][recorded_at]=2025-07-02T12:00:00Z" \ -F "upload[file]=@call-abc123.wav" ``` Successful responses contain the upload **id** and processing status: ```json theme={null} { "data": { "id": "unique_upload_id", "status": "processing", "created_at": "2025-07-02T12:00:00Z", "updated_at": "2025-07-02T12:00:00Z" } } ``` ### Direct upload requirements * **Content-Type:** `multipart/form-data` * **Maximum file size:** 100MB * **Supported audio formats:** `.wav`, `.mp3`, `.m4a` * **Metadata format:** Individual form fields for each metadata attribute ### Supported sources for direct upload | Source key | Description | | - | - | | `xcally` | XCALLY voice recordings | ⚠️ Direct upload currently supports XCALLY sources only. For other sources, use the presigned URL workflow. *** ## Supported sources The `source` attribute defines how Rulebase should interpret the contents of the archive. All sources follow the same two-step upload flow shown above, but each imposes its own file-naming and payload requirements. | Source key | Description | | - | - | | `outreach_kaia` | Outreach Kaia conversation exports | | `xcally` | XCALLY voice recordings | Detailed requirements for each source are provided below. *** ### Outreach Kaia (`outreach_kaia`) #### 1 · Presign request example ```bash theme={null} curl -X POST https://api.rulebase.co/v1/conversation_uploads/presign \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "upload": { "filename": "KAIA-2024-03-20T10:00:00Z+instance123.zip", "source": "outreach_kaia" } }' ``` #### 2 · ZIP file requirements * **Filename convention:** `KAIA-+.zip` * **Contents:** | File | Naming convention | | - | - | | Transcript (`.txt`) | `KAIA-+.txt` | | Media (`.mp4`) | `KAIA-+.mp4` | | Metadata (`.json`) | `KAIA-+.json` | #### 3 · Example ZIP structure ``` KAIA-2024-03-20T10:00:00Z+instance123.mp4 KAIA-2024-03-20T10:00:00Z+instance123.txt KAIA-2024-03-20T10:00:00Z+instance123.json ``` For a complete description of the export schema, refer to the official [Outreach Kaia export documentation](https://support.outreach.io/hc/en-us/articles/28005562346395-Exporting-Data-in-Kaia-using-Daily-Export). *** ### XCALLY (`xcally`) #### 1 · Presign request example ```bash theme={null} curl -X POST https://api.rulebase.co/v1/conversation_uploads/presign \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "upload": { "filename": "xcally-2025-07-02-call-abc123.zip", "source": "xcally" } }' ``` #### 2 · ZIP file requirements * **Contents:** | Required file | Notes | | - | - | | Audio (`.wav` or `.mp3`) | Recording of the call | | Metadata (`.json`) | Call attributes (see below) | * **Filename convention:** none. Any archive name is accepted as long as exactly one audio file and one JSON file are present. #### 3 · Example ZIP structure ``` call-abc123.wav call-abc123.json ``` #### 4 · Metadata JSON schema (example) ```json theme={null} { "unique_id": "abc123", "type": "inbound", "caller": "+2348012345678", "called": "+2349098765432", "connected": "ext-1001", "queue": "customer_support", "disposition": "resolved", "recorded_at": "2025-07-02T12:00:00Z" } ``` | Field | Description | | - | - | | `unique_id` | Unique identifier of the call | | `type` | Call type (`inbound`, `outbound`, `internal`, or `dialer`) | | `caller` | Caller phone number | | `called` | Callee phone number | | `connected` | Extension/agent that handled the call | | `queue` | Queue name | | `disposition` | Call disposition label | | `recorded_at` | ISO 8601 timestamp when the call was recorded | For additional reference see the [XCALLY voice recordings documentation](https://helpdesk.xcallyusa.com/dt_articles/voice-recordings/). To confirm Rulebase received uploads after they are processed, see [Confirm received uploads](/guides/uploads/confirm-received-uploads). # Uploading work items Source: https://docs.rulebase.co/guides/uploads/work-items Push back-office work records and their events to Rulebase one at a time or by CSV, and match each one to the right agent. Upload back-office work item records and their event logs: compliance reviews, fraud investigations, payment disputes, and KYC reviews. All work item requests use `https://api.rulebase.co` (or `https://eu.api.rulebase.co` on EU) and require a Rulebase API key in the `Authorization: Bearer ` header. The canonical paths are versioned under `/v1/work_items`. Deprecated aliases continue to work during their sunset period, but new and updated integrations should use the canonical paths shown here. If you are scripting this from an agent or a CI job, the [Rulebase CLI](/guides/cli) wraps every endpoint below with JSON in and JSON out. ## Rate limits Work item endpoints allow about 5,000 requests per minute per organization. Over that, the API responds with `429 Too Many Requests` and a `Retry-After` header giving the seconds to wait. For large volumes, use bulk CSV upload, which carries many rows in one request. ## When to use work items vs conversations | If the work is... | Use | | - | - | | A customer interaction (call, chat, email) | `/conversations` | | Back-office work an agent did in your internal admin tool (no customer on the line) | `/v1/work_items` | Work items preserve the source work record, the responsible employee, customer-specific attributes, and the source-system events. Uploading a work item stores the record and its events; it does not score or evaluate it. ## Upload methods Pick based on how your source system exports data. 1. **Single work item.** `POST /v1/work_items` with a JSON body. For systems that push individual items in real time. It is create-only: a second push of the same `external_id` returns `409`. 2. **JSON batch apply.** `POST /v1/work_items/batch` with up to 100 work items in one request. Each item is created, updated, or restored by `external_id`, so it is the endpoint to use when you re-send records you may have sent before. Preferred for agents and scripted backfills. 3. **Bulk CSV upload.** `POST /v1/work_items/upload` with a multipart CSV of many items. For periodic exports out of a system that speaks CSV. Use apply rather than CSV upload when you already hold structured records: it is synchronous, tells you per item whether it created, updated, restored, or failed, and supports `dry_run`. CSV upload is asynchronous and reports per-file counts you poll for. *** ## Method 1: single work item `POST /v1/work_items` with the work item details as a JSON body: ```bash theme={null} curl -X POST https://api.rulebase.co/v1/work_items \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "work_item": { "external_id": "TASK-9182", "agent_email": "alex@example.com", "agent_name": "Alex Smith", "agent_external_id": "EMP-447", "type": "fraud_review", "status": "completed", "completed_at": "2026-05-12T14:32:00Z", "custom_attributes": { "risk_score": 78 }, "events": [ { "external_id": "EVENT-551", "actor_external_id": "EMP-447", "content": "Reviewed the account activity and approved the refund.", "occurred_at": "2026-05-12T14:31:00Z" } ] } }' ``` A successful response returns the created work item: ```json theme={null} { "data": { "id": "0197ac58-70c9-7bc0-9ba8-1bb7ce8f78cc", "external_id": "TASK-9182", "agent": { "id": "employee_3kV9m2qL7xN4pR8tW1yB6zAa", "name": "Alex Smith", "email": "alex@example.com", "external_id": "EMP-447", "avatar_url": "https://cdn.example.com/agents/alex.png" }, "type": "fraud_review", "status": "completed", "qa_score": null, "completed_at": "2026-05-12T14:32:00Z", "custom_attributes": { "risk_score": 78 }, "events": [ { "id": "0197ac58-70c9-7bc0-9ba8-1bb7ce8f78dd", "external_id": "EVENT-551", "actor": { "id": "employee_3kV9m2qL7xN4pR8tW1yB6zAa", "name": "Alex Smith", "email": "alex@example.com", "external_id": "EMP-447", "avatar_url": "https://cdn.example.com/agents/alex.png" }, "content": "Reviewed the account activity and approved the refund.", "occurred_at": "2026-05-12T14:31:00Z", "created_at": "2026-05-12T14:35:00Z" } ], "created_at": "2026-05-12T14:35:00Z", "updated_at": "2026-05-12T14:35:00Z" } } ``` The response `id` is a **work item** UUID. Fetch it later with [`GET /v1/work_items/{id}`](/api-reference/work-items/get-work-item). That path accepts a work item id only; a bulk upload job id returns `404`. Full REST spec: [Create a work item](/api-reference/work-items/create-work-item). *** ## Method 2: bulk CSV upload Upload a CSV to `POST /v1/work_items/upload` as `multipart/form-data`: ```bash theme={null} curl -X POST https://api.rulebase.co/v1/work_items/upload \ -H "Authorization: Bearer YOUR_API_KEY" \ -F "upload[file]=@work-items-2026-05-12.csv" ``` A successful response returns an **upload job** UUID and a `pending` status: ```json theme={null} { "data": { "id": "0197ac5b-a956-79b6-8ad0-9c735ee58a49", "status": "pending", "total_rows": 0, "succeeded_rows": 0, "failed_rows": 0, "error": null, "created_at": "2026-05-12T14:35:00Z", "updated_at": "2026-05-12T14:35:00Z" } } ``` That `id` is an upload job id. Poll it with [`GET /v1/work_items/upload/{id}`](/api-reference/work-items/get-work-item-upload): ```bash theme={null} curl -i "https://api.rulebase.co/v1/work_items/upload/0197ac5b-a956-79b6-8ad0-9c735ee58a49" \ -H "Authorization: Bearer YOUR_API_KEY" ``` Do not poll `GET /v1/work_items/{id}` with an upload job id. That endpoint looks up a single work item and returns `404` for an upload job. Poll until the upload reaches a terminal status: * `completed`: every row succeeded. * `completed_with_errors`: at least one row failed, while valid rows were still imported. * `failed`: the file could not be processed, such as when required headers are missing or storage is unavailable. Example terminal response: ```json theme={null} { "data": { "id": "0197ac5b-a956-79b6-8ad0-9c735ee58a49", "status": "completed_with_errors", "total_rows": 968, "succeeded_rows": 885, "failed_rows": 83, "error": null, "created_at": "2026-05-12T14:35:00Z", "updated_at": "2026-05-12T14:35:11Z" } } ``` Invalid rows do not block valid rows. Retrying an upload job skips rows that already succeeded, and duplicate `external_id` values resolve to the same work item. ### Bulk upload requirements * **Content-Type:** `multipart/form-data` * **File format:** CSV with a header row * **Header row required;** column order is flexible * **Maximum file size:** 25 MB Full REST spec: [Upload work items in bulk](/api-reference/work-items/upload-work-items). *** ## Work item schema reference | Field | Required | Type | Description | | - | - | - | - | | `external_id` | Yes | string | Unique identifier for this work item in your system. Used to deduplicate re-uploads. | | `agent_email` | No (recommended) | string | Work email of the agent. Prefer providing email or external ID when available. | | `agent_external_id` | No (recommended) | string | Stable employee identifier from your system. | | `agent_name` | No | string | Display name stored on the employee record. | | `type` | No | string | Customer-defined work item type. New work items default to `unknown` when omitted. | | `status` | No | string | Work item status. One of `pending`, `in_progress`, `completed`, or `cancelled`. New work items default to `pending` when omitted. | | `completed_at` | Required when `status` is `completed` or `cancelled` | string | ISO 8601 timestamp for when the work item reached a terminal state. Sending `pending` or `in_progress` without a timestamp clears a prior completion date. | | `custom_attributes` | No | object | Free-form key/value pairs for anything else, such as outcome or risk category. | | `events` | No | array | Source-system events associated with the work item. | Customer-specific outcomes and metadata go in `custom_attributes`. The keys `type`, `status`, and `priority` are reserved and not stored inside `custom_attributes`; send `type` and `status` as top-level fields. Incomplete imports (for example notes-only exports) may omit agent, type, and status. Rulebase stores stubs so events can attach; UI shows **Unknown agent** until you backfill. AI QA scorecards will not apply until an agent is present. ### How agents are matched On receiving a work item, Rulebase looks up the agent against your existing employee records: 1. **Match by `agent_external_id`** if provided. 2. Otherwise, **match by `agent_email`**. 3. If neither matches and an identifier was provided, a new employee is created from the fields you sent. 4. If neither agent field is provided, the work item is stored without an agent. Batch apply and CSV upload resolve an existing work item by `external_id` within your organization (`PATCH /v1/work_items/{id}` instead targets the Rulebase work item ID, and cannot change `external_id`). Re-sending the same `external_id` refreshes provided fields from the new payload; omitted/null agent, type, and status preserve existing values so a thin re-upload does not erase a prior backfill. Custom attributes are merged: the new upload wins on shared keys, and keys only on the stored record are kept. There is no separate employee roster API. You enrich employee records through the agent objects you carry on the work items you push. Name alone will not match or create an agent. Provide `agent_email` or `agent_external_id`; `agent_name` only enriches the matched employee. *** ## CSV column reference CSV uploads use flat columns that map onto the JSON schema. | Column | Required | Maps to | | - | - | - | | `ID` | Yes | `external_id` | | `Agent email` | No (recommended) | `agent_email` | | `Agent external ID` | No (recommended) | `agent_external_id` | | `Agent name` | No | `agent_name` | | `Completed at` | Required when Status is `completed` or `cancelled` | `completed_at` | | `Type` | No | `type` | | `Status` | No | `status` | | `custom_` | No | `custom_attributes.` | `Status` must be one of `pending`, `in_progress`, `completed`, or `cancelled`, and is normalized to lowercase. On a new work item, blank Status stores `pending` and blank Type stores `unknown`; on a re-upload, blank Status and Type preserve their stored values. `Completed at` is required when Status is `completed` or `cancelled`. A blank completion date with `pending` or `in_progress` clears a prior completion date, while omitting both the Status and Completed at columns preserves it. Legacy snake\_case headers such as `external_id`, `agent_email`, and `completed_at` are also accepted. Any column prefixed with `custom_` is flattened into `custom_attributes`. For example, `custom_risk_score` becomes `custom_attributes.risk_score`. The reserved columns `custom_type`, `custom_status`, and `custom_priority` are ignored. ### Example CSV ```csv theme={null} ID,Agent email,Agent name,Completed at,Type,Status TASK-9182,alex@example.com,Alex Smith,2026-05-12T14:32:00Z,fraud_review,completed TASK-9183,sam@example.com,Sam Patel,,kyc_verification,pending TASK-9184,alex@example.com,Alex Smith,2026-05-12T15:02:00Z,account_closure,cancelled ``` *** *** ## Method 2: JSON batch apply `POST /v1/work_items/batch` takes 1-100 work items in a `work_items` array and applies them in order: ```bash theme={null} curl -X POST https://api.rulebase.co/v1/work_items/batch \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "work_items": [ { "external_id": "TASK-9182", "agent_email": "alex@example.com", "type": "fraud_review" }, { "external_id": "TASK-9183", "type": "kyc_verification", "custom_attributes": { "risk_score": 12 } } ] }' ``` The response keeps your input order and reports each item separately: ```json theme={null} { "data": { "dry_run": false, "results": [ { "index": 0, "external_id": "TASK-9182", "status": "updated", "data": { "id": "0197ac58-70c9-7bc0-9ba8-1bb7ce8f78cc" } }, { "index": 1, "external_id": "TASK-9183", "status": "created", "data": { "id": "0197ac58-8f10-7a41-bd21-4a6d54c1e900" } } ], "summary": { "total": 2, "created": 1, "updated": 1, "unchanged": 0, "restored": 0, "error": 0 } } } ``` * Statuses are `created`, `updated`, `unchanged`, `restored`, or `error`. * A failed item does not roll back the items that succeeded; check `summary.error`. * Work items you leave out of the request are never deleted. * Sending an `external_id` that was deleted restores it and reports `restored`. * Set `"dry_run": true` to get the same result shape without writing anything. Full REST spec: [Apply a batch of work items](/api-reference/work-items/apply-work-items). *** ## Updating and deleting `PATCH /v1/work_items/{id}` updates a stored work item. Omitted fields keep their values, `custom_attributes` merge with the stored ones, and `external_id` cannot be changed. ```bash theme={null} curl -X PATCH https://api.rulebase.co/v1/work_items/0197ac58-70c9-7bc0-9ba8-1bb7ce8f78cc \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "work_item": { "status": "completed", "completed_at": "2026-05-12T14:32:00Z" } }' ``` `DELETE /v1/work_items/{id}` soft-deletes a work item. It disappears from reads and lists and stops being summarized, while existing evaluations that reference it keep resolving. The call is idempotent: deleting an already-deleted work item still returns `200` with `"deleted": false`. There is no hard delete and no bulk delete. To bring a work item back, apply its `external_id` again. *** ## Listing work items `GET /v1/work_items` returns work items newest-completion-first with cursor pagination: ```bash theme={null} curl "https://api.rulebase.co/v1/work_items?limit=50&status=completed" \ -H "Authorization: Bearer YOUR_API_KEY" ``` Filters: `external_id` (exact match), `status` (repeatable), `completed_after` and `completed_before` (ISO 8601), and `query` (substring match on external ID and type). Pass the returned `meta.page.next` value back as `cursor` to fetch the next page; it is `null` on the last page. Deleted work items are excluded. *** ## Retrieving a work item Fetch a work item by its Rulebase work item id (from `POST /v1/work_items` or a successful import row) with [`GET /v1/work_items/{id}`](/api-reference/work-items/get-work-item). For bulk upload progress, use [`GET /v1/work_items/upload/{id}`](/api-reference/work-items/get-work-item-upload) with the upload job id. ```json theme={null} { "data": { "id": "0197ac58-70c9-7bc0-9ba8-1bb7ce8f78cc", "external_id": "TASK-9182", "agent": { "id": "employee_3kV9m2qL7xN4pR8tW1yB6zAa", "name": "Alex Smith", "email": "alex@example.com", "external_id": "EMP-447", "avatar_url": "https://cdn.example.com/agents/alex.png" }, "type": "fraud_review", "status": "completed", "qa_score": null, "completed_at": "2026-05-12T14:32:00Z", "custom_attributes": { "risk_score": 78 }, "events": [], "created_at": "2026-05-12T14:35:00Z", "updated_at": "2026-05-12T14:51:00Z" } } ``` `qa_score` is `null` until the work item is evaluated; after a QA evaluation is saved, it holds the current score from `0` to `1`. ## Adding work item events Send events inline when creating a work item, or add them later with `POST /v1/work_items/{id}/events`. Rulebase upserts events by `external_id` within your organization. ```bash theme={null} curl -X POST https://api.rulebase.co/v1/work_items/0197ac58-70c9-7bc0-9ba8-1bb7ce8f78cc/events \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "event": { "external_id": "EVENT-552", "actor_email": "alex@example.com", "content": "Submitted the review.", "occurred_at": "2026-05-12T14:32:00Z" } }' ``` Only `external_id` is required. Actor identity, raw `content`, and `occurred_at` are optional. Actors use the same email/external-ID matching behavior as work item agents. `GET /v1/work_items/{id}` returns events in chronological order.