How a care pathway is graded, tested and approved

This page states what each mark on a CarePathIQ care pathway means and what it takes to earn it. The software sets each mark from the pathway’s own records. An author cannot type a badge in, and AI cannot record any decision that a badge depends on.

A badge records what was done for that version. It does not show that a pathway improves patient outcomes. A step that did not run shows no badge, and a missing badge is not a failing grade.

To cite this page: CarePathIQ. Criteria for Validation badges, evidence grading and approval. Version 2026-11-09. https://carepathiq.org/criteria

Evidence grading

The Library shows an Evidence check when every recommendation in the pathway is graded and cited. Grading follows published frameworks, not rules of our own.

Risk of bias

Each study is judged with the published tool that fits its design:

Tool	Used for
RoB 2	Randomized trials
QUADAS-2	Diagnostic accuracy and decision rules
QUIPS	Prognostic cohorts
ROBINS-I	Non-randomized comparisons of actions
PROBAST	Prediction models

Every domain is rated low, high or unclear, and an item the study does not report counts as unclear. A study is low risk when every domain is low, high when any domain is high, and unclear otherwise. For the GRADE certainty rating, low is no concern, unclear or high is a serious concern, and high in half or more of the domains is very serious. Reviews, case series, case reports and expert opinion take no tool.

Certainty of evidence

Certainty is rated with the GRADE approach (high, moderate, low, very low) across five domains: risk of bias, inconsistency, indirectness, imprecision and publication bias. A difference in population counts as indirectness only when it would likely change the result, and the author decides each population flag. A flag is never raised on race or ethnicity. Each recommendation’s certainty is rated per outcome across the studies that report it and set by the lowest-rated critical outcome. Where the authors depart from that, the evidence profile gives their reason.

Strength of recommendation

Each recommendation carries a GRADE strength, strong or conditional, with its direction (for or against). Where a Level of recommendation is given, it is computed from the Class of Evidence of the supporting studies: Level A needs at least one Class I study or multiple consistent Class II studies; Level B, at least one Class II study or multiple consistent Class III studies; Level C, Class III evidence or expert consensus.

AI may propose a tool, a domain rating or a grade with a reason. The author accepts or changes each one, and what AI proposed is kept beside the author’s choice in the Approval Record.

Validation badges

Three badges, in the order the AI Studio runs them.

Consensus-recorded

Earned when at least one modified Delphi panel round has closed on answers collected online in CarePathIQ.

• Panelists answer “Include this recommendation as written?” on a 1 to 5 scale, and rate strength with its direction. Certainty is fixed from the evidence review.
• The agreement threshold is set before Round 1, 80% unless the author sets another. A recommendation is included when at least that share answers 4 or 5, and left out when at least that share answers 1 or 2.
• There are at most two rounds. Round 2 rates only what Round 1 left open, and anything still open is recorded as no consensus.
• Answers are anonymous to fellow panelists. The panel is reported by role, and results only in aggregate.
• The author of record chooses the panel, or a named role approves the roster. CarePathIQ does not curate or vet panelists.
• AI has no seat on the panel.

Case-stressed

Earned when at least one synthetic case has been walked through the pathway in the Case Simulator (beta) and scored. A case scores only against a source route that a person confirmed; routes drafted by AI are never scored.

The Library shows how many cases matched the source route, of how many were scored, and by how many raters. It is a count, not a pass mark. When the case set was written for a different setting, the badge says so. The result measures agreement of the pathway’s route with the expected route on synthetic cases; it is not a measure of clinical accuracy.

Frontline-tested

Earned when at least one frontline clinician has answered the Frontline survey sent from the AI Studio, after using the pathway. The survey covers usability (System Usability Scale), workload (NASA Task Load Index) and friction items. CarePathIQ counts the answers itself; the count cannot be typed in. Versions published before tracked surveys may read “author-reported”.

A version page also lists, under Adapted by, the published pathways that were built from it.

Approved

After the panel’s rounds close, the department’s final approvers, listed by role (for example a department chair or medical director), each receive their own form. For every recommendation they record Approve, Approve with a condition, or Return to the panel with a reason.

A version is Approved when every recommendation the panel agreed to include is approved, or approved with a condition, by every listed approver, and no recommendation returned to the panel is still open.

The Approved badge on the Library page is planned. Until it is built, each approver’s decisions are recorded in the Approval Record, by role.

How the Approval Record is kept

• Every decision in the AI Studio is saved as a record entry: what was chosen, the role that decided it (never a person’s name), when, and how it was entered. Where AI, a scoring engine or the search recommended something else, that recommendation is kept beside it.
• AI never decides a gate: the scope, the panel and its results, the approvers and their decisions, frontline responses and Publish are recorded by people or tracked from their answers.
• Each AI proposal notes the model version that served it and a fingerprint of the prompt template.
• The record is saved with the version at Publish. A published version, its page, its Pathway Paper and its Approval Record never change. Any change makes a new version with its own identifier, and an older version names the newer one.
• Each version shows the date its evidence was reviewed through. Where the author saved the search, it is re-run on PubMed every three months and new records go to the authors. If they find no change is needed, that decision is recorded and the date moves forward.

When a clinician relies on a pathway

Four conditions, stated in each version’s record and in its Pathway Paper:

1. A defined scope. The condition, setting and population, registered before the first evidence search. A question outside that scope is not covered.
2. An evaluation rung reached. On the four-level ladder in Worth et al. 2026 (knowledge benchmark, simulated scenarios, retrospective real data, prospective trial), a Case-stressed pathway has reached rung 2, simulated scenarios. No claim is made above the rung reached. Consensus and frontline testing are clinician review, not rungs.
3. Outputs a clinician can evaluate. Each step links to its evidence, with its certainty and strength.
4. Changes only through a recorded process. The Approval Record, a new version for any change, and the date the evidence was reviewed through.

Changes to these criteria

A change to these criteria gets a new version date here. A badge names the version it was earned on and is read against the criteria in force on that date.

• 2026-11-09: first version.

See also Security and data and the Terms of Use.