AI Evaluation Framework Builder

Runs in your browser

Use this when: planning repeatable evaluations with representative cases, criteria, results and human sign-off—not just informally reviewing a draft.

WHAT YOU'LL ACHIEVE

Build an editable evaluation plan with weighted criteria where valid, representative cases, thresholds, limitations and human sign-off gates.

THE TOOL: EVALUATION PLAN

Define what you need to learn.

Scores summarise your own evidence against your own criteria. They do not establish safety, compliance or launch readiness.

Evaluation purposeRequired before the plan is complete.Use case and intended usersScope boundariesPass threshold (%)Example default: 80%. Set a justified threshold for this evaluation.

CRITERIA AND WEIGHTING

Score against explicit criteria.

Example dimensions are starting points. Scores are 0–5; weights are non-negative percentages and must total exactly 100%. Every dimension needs a score before a weighted result appears.

Dimension 1

DimensionWeight (%)Score (0–5)Criterion / evidence rule

Dimension 2

DimensionWeight (%)Score (0–5)Criterion / evidence rule

Dimension 3

DimensionWeight (%)Score (0–5)Criterion / evidence rule

Weighted score unavailable until every score is 0–5 and weights total exactly 100%. Threshold assessment unavailable: resolve the planning validation issues below.

REPRESENTATIVE TEST CASES

Capture expected and observed results.

Use examples that reflect expected use, edge cases and known failure modes. Record evidence references, not sensitive source data.

No test cases yet. Add representative examples before interpreting a score.

HUMAN REVIEW AND LIMITATIONS

Keep evidence and sign-off separate.

Human reviewer / accountable roleReview dateHuman sign-off statusChoosing a status records your entry; it does not create or verify approval.Limitations and unresolved questions

Plan needs attention:

Add the evaluation purpose.
Describe the use case and intended users.

For broader evaluation planning, see the AI Agent Testing Checklist and Compare Two Drafts.

AI EVALUATION FRAMEWORK

Purpose: Not specified
Use case / users: Not specified
Scope and limitations: Not specified
Pass threshold: 80% (user-set; a planning threshold, not a quality guarantee)
Human reviewer: Not assigned
Review date: Not set
Human sign-off: Unknown
Known limitations: Not recorded

DIMENSIONS AND WEIGHTED SCORES
Accuracy weight 40%; score not scored/5; criterion: Define what counts as correct and how claims are verified.
Safety weight 35%; score not scored/5; criterion: Specify unacceptable failures and escalation behaviour.
Usefulness weight 25%; score not scored/5; criterion: Describe what makes the result usable for the intended task.
Weight total: 100%
Weighted score unavailable until every score is 0–5 and weights total exactly 100%. Threshold assessment unavailable: resolve the planning validation issues below.

PLAN VALIDATION
- Add the evaluation purpose.
- Describe the use case and intended users.

REPRESENTATIVE TEST CASES
No test cases added.

HUMAN REVIEW: Not assigned; sign-off status: Unknown. A threshold result is not an automated approval. Evaluation is limited by the selected cases, criteria, evidence and assumptions. This tool makes no legal or compliance determination. Inputs remain in this browser session only.