To build a grading rubric with AI, begin with the assignment's approved objectives and the observable evidence learners should produce. Choose an analytic or holistic structure, separate independent criteria, write performance levels that describe evidence rather than vague quality labels, and test the rubric on de-identified or synthetic examples. Teachers must calibrate scoring, resolve disagreements, decide final grades, and provide a review or appeal route. The same applies to scoring videos or recorded presentations in a standardized way: AI can help draft observable criteria, but consistent scores come from a calibrated rubric and teacher review, not from the model alone.

This guide is about evaluation design and scoring consistency. It does not repeat making quizzes with AI, and it does not authorize automatic grading. The general AI guide provides the broader safety model; here, AI is a drafting assistant for a teacher-owned measurement tool.

Key Takeaways

Map every criterion to an approved learning objective.

Score observable evidence, not personality, effort guesses, or polished style unrelated to the task.

Choose analytic or holistic design deliberately.

Describe meaningful differences between performance levels.

Review weights for double-counting and irrelevant barriers.

Calibrate with examples before consequential scoring.

Keep final grades, feedback, reconsideration, and appeals under human control.

Freeze the assignment and learning goals

A rubric cannot repair an ambiguous assignment. Preserve the exact prompt, approved learning objectives, required deliverable, permitted resources, constraints, submission mode, and any existing grading policy. If the assignment changes, version the rubric too.

Create an alignment header:

Field	Record
Assignment	Exact version and date
Learning objectives	Verified identifiers and wording
Required evidence	What learners must show or produce
Constraints	Length, tools, collaboration, sources, format
Exclusions	Traits that must not affect the score
Decision owner	Teacher or authorized assessment lead

Carnegie Mellon University's rubric guidance explains that rubrics articulate expectations, describe levels of quality, and can support consistent assessment and feedback.［1］ The first task is therefore alignment, not generating attractive descriptors.

What counts as observable evidence?

For each objective, ask what a reviewer can actually see or hear in the submission. Evidence might be an accurate explanation, a justified method, a working procedure, relevant source use, a defensible interpretation, or a demonstrated skill.

Avoid proxies unless they are genuinely part of the objective. Page count does not prove depth. Complex vocabulary does not prove reasoning. Visual polish does not prove factual accuracy. Confidence during a presentation does not necessarily prove subject knowledge.

Write evidence statements before criteria:

Objective	Observable evidence	Not evidence by itself
Support a claim	Relevant evidence is selected and connected to the claim	Number of citations
Apply a method	Steps are appropriate, visible, and checked	A correct final answer alone
Communicate findings	Structure makes reasoning understandable to the audience	Decorative design

AI can suggest missing evidence types, but the teacher approves the mapping.

Should you use an analytic or holistic rubric?

An analytic rubric scores separate criteria and gives targeted feedback. It is useful when learners and scorers need to see distinct strengths, weaknesses, or objective mappings. It takes longer to design and score, and poorly separated criteria can double-count the same evidence.

A holistic rubric assigns one overall level from an integrated description. It may suit rapid judgments or work whose qualities are tightly connected. It provides less diagnostic detail and can hide scorer differences.

Do not choose analytic simply because a table looks rigorous. Do not choose holistic simply because it is shorter. Record the decision, intended use, number of scorers, feedback need, and consequences.

How do you keep criteria independent?

Draft a short list of criteria, each representing one meaningful dimension. Test each pair: could a submission perform strongly on one and weakly on the other? If not, they may overlap.

Common problems include scoring “analysis,” “critical thinking,” and “quality of reasoning” separately without distinctions; including mechanics inside every criterion; or awarding presentation points that duplicate communication.

For each criterion, complete:

Field	Question
Criterion	What dimension is judged?
Objective mapping	Which approved objective does it measure?
Evidence	What observable features support the judgment?
Exclusions	What irrelevant traits must not influence it?
Weight	Why does this dimension have this consequence?

Remove criteria that cannot be traced to the assignment or policy.

Write evidence-based rubric performance levels

Labels such as “excellent,” “good,” “fair,” and “poor” do not tell a scorer what to observe. Describe how evidence changes across levels. Use distinctions in accuracy, completeness, relevance, reasoning, independence, transfer, or consistency only when they fit the objective.

Example:

Criterion: use of evidence	Level 4	Level 3	Level 2	Level 1
Descriptor	Selects relevant evidence, explains its connection to each main claim, and addresses a material limitation	Selects relevant evidence and connects it to main claims; one connection or limitation is incomplete	Includes some relevant evidence but connections to claims are asserted or inconsistent	Provides little relevant evidence or does not connect it to claims

Avoid counting minor errors as an automatic level unless the count has a defensible relationship to learning. Avoid descriptors that merely add intensifiers: “very clear,” “mostly clear,” “somewhat clear.”

Write adjacent levels so a scorer can explain why the evidence fits one better than the other. Ask AI to identify undefined adjectives, compound descriptors, impossible standards, and level gaps.

How do you check weights and irrelevant factors?

Weights communicate priorities. Map them to instructional time and objective importance, not to what is easy to count. Confirm that combined criteria do not score the same error multiple times.

Carnegie Mellon University's inclusive grading guidance recommends transparent expectations, alignment, attention to bias, and grading practices that support learning.［2］ Review whether language fluency, handwriting, speed, formatting, accent, access to technology, or prior cultural knowledge affects a score when it is not part of the objective.

Name excluded features explicitly. If mechanics matter only in a writing objective, do not let them lower content scores elsewhere. If timeliness is governed by a separate policy, do not hide it in every descriptor.

CAST's UDL Guidelines encourage multiple means of engagement, representation, and action and expression.［3］ Where the response mode is not itself the target, allow valid alternatives that produce equivalent evidence. Formal accommodations remain under established educator processes, not AI.

Build the rubric table

Use a complete working table rather than only learner-facing descriptors:

Criterion	Objective	Evidence	Level 4	Level 3	Level 2	Level 1	Weight	Exclude	Examples/calibration
Evidence and reasoning	OBJ-2	Claims linked to relevant sources	Specific descriptor	Specific descriptor	Specific descriptor	Specific descriptor	35%	Citation count alone; prose style	Sample IDs and notes

Keep internal calibration notes separate if they would reveal protected student information or answers. The published rubric should still be clear enough that learners understand expectations before submitting.

Build a grading rubric with AI using a bounded prompt

Draft a rubric from the supplied assignment, objectives, and evidence map.
Do not add objectives or grade student work.
For each criterion return: objective mapping, observable evidence,
four distinct performance descriptors, proposed weight with rationale,
and irrelevant features to exclude.
Flag overlap, vague adjectives, compound descriptors, and missing levels.
Mark unsupported assumptions as VERIFY.
Final grades and accommodations remain human decisions.

Review every generated descriptor against the actual task. AI may invent requirements, overvalue surface polish, or produce symmetrical language that does not represent meaningful learning differences.

Why calibrate the rubric with examples before scoring?

Before consequential scoring, apply the draft rubric to a deliberately varied set of de-identified prior samples, authorized exemplars, or synthetic examples. Do not upload identifiable student work to an unapproved model. Human scorers should score independently before discussing.

Record calibration:

Sample	Criterion	Scorer A	Scorer B	Evidence cited	Difference	Resolution/rubric change
S-03	Evidence	3	2	Paragraphs 2 and 4	“Explains connection” interpreted differently	Add boundary example; rescore all calibration samples

Look for recurring disagreement, criteria that cannot be applied, descriptors that collapse into the same level, and examples where irrelevant traits drive scores. Revise the rubric, then repeat calibration. Do not merely average disagreements; understand their cause.

Set scoring and disagreement rules

Define whether scores are criterion-by-criterion, whether boundaries can use professional judgment, how totals are rounded, how missing evidence is handled, and when a second scorer is required. Document who resolves disagreement and whether rescoring applies to all comparable work.

AI can summarize de-identified calibration notes, but it must not decide which scorer is correct or assign the final grade. A model score is not an independent human scorer.

Publish expectations and preserve review

Give learners the assignment and rubric before submission whenever possible. Explain criteria, weights, examples, permitted tools, attribution rules, and the process for questions. If learners may use AI, define acceptable help and verification; see responsible AI homework use.

During grading, cite observable evidence in feedback. Preserve a route for clarification, reconsideration, or appeal consistent with institutional policy. The teacher or authorized body makes the final decision and documents any rubric exception.

UNESCO's guidance for generative AI in education emphasizes human agency, ethical use, privacy, and validation.［4］ Automatic final grading without meaningful teacher review conflicts with this workflow.

Monitor the rubric after use

After a scoring cycle, review criterion distributions, missing evidence, frequent questions, scorer disagreements, appeal themes, accessibility barriers, and whether the rubric encouraged the intended learning. Patterns are signals for investigation, not proof that a learner group or criterion is defective.

Version changes. Do not silently alter descriptors while submissions are being scored. If a flaw requires an immediate correction, document the decision and apply it consistently to affected work.

Final quality checklist

The assignment and objectives are frozen and verified.
Each criterion maps to observable evidence and an objective.
Criteria are distinct and do not double-count the same feature.
Every level contains observable, adjacent distinctions.
Weights reflect learning priorities and have a rationale.
Irrelevant traits and access barriers are explicitly excluded.
Calibration used authorized de-identified or synthetic examples.
Scorer disagreements produced documented resolutions.
Learners received the rubric and review route in advance.
AI did not receive identifiable work or determine final grades.
Any factual or policy suggestion was checked against its source.

FAQ

What is a rubric in AI-assisted grading?

A rubric is a scoring tool that states the expectations for a piece of work as criteria and performance levels.［1］ In AI-assisted grading, the rubric is the written standard a model may be asked to apply; vague criteria produce equally vague output. In this workflow, AI only helps draft the rubric, and teachers apply it.

Can AI grade assignments with this rubric?

Not in this workflow. AI may help draft the rubric, but teachers inspect evidence, apply judgment, provide feedback, and decide final grades.

Can AI score video submissions with a standardized rubric?

Videos and recorded presentations still need criteria tied to observable evidence, such as a demonstrated step or an explained choice. Rubrics help keep scoring consistent across time and graders.［1］ That consistency still depends on calibrated human scorers. Do not upload identifiable student recordings to an unapproved tool, and keep final scores with the teacher.

What is the difference between analytic and holistic rubrics?

Analytic rubrics score separate criteria; holistic rubrics assign one integrated level. Choose based on evidence, feedback needs, and scoring context.

How many criteria should a rubric contain?

Use the fewest distinct criteria needed to represent the approved objectives. Too many can fragment judgment and double-count evidence.

Are labels such as excellent and good enough?

No. Each level needs observable descriptors that explain what evidence distinguishes it from adjacent levels.

Should grammar always be a criterion?

Only when language mechanics are part of the learning objective or stated assignment requirement. Otherwise they may be an irrelevant barrier.

Can I upload student samples for calibration?

Use authorized, de-identified samples or synthetic examples in an approved environment. Do not upload identifiable student work to a general AI tool.

What if two graders disagree?

They should cite evidence, identify the ambiguous descriptor, resolve it through the defined process, revise if needed, and recalibrate.

When should learners see the rubric?

Before submission whenever possible, with criteria, weights, examples, tool rules, and the reconsideration process explained.

Sources

Carnegie Mellon University, “Creating and Using Rubrics” — https://www.cmu.edu/teaching/assessment/assesslearning/rubrics.html
Carnegie Mellon University, “Inclusive Grading and Feedback” — https://www.cmu.edu/teaching/designteach/inclusiveexcellence/gradingandfeedback/index.html
CAST, “UDL Guidelines 3.0” — https://udlguidelines.cast.org/
UNESCO, “Guidance for Generative AI in Education and Research” — https://www.unesco.org/en/articles/guidance-generative-ai-education-and-research

Sources checked 4 October 2026.

How to Build a Grading Rubric with AI