Rubric design validation

Skill a5c-ai/babysitter/library/specializations/domains/social-sciences-humanities/education/skills/rubric-design-validation

Babysitter enforces obedience on agentic workforces and enables them to manage extremely complex tasks and workflows through deterministic, hallucination-free self-orchestration

Install

npx -y skills add a5c-ai/babysitter --skill rubric-design-validation

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

What its author says it does

Copied from the file, not written here

Develop clear scoring rubrics with defined criteria, performance levels, and anchor examples ensuring inter-rater reliability

SKILL.md

2.7 KB, as published. Nobody here has run it

Rubric Design and Validation

Develop clear scoring rubrics with defined criteria, performance levels, and anchor examples ensuring inter-rater reliability.

Overview

This skill enables the development and validation of scoring rubrics for educational assessment. It encompasses criteria definition, performance level articulation, anchor example selection, and reliability validation to ensure consistent and fair evaluation of student work.

Capabilities

Criteria Development

Identify essential performance dimensions
Define observable indicators
Weight criteria appropriately
Ensure comprehensiveness
Avoid overlap between criteria

Performance Level Definition

Articulate distinct levels
Write clear descriptors
Ensure progressive differentiation
Define score points
Create level labels

Anchor Examples

Select representative samples
Document exemplars for each level
Annotate scoring rationale
Create training materials
Validate with raters

Reliability Validation

Conduct inter-rater reliability studies
Calculate agreement statistics
Identify scoring inconsistencies
Revise for clarity
Train and calibrate raters

Usage Guidelines

Rubric Development Process

Define purpose and use
Identify assessment criteria
Describe performance levels
Draft rubric descriptors
Select and annotate anchors
Validate with multiple raters
Revise based on feedback

Descriptor Writing

Use concrete, observable language
Avoid vague qualifiers
Ensure parallel structure
Include critical attributes
Distinguish adjacent levels clearly

Validation Process

Select diverse raters
Provide calibration training
Score common samples
Calculate reliability statistics
Revise unclear criteria

Integration Points

Related Processes

Rubric Development and Validation
Summative Assessment Development
Formative Assessment Design

Collaborating Skills

assessment-item-development
learning-objectives-writing
quality-assurance-review

References

Stevens and Levi rubric design
Brookhart rubric guidelines
Inter-rater reliability methods
Performance assessment standards