eval-harness-first
Build the evaluation harness that gates every fine-tuning run — golden sets, per-failure-mode graders, judge calibration, and base-model baselines. Use when starting a fine-tuning effort, when converting traces into an eval set, or when calibrating a judge against human labels.
| Field | Value |
|---|---|
| Owner | wshobson |
| Curator | Thomas Cherickal |
| Source | wshobson/agents |
| Repo stars | 42,512 |
| License | MIT |
| Repo last push | 2026-11-09 |
Links
- Roles: ai-engineer
- Source repo: wshobson/agents
- Same repo: evaluation-methodology, dataset-curation, finetuning-method-selection, grpo-rlvr-training, lora-qlora-recipes, python-anti-patterns, python-background-jobs, python-code-style
- Registry seed skills for this role: ai-engineer seed skills