type skillowner Thomas Cherickalcurator Thomas Cherickalrole llm-engineercategory ai-ml/llm-systemsversion 0.1.0status seedupdated 2026-11-15
finetune-data-prep
Steps
- Validate schema of every record.
- Deduplicate exact and near-duplicate examples.
- Scan for PII and secrets; remove or mask.
- Check label and length distributions.
- Split train/validation with no leakage.
Done When
- [ ] No duplicates across splits
- [ ] PII scan report attached
Registry
| id |
role |
version |
status |
owner |
last_tested |
| SK-LLM-ENGINEER-finetune-data-prep |
llm-engineer |
0.1.0 |
seed |
Thomas Cherickal |
- |
Links