Eval Leakage Audit · AlexZio00/sovereign-skills
Check whether an eval, benchmark, or holdout is circular
Reviews an eval, metric, benchmark, or holdout using an 18-pattern taxonomy to check whether it draws on independent external evidence or is just the designer, model, and scorer confirming each other. Useful before trusting a suspiciously clean result.
Good for
- Spot circular logic in an A/B test
- Check if a benchmark score is self-confirming
- Review a holdout set for independent ground truth
- Source repository
- AlexZio00/sovereign-skills
- Category
- General
Open-source skills are maintained by their authors and listed as published, with attribution. Results depend on how well the skill fits your task and material.
A good place to start
Can you audit this eval for leakage and tell me if the benchmark is actually circular?
Make your next great thing.
Bring a question, a file, or an idea that’s not quite there yet.