skillZs
★ LIVE SKILL TAGS ★
>>> LIVE SKILLS INDEX <<<
* OPEN SOURCE *
NO LOGIN, NO TRACKING
※ REAL INSTALL DATA ※
← back to all skills
jurgendn/agent-skills69 installs

model-eval-error-analysis

Evaluate a model rigorously and inspect where it fails. Use after training, during comparison to baselines, for LLM/system evaluations, noisy or suspiciously clean results, or whenever benchmark numbers alone are insufficient and the user needs slices, failure categories, and explanations.

How do I install this agent skill?

npx skills add https://github.com/jurgendn/agent-skills --skill model-eval-error-analysis
view source ↗

Is this agent skill safe to install?

  • Gen Agent Trust Hubpass

    The skill provides a standard workflow for evaluating AI models and analyzing failure modes. It contains no executable code, remote dependencies, or suspicious instructions.

  • Socketpass

    No alerts

  • Snykpass

    Risk: LOW · No issues

What does this agent skill do?

Model Eval Error Analysis

Do not stop at one headline metric.

Intake

Collect:

  • task, dataset, split, metric, and baseline;
  • model outputs or failure examples if available;
  • aggregate numbers with seeds/uncertainty;
  • deployment or paper claim the evaluation must support;
  • known risky slices.

Workflow

  1. Verify the metric implementation and data split.
  2. Report the headline result with provenance.
  3. Slice performance by relevant factors such as:
    • difficulty
    • length / scale
    • class imbalance
    • domain shift
    • prompt family
    • robustness setting
  4. Inspect failures manually and cluster them.
  5. Separate:
    • random noise
    • systematic failure modes
    • evaluator bugs
  6. Compare against the strongest baseline on the same slices.
  7. End with concrete next actions.

Rules

  • Never smooth away inconvenient variation.
  • If confidence intervals or multiple seeds matter, say so.
  • A higher average may still hide a worse model for important slices.

Output shape

Return:

  • Metrics checked
  • Slice analysis
  • Failure taxonomy
  • Suspected causes
  • Recommended fixes or follow-up experiments

Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.

<a href="https://skillzs.dev/skills/jurgendn/agent-skills/model-eval-error-analysis">View model-eval-error-analysis on skillZs</a>