evals-skills
Evaluate skills for AI coding agents to help build LLM evaluations.
- Type
- Agent Skill
- Open source
- Yes
- GitHub Stars
- ★ 1.4k
- Source
- skill-github
- Repository
- github.com/hamelsmu/evals-skills
Overview
These skills are designed to guide AI coding agents and help users build LLM evaluations. They can help avoid common mistakes and recommend other skills to resolve issues. Suitable for engineers and product managers, especially those new to evaluation. After installing the plugin via Claude Code or Skills CLI, use the command `/evals-skills:<skill-name>` to invoke various skills.
Capabilities
- ▪Audit evaluation pipelines and identify priority issues
- ▪Guide users to review traces and categorize failures
- ▪Generate diverse synthetic test inputs
- ▪Design evaluators using LLMs as judges
- ▪Calibrate LLM judges against human labels
- ▪Evaluate retrieval and generation quality in RAG pipelines
Use cases
Setup
Run the following commands in Claude Code: "/plugin marketplace add hamelsmu/evals-skills" and "/plugin install evals-skills@hamelsmu-evals-skills"; or use npx: "npx skills add https://github.com/hamelsmu/evals-skills"
This information was compiled by AI from public sources and may contain inaccuracies — please refer to the source.
FAQ
How do I get started with these skills?
Start with the `eval-audit` skill and follow the steps in the README to install and run it.
Who are these skills for?
Suitable for engineers and product managers, especially those just getting started with evaluation.
Related skills
gemini-cli
Gemini CLI is an open-source AI agent that brings Gemini's powerful capabilities directly into the terminal.
private-gpt
Provides a complete API layer for local models, supporting RAG, skills, tools, and more.
planning-with-files
File-based persistent planning skill, ideal for AI coding agents and long-running tasks.
scientific-agent-skills
Transform any AI agent into a scientific assistant with 147 ready-to-use research skills.
susi_alexa_skill
A skill that enables question-and-answer interactions between Alexa and Susi AI.
claude-context
Provides code search capabilities for Claude Code, turning the entire codebase into context.