Skillet
An eval harness for agent skills: point it at a SKILL.md and it runs real agent loops against Claude, GPT and Gemini, mocks the APIs the skill expects, and grades the transcripts.
Skillet runs a skill the way an agent would use it. Point it at a directory holding a SKILL.md and it starts a real agent loop against Claude, GPT and Gemini in one command, with shell and file tools in a sandboxed working directory, and a judge grades each transcript against the assertions the skill declares.
A skill that calls an external API gets a mock instead of production. Importing an OpenAPI spec or an MCP server turns it into a deterministic local server, and each eval case brings its own scenario state, so a case can stand a service up as unhealthy and assert the agent found the failed deploy behind it.
Every run writes a manifest beside its benchmark: the version, the hash of the skill, the models, the mock specs, the eval file. Model calls do not replay identically, so when a score moves the manifest shows which input moved with it.
Scores come out per provider, so a skill that holds on Claude and falls apart on Gemini says so in the same run.