Intent Solutions
Intent Labs

Start here

How do you prove an agent did the job?

Start with one claim you can test. Not “is this agent good?” Try “does this skill find the documented failure and return the required evidence?” A useful evaluation makes the claim, conditions, decision rule, and receipt inspectable.

The evaluation path

  1. State the claim.

    Name the behavior that must be demonstrated and the conditions that count as success.

  2. Publish the test before the answer.

    An eval set records the cases, policy, and stopping conditions. This keeps the result tied to a visible method.

  3. Run the system.

    Deterministic checks and behavioral judges evaluate the work. A judge that cannot decide produces an advisory or error, not a quiet pass.

  4. Publish the decision and its evidence.

    The result points to a content-addressed Evidence Bundle. The bundle records what ran, what it saw, and why it reached that decision.

  5. Verify before display.

    Labs checks the signature, expected publishing identity, and public transparency-log entry before a result appears here.

Four things that used to sound like one thing

Eval set
The test specification. It says what will be measured.
Predicate
The stable meaning of a signed statement. Evals is the authority for those definitions.
Result
The decision from one run: pass, fail, advisory, or error.
Evidence Bundle
The receipt that connects the decision to the exact inputs, policy, and execution.

See it before you build it

Inspect the failure we published about our own work.

The dogfood scorecard applies the same method to Intent Solutions skills. It shows a ship decision and a block decision with the same visual weight, plus the commands and public log entries needed to verify them.

Open the dogfood scorecard

Run J-Rig in your repository

J-Rig is the command-line runner behind the skill evaluation path. Pin it per repository, then run an eval against a published spec.

pnpm add -D @intentsolutions/jrig-cli
pnpm exec j-rig check ./my-skill
pnpm exec j-rig eval ./my-skill --spec ./eval-spec.yaml
pnpm exec j-rig report

The judge provider is selected for the constraint. The evaluation contract and evidence format stay stable.