Intent Labs

Every published test

This page lists every automated test result we publish. Each row is one test that ran on its own. Results are never added up into a single score.

How to read a result

Words you will see

See the test opens exactly what was checked. Signed record opens the tamper-evident proof of the result.

Last updated: . Some projects may be newer; each row shows its own test date.

AI skill tests

Every night we give each AI skill a set of real tasks and grade the finished work against its written rules. A skill passes only if it meets every required rule.

SkillResultWhat happenedTested onLinks
Audit tests Needs attention
  • 3/5 criteria passed on MiniMax-M3 (provider: minimax)
  • 1 criteria could not be judged (unsure)
  • 1 non-blocker criterion evaluation(s) failed
  • thresholds did not pass
  • regression comparison was not run
 See the test · Signed record
CoreWeave fabric diagnostics Failed
  • 9/11 criteria passed on MiniMax-M3 (provider: minimax)
  • 2 blocker criterion evaluation(s) failed
  • thresholds did not pass
  • regression comparison was not run
 See the test · Signed record
CoreWeave GPU cost leak hunter Failed
  • 7/11 criteria passed on MiniMax-M3 (provider: minimax)
  • 2 blocker criterion evaluation(s) failed
  • 1 criteria could not be judged (unsure)
  • thresholds did not pass
  • regression comparison was not run
 See the test · Signed record
CoreWeave GPU node forensics Failed
  • 13/18 criteria passed on MiniMax-M3 (provider: minimax)
  • 1 blocker criterion evaluation(s) failed
  • thresholds did not pass
  • regression comparison was not run
 See the test · Signed record
Databricks bundle medic Couldn't finish
  • provider_failure/judge [minimax rate_limit, retryable]: judge provider failed on 1 of 13 judged criteria; the evaluation is incomplete: Token Plan rate limit reached: Upgrade your Token Plan or switch to pay-as-you-go API usage. (2062)
  • nightly eval did not complete for databricks-bundle-medic (see databricks-bundle-medic.error.log in the run)
 See the test · Signed record
Databricks cluster forensics Couldn't finish
  • provider_failure/trigger [minimax rate_limit, retryable]: trigger provider failed on 7 of 7 routing case(s); the evaluation is incomplete: Token Plan rate limit reached: Upgrade your Token Plan or switch to pay-as-you-go API usage. (2062)
  • nightly eval did not complete for databricks-cluster-forensics (see databricks-cluster-forensics.error.log in the run)
 See the test · Signed record
Databricks cost leak hunter Failed
  • 4/10 criteria passed on MiniMax-M3 (provider: minimax)
  • 3 blocker criterion evaluation(s) failed
  • thresholds did not pass
  • regression comparison was not run
 See the test · Signed record
Databricks streaming guardian Couldn't finish
  • provider_failure/judge [minimax rate_limit, retryable]: judge provider failed on 3 of 15 judged criteria; the evaluation is incomplete: Token Plan rate limit reached: Upgrade your Token Plan or switch to pay-as-you-go API usage. (2062)
  • nightly eval did not complete for databricks-streaming-guardian (see databricks-streaming-guardian.error.log in the run)
 See the test · Signed record
Databricks UC migration pilot Couldn't finish
  • provider_failure/trigger [minimax rate_limit, retryable]: trigger provider failed on 7 of 7 routing case(s); the evaluation is incomplete: Token Plan rate limit reached: Upgrade your Token Plan or switch to pay-as-you-go API usage. (2062)
  • nightly eval did not complete for databricks-uc-migration-pilot (see databricks-uc-migration-pilot.error.log in the run)
 See the test · Signed record
HubSpot warehouse sync Couldn't finish
  • provider_failure/trigger [minimax rate_limit, retryable]: trigger provider failed on 6 of 8 routing case(s); the evaluation is incomplete: Token Plan rate limit reached: Upgrade your Token Plan or switch to pay-as-you-go API usage. (2062)
  • nightly eval did not complete for hubspot-warehouse-sync (see hubspot-warehouse-sync.error.log in the run)
 See the test · Signed record
Sentry data handling Needs attention
  • 11/11 criteria passed on MiniMax-M3 (provider: minimax)
  • thresholds passed
  • regression comparison was not run
 See the test · Signed record
Skill creator Needs attention
  • 6/10 criteria passed on MiniMax-M3 (provider: minimax)
  • 4 non-blocker criterion evaluation(s) failed
  • thresholds did not pass
  • regression comparison was not run
 See the test · Signed record
Supabase known pitfalls Failed
  • 10/11 criteria passed on MiniMax-M3 (provider: minimax)
  • 1 blocker criterion evaluation(s) failed
  • thresholds did not pass
  • regression comparison was not run
 See the test · Signed record
Validate SKILL.md Needs attention
  • 2/5 criteria passed on MiniMax-M3 (provider: minimax)
  • 3 non-blocker criterion evaluation(s) failed
  • thresholds did not pass
  • regression comparison was not run
 See the test · Signed record

Platform checks

Automated quality checks on the code behind this lab and our skills marketplace, such as tests, security scans and catalog rules.

What was testedProjectResultWhat happenedTested onLinks
Harness hash Audit harness Passed .harness-hash manifest verified consistent Signed record
Architecture Contracts kernel Passed — Signed record
Coverage Contracts kernel Passed — Signed record
Coverage J-Rig evaluator Passed — Signed record
Harness hash Methodology lab Passed
  • .harness-hash manifest verified consistent
  • pins the spec-integrity surface: CI workflows (incl. partner-name-guard + schema-drift), schema redirect stub, vendored audit-harness
 Signed record
Harness hash Rollout gate Passed .harness-hash pinned policy artifacts verified consistent Signed record
Reproducible dist Rollout gate Passed committed dist/index.js is byte-reproducible from src/ Signed record
Catalog invariants Skills marketplace Passed Catalog invariant check passed (434 published plugins; 3 quarantined records). Signed record
CI required Skills marketplace Passed ci-required completed successfully Signed record
Gitleaks Skills marketplace Passed gitleaks completed successfully Signed record
Skill conform Skills marketplace Passed skill-conform completed successfully Signed record
Unicode hygiene Skills marketplace Passed no BLOCKER-severity unicode findings (tag chars, bidi overrides) Signed record
Govern decision Team knowledge base Passed
  • sec-hex-aws-01 boundary-disclosure class=secret surface=hex-encoded documented=true
  • pii-inline-email-01 boundary-disclosure class=pii surface=inline documented=true
  • ci-govern-decision OK — zero undocumented false-negatives across both sections and every decision precision floor held (documented misses and known FPs reported above, tracked, not hidden).
 Signed record
Provenance integrity Team knowledge base Passed
  • anchored-brain verdict: {"memories_checked":1,"content_hash_mismatches":0,"invalid_sources":0,"audit_chain_rows":3,"tamper_signatures":0,"chain_forks":0,"documented_exceptions":0,"exception_manifest_loaded":false,"anchor_count":1,"anchor_status":"consistent","anchor_history_truncated":0,"anchor_hist
  • truncated-brain verdict: {"memories_checked":1,"content_hash_mismatches":0,"invalid_sources":0,"audit_chain_rows":2,"tamper_signatures":0,"chain_forks":0,"documented_exceptions":0,"exception_manifest_loaded":false,"anchor_count":1,"anchor_status":"divergent","anchor_history_truncated":1,"anchor_hist
  • ci-provenance-integrity OK — forks pass (disclosed), tamper fails closed, anchor truncation fails closed
 Signed record
Retrieval ratchet Team knowledge base Passed
  • tokenization measured=1.0000 baseline=1.0000 floor=0.9990 OK
  • eval artifact written: /home/runner/work/bobs-big-brain-registrar/bobs-big-brain-registrar/packages/qmd-adapter/eval-results/synthetic-v1.json
  • RATCHET PASS — no stratum regressed below its committed floor.
 Signed record

Technical note: each row is one signed, tamper-evident record (https://evals.intentsolutions.io/gate-result/v1). Open "Signed record" for the hashes, timestamps and transparency-log entry. How to read the evidence.

Browse results by project →