9.3 Evidence Packs for Testing Governance

How to export a single dated report of everything CertiQA already measured about a project – pass rates, fault-injection recovery, security posture, judge calibration, traced behavior, and the audit trail – for internal reviews, customer due-diligence questionnaires, or your own compliance team’s mapping work.

What an Evidence Pack Is

Every other guide on this page describes one measurement: a pass rate, a fuzzing scorecard, a judge’s calibration run. An evidence pack collects all of it for one project over a date range you choose into a single JSON manifest: run outcomes and pass rates, fault-injection recovery rates, security posture history by attack category, judge calibration runs, trajectory and tool-call records, and an excerpt of the audit trail for that window. Nothing is measured freshly to build a pack – it is a report over data your test suite, fuzzer, and judge calibration runs already produced.

How to Generate One in App

• 1. Open the project’s Evidence Packs tab.
• 2. Choose a reporting period (start and end date).
• 3. Click “Generate” – the pack is assembled and saved with a content hash and a timestamp.
• 4. Download the JSON manifest, or hand a reviewer the in-app link to browse the same sections there.

What It Is Not

An evidence pack is evidence – the underlying data a human reviewer, an auditor, or your own legal/compliance team can use to support a conformity or risk assessment they are doing. It is not itself a certification, an attestation, or a compliance determination, and CertiQA does not represent it as one. Treat it the way you’d treat a test report handed to an auditor: primary material for someone else’s judgment call, not a verdict CertiQA is rendering on your behalf.

1-Click Executive Release Quality Signoff

For engineering leadership, compliance officers, and release managers who need an audit-ready signoff before approving a deployment, CertiQA provides a 1-Click Executive Quality Signoff view (rendered via ReleaseCertificateModal). This view synthesizes the evidence pack into a formal, printable release readiness certificate featuring:

Certificate Section	Key Metrics & Evidence Displayed	Compliance Purpose
Executive Summary	Overall Suite Pass Rate, Wilson 95% Confidence Interval, Total Runs Analyzed, and Deployment Go/No-Go Recommendation.	A statistically grounded view of release readiness rather than raw pass counts.
Chaos & Fault Resilience	Fault injection recovery rate across TIMEOUT, HTTP_500, HTTP_429, and MALFORMED_JSON test scenarios.	Verifies the agent handles adverse external conditions without unhandled cascading failures.
Security & Policy Posture	OWASP LLM sweep results by attack category (prompt injection, PII leakage, tool misuse and the other categories the fuzzer covers).	Summarizes the security sweep results for the period.
Cryptographic Verification	SHA-256 manifest hash, generator identity, timestamp, and audit trail ledger ID.	Provides tamper-evident provenance for external compliance and due-diligence audits.
Signoff Endorsements	Signature lines for QA Lead, Security Officer, and Engineering Director with date and build tags.	Records who signed off, when, and for which build.

Three Scoping Details Worth Knowing Before You Share One

• The audit-trail section of a pack is not scoped to just this project – it is every audited action platform-wide in the date range. The pack says so in its own metadata; don’t represent that section as project-only when sharing it onward.
• Judge calibration rows include any org-shared rubric used to grade this project’s cases, not only project-owned rubrics – correct for “was this project’s testing measured with a validated judge,” worth knowing if you’re auditing rubric ownership specifically.
• A pack’s manifest is the durable artifact; re-hashing a copy you’ve pasted elsewhere isn’t guaranteed to match the stored hash byte-for-byte once it’s passed through another system’s JSON formatting – verify by regenerating from CertiQA, not by re-hashing a downstream copy.