Verification and Evaluation

Verification turns claims about a system into evidence that the institution can use for decisions. Sovereign Source AI examines whether work meets its specification, stays within its authority, and gets the intended result. The platform runs technical checks, reviews agent work, links each requirement to evidence, and measures how users experience the result. A plausible answer, a clean build, or a test suite that passes is only part of the proof.

Verification and validation

Verification asks whether the result agrees with its specification. Validation asks whether the product or action solves the intended problem. Both link to the original intent. They answer different questions.

A feature can meet every implementation requirement and still not improve the experience that it was for. It can also improve a main metric and break a safety or accuracy constraint. So the evidence model keeps objectives and guardrails together.

An experience contract makes those expectations explicit. It links a requirement or feature to measurable outcomes and to conditions that must stay true. After deployment, measurements help the institution decide to continue, revise, or roll back the feature. They can also start a new intent.

Four areas of verification

Technical verification examines the artifact and the process that made it. The evidence includes these items:

Each check supports a defined claim about a specific version.

Agent verification compares the assigned work with what the agent did. It examines these items:

An independent reviewer checks the agent’s work, so you do not have to trust the agent’s own report. Human authority is still necessary where policy requires it.

Requirement verification links each claim to explicit evidence. Each requirement has an acceptance condition. It also names the tests, observations, or reviews that show whether the work met that condition. The link must identify the artifact or deployment under evaluation.

Experience verification examines use after release. These measures help show whether the feature works in practice:

To measure institutional outcomes, you can also need authoritative data from operations, not only interface analytics.

The evaluation model

Evaluation has eight levels. A pass at one level does not prove a pass at another:

LevelQuestion
SchemaDid the output satisfy its contract?
CapabilityDid the operation stay inside its declared function?
ModelWas the result reliable enough?
PolicyWas the action authorized?
SecurityDid the work obey the isolation, network, dependency, and secret rules?
ExperienceDid the feature work for users?
OutcomeDid it meet the institutional objective?
ProvenanceCan you reconstruct and explain the result?

Each result shows what is not known. The proposed privacy assessment model uses PASS, FAIL, UNKNOWN, and NOT APPLICABLE. If there is no evidence for a check, the result is UNKNOWN, not PASS. An automated assessment is not the same as an independent certification.

Reference build

A reference build has these parts:

Each result links to its work order, artifact, policy, and evaluation context.

Release gates use those results under defined authority. A model can explain a result or propose more checks. A clear explanation from a model does not make a control pass. Later outcome measurements add to the evidence record and inform future intent.

From the research

All topics