Skip to main content

Understand compliance evidence and scores

Documentation for Tutela v3.9.0.

Use evidence scores to identify which mapped checks have supporting information and which need work. A Tutela score describes its mapped controls and recorded evidence; it does not assess every obligation or process in your organization.

Release applicability​

The saved-scope recovery, period comparisons and source-availability explanations below apply to v3.9. Earlier releases may not have these controls. Check your deployed release before following these workflows.

Prerequisites​

  • Permission to view Evidence > Compliance.
  • Agreement with your compliance owner about the frameworks applicable to your organization.

Review a framework​

  1. Open Evidence > Compliance and review the selected framework scope with its owner. Confirm which frameworks apply before comparing scores or preparing an export.
  2. Open the framework and inspect its control results. Read the evidence source and explanation, not only the overall percentage.
  3. For an incomplete control, identify whether it lacks configuration, activity, or supporting evidence. Assign the corrective task to the relevant owner.
  4. After that task is complete, refresh the control and inspect its updated evidence. Saving a setting alone does not establish that the intended workflow ran.

Saved scope and unsaved choices​

The selected count describes the framework checkboxes you are editing. Unsaved choices do not change the displayed assessment or its export. Use Save scope to submit the selection, or Discard changes to return to the saved selection. An empty selection does not mean every framework applies.

After saving, the assessment must match the saved framework selection before readiness and exports are available. If the assessment does not match, use Refresh assessment. If settings or the catalog cannot be loaded, retry that read rather than choosing a replacement scope just to restore a score.

A save that could not be confirmed may still have reached the service. Your draft stays visible. Use Refresh saved scope, compare the returned selection with your intended scope, and only then decide whether to submit it again.

What the score means​

Controls marked Not applicable are excluded from the score. Controls with missing configuration or no evidence reduce coverage. Check scope and missing sources before comparing two scores or interpreting a change over time.

The frameworks available in Tutela map specific deployment and AI controls. The evidence does not by itself establish organization-wide compliance with a law or framework, or replace your organization's assessment.

What a framework mapping means​

Mapping counts are catalog references, not passed controls. A reference connects a feature to a framework check; it does not confirm that the feature is enabled, exercised or effective in your environment. Read the check's actual evidence, scope and limitations before relying on it. A missing or unavailable catalog does not establish that there are no applicable controls.

Why a corrected reference says Not assessed​

For corrected references in v3.9, Tutela shows the official requirement separately from what the technical check measures. For example, counting AI requests does not demonstrate that an AI system is valid and reliable. Not assessed means this check has not established that requirement; it does not mean your organization failed it.

Read the source limitation and next action with your compliance owner. If the definition differs from the reviewed definition, review that difference before relying on the reference. More traffic or changing a threshold does not establish conformance to the official requirement.

A corrected reference does not rewrite an older saved assessment. Compare the original result with a new assessment, keeping each one's date, scope and stated limitations. An older favorable result does not establish current conformance.

Compare periods without confusing them with saved assessments​

Check scores by period applies current check definitions and saved scope to the retained records for each period. These are not saved past assessments. The chart shows a weighted check score, not the percentage of controls with complete evidence. It does not certify compliance or measure organization-wide compliance.

Use Compare to select the overall score or a framework in your saved scope. Open View period values for exact scores and period-start dates in UTC. Each point represents a separate period. Missing values are not zero; a missing period does not establish that no activity occurred.

Changing definitions, saved scope, exclusions or available evidence can change a recalculated score. That does not mean the historical action changed. When a comparison cannot be refreshed, retry the read before relying on its results. For a formal review, preserve the exported result, its evaluation time, selected period and limitations through your organization's evidence process.

Read evidence states​

StateMeaningNext step
PassSufficient evidence meets this mapped checkInspect the source, time and scope before relying on it.
PartialEvidence exists but is incomplete or below the pass thresholdIdentify the missing part in the explanation and assign an owner.
FailRecorded evidence does not meet the configured requirementInvestigate the stated failure and verify the correction.
No evidenceThe expected source has not produced evidenceCheck the connection and activity rather than assuming the control passed.
Not configuredThe required measurement or configuration is unavailableRead the explanation before changing settings. Some checks need evidence collection that is not yet supported.
Not applicableThe control is outside the selected scopeConfirm the exclusion with the framework owner.

Some detection checks use the software release's benchmark results. These describe a tested platform capability, not inspection of your own requests. See Understand detection benchmarks.

When a measurement is unavailable​

An absent measurement is not a measured zero. Zero means an available source was measured and returned none; unavailable means the assessment cannot establish the value. More traffic will not repair a measurement that Tutela does not collect.

An unavailable action summary does not mean no actions occurred. A failed policy-decision summary makes its rate unavailable; a failed inspection-action summary makes the coaching count unavailable. Other available sources retain their results. Retry the assessment after the source recovers. Changing a policy does not repair a failed read. Keep the source limitation with any export.

What the policy-decision rate counts​

The rate counts recorded decisions to block, redact, quarantine or require approval as a share of all recorded policy decisions. Both counts come from the same policy decision records and selected period. Inspection actions are counted separately. It is not the percentage of requests protected or actions successfully stopped. Review the linked execution evidence to understand what actually ran.

No policy decision records means the rate is unavailable, not zero. A measured zero means decisions were recorded, but none selected those four actions. Missing or inconsistent summaries stay unavailable; they are not replaced by inspection totals. Do not change policy just to increase this rate.

A block recorded for one kind of request does not prove protection against instruction manipulation or unsafe responses. Review the matching detection, policy decision and input/response records; this check cannot yet assess them together. These named checks remain unavailable even when the generic policy-decision rate is measured. This is not a new evidence source or a request to supply a manual attestation as evidence of runtime protection.

For example:

  • A detection does not confirm that an alert was configured or delivered.
  • An inspection total does not separate inputs from responses.
  • A policy action does not establish its version history.
  • Separate detection and block totals do not establish that the same request was blocked.

Read the check's explanation with the owner of the relevant policy, notification configuration or Activity records. If the explanation says automated collection is unavailable, give your administrator or support team the check name, selected period and stated source. Do not generate sensitive traffic, change enforcement or mark a check Not applicable just to improve its score. Keep the limitation with any evidence you share; do not describe missing evidence as a confirmed control failure or success.

Activity is not a broader assurance claim​

A coaching message is not a human review. Counting data types does not assess fairness or categorize an AI system. A tool call does not prove that autonomy limits were applied. Policy counts and enforcement rates do not establish legal review or board oversight.

Having enforcement policies also does not show that an AI governance policy was approved or that improvement actions were reviewed and completed. Those checks need the governance and improvement records themselves. Tutela does not yet collect those records for these checks; review them with your compliance owner and keep them through your organization's evidence process.

Likewise, policy activity alone does not prove organizational governance or security-event handling. Those checks need approved procedures and the relevant governance or response records, not a higher number of blocks.

An authenticated-identity count does not show that every agent action met its identity requirements. A recorded policy decision does not, by itself, prove that access was authorized, tool misuse was prevented, or permissions could not be exceeded. Review the applicable access rules and the matching action records. These broader checks remain Not assessed until Tutela can assess that evidence; the underlying activity measurements remain available separately.

A count of configured policies does not show that request limits were applied, that access to embeddings or payment-card data was restricted, or that technical documentation is current. Access and request-limit checks need the applicable rules and matching decisions. Documentation checks need the system's documents and review records. Automated collection for these checks is not available; review the stated evidence with the service, data or documentation owner. Adding more policies does not supply that evidence.

Detecting sensitive data does not show that a leak was prevented. A count of detected data categories is also not evidence of an approved data-governance process, privacy by design, or a documented risk assessment. These checks need the relevant procedures, assessments and matching protection records, not a higher count of detections. Review those records with the data or compliance owner. The underlying detection counts remain available separately.

A general prompt-injection detection does not establish a specific attempt to extract a system prompt. Likewise, a recorded policy event involving health data does not establish that all health-data access was logged. Review the relevant detection details or access records with the system owner. Automated collection of the evidence needed for these broader checks is not available; changing a threshold or generating more activity will not complete them.

These checks need their own supporting assessment or decision records. When Tutela does not collect those records for the check, its control row shows Not assessed, even when related activity exists. Exports identify the missing measurement as Not configured. The activity remains available; the broader claim is not assessed. Review the explanation with the system or compliance owner, and retain the relevant records through your organization's evidence process. A link to guidance is not an evidence-upload or verification workflow.

The same distinction applies to deployment evidence such as encryption, security scanning and retention settings. When a check says collection is unavailable, Tutela does not offer an upload or configuration step that completes it. Review the required records with your platform administrator and retain them through your organization's evidence process. An unavailable measurement does not mean that encryption, scanning or retention is disabled.

Review action evidence​

This section is for analysts and compliance reviewers with access to Evidence > Compliance in v3.9.

  1. Select a framework in the confirmed scope, then find Action evidence.
  2. Check the recorded activity period and the separate Assessed time. The period selects actions with retained records in that window. Later execution records can fall outside that period without belonging to a different action.
  3. Read the result, Execution, and Response together. Last recorded is the action record's time, not the time this page was assessed.
  4. Use Review session for the supporting activity. Open Decision and record details when you need the policy version or record identifiers.
ResultWhat it establishesNext step
Block recorded; action did not startThe retained records link a block decision and a finalized not-started result to one invocation.Review the session if you need the surrounding activity or decision explanation.
Block recorded after executionA block is recorded, but the action already ran. The response may have been withheld.Review both execution and response before deciding what corrective action is needed.
No confirmed pre-execution blockThese records do not establish that this action was blocked before it ran. This is not automatically a failed control.Compare the recorded decision with the policy intended for this action.
Result not establishedRecords are missing, conflicting, or cannot be tied to one unambiguous attempt.Review the stated limitation and session; do not repeat the action just to create evidence.

A detected category does not explain why the policy matched. This view does not change control scores, certify compliance, or establish that every action was recorded. A partial result covers only the records shown. No retained invocations means none were found in this source and period, not that no activity occurred.

If the read fails, use Retry after the source recovers. If the service does not supply this evidence, contact your administrator with the selected framework, period and displayed limitation. Do not change policy or generate sensitive traffic to make the section appear complete.

JSON includes these action records; CSV does not. This live section is separate from saved assessments and does not alter their history. Before sharing a result, retain its period, assessment time and limitations with the export.

Understand setup status​

Not required means the confirmed scope and loaded control catalog contain no applicable controls that need that evidence source. It is not a completed check. Missing scope or an unavailable catalog is not evidence that setup is unnecessary.

Source setup and control results are separate. A working source can report a failed control result: Complete in the setup checklist does not change that control's result. Needs setup means required evidence is not yet available; review the control's explanation before changing a connection or configuration.

Share the result with its context​

Keep the source explanation, suggested next action and any scope-exclusion reason with the result. JSON exports retain that evaluated context; CSV exports append context columns after the existing result columns. Older exports may not contain them. A missing context field is not confirmation that no limitation exists.

These are current evaluations, not immutable historical assessments. An evaluation time tells you when the check was assessed; it does not establish when the source was last complete or when the underlying action occurred. Preserve the selected period and limitations when sharing the result with a reviewer.

Where saved assessments are available in your release, older snapshots retain their original evaluation, scores and exports. They are not silently rewritten when a check changes. Compare an old snapshot with a new assessment and read each one's version and limitations. Recovering an original request returns that original record; it does not rerun the check. A pending older request remains pending rather than being completed under a different evaluator.

The Earlier evaluation rules notice identifies a completed assessment made under an older set of checks. Its original scores and exports remain unchanged; the notice does not mean the assessment was reassessed or newly verified.

Newer saved-assessment CSV files include the saved versions, whether a measurement was available, the result's limitations, and evaluation times. A row labeled assessment describes an empty assessment; it is not a passed or failed check. Rows labeled control describe individual checks. Older CSV files keep their original format. These saved records do not yet include each check's detailed source explanation or exclusion reason; a missing explanation is not an all-clear. File and result digests help identify the exported records. They are not signatures and do not establish that all required evidence was collected.

Signed evidence packages​

Signed export is available in v3.9. The package contains the retained assessment, its saved configuration and a summary. It does not collect new evidence or revise the original result.

A signature can establish the package's origin and whether its contents changed; it does not certify compliance or fill gaps in the assessment. Verification needs an independently trusted public key, the expected organization and the selected assessment. Do not treat a downloaded file as a verified signature. Your platform administrator must confirm the signing key's trust and revocation status before you rely on verification. Keep the original file and its limitations together.

Record supporting evidence for a saved check​

Audience: analysts submitting evidence and administrators reviewing it. This workflow applies to v3.9; confirm availability in your deployed release before following these steps.

Use supporting evidence to record which external document or controlled test supports a particular saved check, its scope, and a separate review decision. This does not change the saved assessment or its score. Acceptance is a recorded review of the submitted material, not a legal opinion. It does not certify compliance.

Before you start​

  • Open a completed assessment in Evidence > Saved assessments. Reading this view requires dashboard access (view_dashboard).
  • Keep the source material in your organization's authorized evidence system. This workflow does not upload or retrieve the source document. The submitter and reviewer need independent access to it.
  • Obtain the source and review-method reference IDs, their SHA-256 fingerprints, the workflow and environment IDs, and an appropriate expiry time. A fingerprint identifies the exact bytes; entering one does not prove that the material is genuine, sufficient, or current. Do not enter document contents, personal information, passwords, URLs or access tokens into reference fields.
  • Submission requires a signed-in analyst or administrator with submit_compliance_attestations. Review requires a separate signed-in administrator with review_compliance_attestations. A submitter cannot review their own submission. Machine credentials cannot submit or review.

Submit and review​

  1. Open the saved check's Supporting evidence panel. Confirm the retained observation window and the time the assessment was saved.
  2. Choose Add supporting evidence. Supply the workflow, environment and governed path that the material actually covers. These are your declared scope, not automatically verified coverage.
  3. Enter the Source reference and Source SHA-256. Choose Document review or Controlled test, then enter its Method reference and Method SHA-256. Keep both referenced records accessible to the reviewer.
  4. Set Expires at (UTC) and choose Submit supporting evidence. Wait for Submission confirmed before treating the write as confirmed.
  5. A separate authorized reviewer opens Review record, checks the source, scope and method outside Tutela, and chooses Accept or Reject with the applicable reason. Then choose Accept supporting evidence or Reject supporting evidence. Wait for Review confirmed. If the evidence is insufficient or conflicts with the relevant runtime records, do not accept it merely to finish the queue.
  6. Use Show review history to see the recorded submission and decision. State as of describes the last successful read. Expiry does not erase the original review or change the saved score. Expired or already-reviewed records cannot receive another review; changed material needs a new submission.

Tutela retains the reference, declared fingerprints, scope, expiry and review history. It does not independently validate the external source or automatically compare it with runtime evidence. Review acceptance does not fill a missing automated measurement. Keep investigation ownership and follow-up notes in Cases.

When a submission or review is not confirmed​

An interrupted response does not tell you whether the write was saved. Use Recover original request to check the original submission or decision rather than creating a replacement. Stop waiting stops the browser's wait; it does not undo a write already accepted by Tutela.

Closing the panel retains the original request while this page remains open. Leaving or reloading the page loses its in-memory recovery request. Signing out or changing account also clears it. Reopen the supporting-record list and review the history before submitting anything again. Access failures require an administrator's review, not another person's credential. A recovered receipt confirms the original write; check the current record separately for expiry and review status.

Validate​

  • The framework scope and exclusions are intentional.
  • Each reported result includes understood evidence and remaining gaps.
  • An export is described with its actual scope, rather than as a blanket assurance claim.

Next steps​