← All writing
Artificial intelligenceProduct engineeringCloud securityEvidence

AI should explain the evidence, not become the evidence

In a cloud audit, deterministic collectors and rules should establish findings while AI helps people understand what those records mean.

An AI-generated security finding can be persuasive before it is correct.

That is a poor failure mode for a cloud audit. A customer may use the result to change access policy, fund remediation, accept risk, or challenge an operating team. The finding needs to survive a more demanding question than “Does this sound plausible?”

It needs to be reproducible from the provider evidence and the rule that evaluated it.

The audit-product architecture I have been working through draws a hard line: deterministic collectors and versioned rules establish findings; AI explains, prioritizes, and answers questions about records that already exist.

Collection should not be a conversation

Cloud providers expose structured APIs for inventory, identity, network exposure, encryption, logging, backup, cost, and resilience. A collector can call a declared read surface, preserve the response, and record which version of the collector produced it.

That operation can fail, return partial coverage, or encounter a capability the current adapter does not support. Those outcomes should be represented directly. They should not invite a model to fill the gap with a likely answer.

A useful evidence record includes:

  • the tenant and scan boundary;
  • the provider-native resource identity;
  • the collector and ruleset versions;
  • a reference to the immutable raw payload;
  • an integrity hash; and
  • an explicit unavailable or unsupported state when collection is incomplete.

The model never needs the authority to create that record. It needs a bounded, inspectable record to reason over.

A finding is a rule outcome

A normalized inventory is not yet an audit. The system still needs to decide whether an observed condition violates a policy or represents a risk.

That decision belongs to versioned rule logic. The finding records the rule, severity, rationale, evidence references, and safe remediation guidance. Given the same evidence and ruleset, the result should be reproducible.

Reproducibility does not make the rule infallible. It makes disagreement useful. A reviewer can inspect the source payload, challenge the normalization, revise the rule, and identify which findings need to be regenerated.

If the finding came from an unconstrained conversation with a model, the review often collapses into prompt archaeology.

Explanation is a separate artifact

AI still has a valuable role.

A deterministic record can be technically complete and difficult to use. A model can explain why several findings matter together, translate provider terminology, prioritize questions for a human reviewer, or answer a customer’s question in the context of the evidence it has been shown.

That output is stored as an explanation with references back to the findings and evidence it used. It is not promoted into a primary record by sounding confident.

This creates four distinct layers:

  1. Evidence records what the provider returned.
  2. Resources normalize provider-specific objects without discarding native identity.
  3. Findings record deterministic rule outcomes over that evidence.
  4. Explanations help a person interpret those outcomes.

Each layer may be improved independently. A clearer explanation does not rewrite the finding. A corrected rule does not mutate the raw evidence. A provider-schema change does not have to erase the native object that exposed it.

Read-only must be proven, not described

The first version of the audit is designed to be read-only. It offers remediation guidance and links, but it does not change a customer’s cloud, run arbitrary commands, or let an explanation trigger a repair.

That boundary reduces both permission scope and product ambiguity. The system is collecting evidence and presenting accountable advice. It is not quietly becoming a change-management platform.

The connection should also fail closed when it cannot prove the granted access is read-only. Successful authentication proves identity and reachability. It does not prove that the credential lacks write capability.

Any future assisted remediation would need a new consent, authority, approval, rollback, and audit model. It should not arrive as an incidental feature of the explanation layer.

Preserve the native source beside the normalized model

Cross-cloud comparison needs a shared vocabulary. Defensibility needs the exact provider object.

Keeping only normalized fields makes the system easier to query but harder to challenge. Keeping only raw provider payloads preserves fidelity but forces every consumer to relearn four provider schemas.

The design accepts both costs. A resource carries a canonical type and the provider-native type and identity. A finding links to immutable raw evidence. Normalization becomes an index over the source rather than a replacement for it.

That is particularly important when AI is present. The system can show which facts the explanation received instead of asking the reader to trust that the model was “grounded.”

The boundary makes AI more useful

Keeping AI away from primary records is not a claim that models are incapable of analysis. It is a decision about where probabilistic interpretation helps and where it weakens the product’s contract.

Customers do not need every sentence to be deterministic. They do need the finding, severity, and source to remain inspectable after the conversation ends.

The underlying ADRs and domain model are accepted design records. They establish the intended product architecture, not a claim that the complete multi-cloud audit has been implemented, adopted, or operated for customers. Some adjacent collector work can inform implementation, but the decided deterministic audit spine still needs to be built and validated as a whole.

That evidence boundary should remain visible in the product and in any account of the work. AI can make evidence easier to understand. It should not become the reason the evidence is believed.