Ask most teams whether they log their AI decisions and they will say yes. Ask them to explain one specific decision from four months ago, for one named person, and the answer changes. They have the output. What they cannot produce is the reasoning, the version, the threshold in force that day, or what a human did next.
That gap is now expensive. California's automated decision rules give people a right to an explanation of decisions made about them, the EU AI Act pushes in the same direction for high-risk systems, and enterprise customers have started asking the question in security reviews. But the honest reason to build this is duller and better: when a model starts behaving oddly, the team with a decision log finds the cause in an afternoon and the team without one argues for a week.
What an audit trail is actually for
Three different people show up asking questions, and they want different things.
A regulator or a customer wants to know why one specific decision came out the way it did, in language a normal person can follow. An engineer wants to know what changed between last Tuesday, when the numbers looked fine, and today. A lawyer wants to know whether you can prove the safeguards you told someone about were actually running.
One log can serve all three, but only if you design for the first. The narrow engineering version (dump the prompt, dump the response) satisfies the debugger and nobody else.
What to record at the moment of the decision
The rule of thumb: record whatever you would need to explain the decision to a stranger without rerunning anything. Reproducing is not the goal. Reconstructing is.
| What to record | Why it matters later |
|---|---|
| Decision ID and subject reference | Every question you will be asked starts with one person and one decision |
| Inputs, or stable references to them | "The model saw these fields" is the first thing anyone asks |
| Model, prompt and ruleset versions | Behaviour changes come from here far more often than from the data |
| Raw output plus the parsed value | The output you acted on is not always what the model returned |
| Thresholds and config in force | The most common silent cause of drift, and almost never versioned |
| Human interaction, if any | Who saw it, when, and whether they changed it |
| Downstream effect | What the system did next, which is what the person actually experienced |
| Timestamp and pipeline version | Ties the record to what was deployed at the time |
Two of these carry most of the weight and get skipped most often.
Thresholds and config. Teams version the model religiously and treat the cutoff as an environment variable. Then somebody moves a score threshold from 65 to 70 on a Thursday, quality metrics shift, and there is no record that anything changed. If a number decides outcomes, it is part of the decision and belongs in the log.
What happened downstream. The model produced a 0.42. Fine. Did that route the application to manual review, or reject it outright with an email? The person on the other end experienced the consequence, not the score, and the consequence is what you will be asked to justify.
Where teams get this wrong
The pattern I see most is logging that lives at the wrong layer. Observability tooling captures the model call beautifully, because that is what observability tooling is for. But a decision is rarely one model call. It is a retrieval step, a call, a parse, a rules pass, and a write, and the interesting failures live in the seams. If your trail is a span in an APM tool, you can debug latency and you cannot answer "why me?"
Second pattern: logs with a thirty-day retention on decisions that people can ask about for years. Nobody chose that. It is the platform default nobody revisited.
Third: personal information sprayed through log lines because logging came after the feature. Now the decision trail is itself a privacy problem, and it lives in a system with much looser access control than your database. Reference the inputs where you can, store the sensitive parts in the same place your other personal data lives, and keep the pointer in the log.
Fourth, and this one is subtle: the log records that a human reviewed something, when what actually happened was a human clicked past a screen. If your override rate is zero across thousands of decisions, you do not have human review. You have a checkbox, and writing it down does not make it true. That distinction is the whole argument in keeping a human in the loop.
A worked example
A lender scores applications. Every decision writes one row: application ID, feature snapshot, model version 4.2.1, cutoff 0.61, score 0.58, outcome "decline", reviewer null, notification "email template D", plus a timestamp.
Nine months later the applicant asks why. With that row, the answer takes ten minutes and reads like a sentence: the model looked at these things, produced a score below the cutoff in force that day, no human reviewed it, and this email went out. Without the row, the team reruns today's model, gets 0.63 because the model was retrained in March, and now has to explain a number that was never the one used.
The interesting part is that the second team is not less competent. They are one design decision behind. Nobody asked what the log had to survive.
Retention, access and the request path
Three decisions to make on purpose rather than by default.
How long. Match the window in which somebody can plausibly ask, then add margin for the investigation that follows. For decisions that fall under access rights, thirty days is not a serious answer.
Who can read it. Decision logs concentrate sensitive information in one queryable place, which is convenient for you and attractive to everyone else. Treat access like production data access, with the same review and the same audit.
How a request gets fulfilled. Someone has to turn a row into a paragraph a person can read. Write the template before the first request arrives, because the first one always arrives on a bad week. If your model has genuine explanation needs, look at whether retrieval or fine-tuning changes what you can even claim about the logic, since a retrieval system can cite its sources and a fine-tuned model mostly cannot.
What this means for your team
- Start logging before you finish designing the log. An imperfect record beats a perfect plan, because the missing months never come back.
- Version the thresholds. It is a one-line change and it prevents the most common category of unexplainable drift.
- Log the consequence, not just the score. That is what the person experienced and what you will defend.
- Set retention from the question window, not from whatever your platform does by default.
- If you have a human-review step, measure the override rate. Zero means the step is decorative, and now you know before someone else points it out.
Most teams do not need a governance platform for this. They need one table designed on purpose, wired in at the right layer, before the decisions they will be asked about have already happened. If you are adding AI to a workflow where the output affects a person, tell us what the decision looks like and we will show you the log we would write first. It is usually a smaller piece of work than the compliance conversation around it suggests. The California specifics live in our companion piece on what the ADMT rules require you to build.