Financial services
Key risk indicators don't watch your knowledge
Every exposure in a regulated firm has a number, a threshold and an owner. The knowledge your AI assistant answers from has none of the three. Four leading indicators that would change that.
Cognatum Team · Oct 15, 2026 · 6 min read
Every risk function in a regulated firm runs on indicators. Delinquency rates. Failed logins. Open audit findings. Vendor service breaches. Each one has a number, a threshold, and a person who has to act when it moves. A key risk indicator exists to give early warning, a signal that exposure is rising before anything has gone wrong. So here is a question for your next risk committee. Which indicator tells you that the knowledge your AI assistant answers from has gone out of date?
Cognatum governs the entry
source · version · approver · permissions
What a key risk indicator is supposed to do
The definitions are consistent. Thomson Reuters calls a KRI a metric that spots a threat before it lands. It sets that against the performance indicator, which reports what already went wrong. The ISACA Risk IT framework puts it another way. A KRI shows that the firm is running past the risk it agreed to take. MetricStream adds four tests. A good one is measurable, predictive, comparable and useful. All three agree on the job. A KRI is a warning light wired to an exposure.
The features the guidance asks for
The Institute of Operational Risk is more specific. Its sound practice guidance says an indicator should be relevant, measurable, forward-looking, easy to collect, comparable and auditable. It adds a warning. An indicator is a proxy, so the aim is not to manage the indicator but the exposure behind it. A breach is a signal of threats ahead. Thresholds come down from the risk appetite statement. Trigger conditions name both the response and the person who owns it.
Read the data source column
MetricStream publishes a worked KRI table, and the most revealing column is the one naming where each number comes from. Credit risk reads core banking system reports. Operational risk reads IT service management logs. Cyber risk reads security tools and SIEM logs. Compliance risk reads the compliance tracking system. Liquidity risk reads treasury dashboards. Third-party risk reads vendor performance reports. Every feed is a system that emits events, and every indicator counts those events.
What none of them measure
Not one of those feeds is the knowledge your people and your AI systems answer from. Thomson Reuters lists four risks worth an indicator: fraud, supply and demand, noncompliance, and IT failure. Now add a fifth. Your firm tells a client, an examiner or a new hire something confidently wrong, at scale, from a document nobody has read since 2019. That risk appears on none of these lists. It has no feed, no threshold and no owner.
This is operational risk, not an IT problem
The Basel Committee revised its operational risk principles in March 2021. Its own account of why is worth reading. A 2014 review found that several principles had not been put in place properly. It also found they did not cover some important sources of operational risk. An assistant answering from unapproved content is one of those sources. It sits inside the control environment, and it produces statements people act on.
Someone has already measured the error rate
A preregistered study tested the commercial legal research tools lawyers already pay for. The LexisNexis and Thomson Reuters products hallucinated between 17 and 33 percent of the time. Both vendors had claimed to eliminate hallucinations. These are purpose-built, retrieval-grounded, professionally maintained systems. If a measured error rate exists for them, one exists for the assistant your team installed last quarter. The difference is that nobody is measuring it.
The exposure is rising on its own
An APQC survey of 1,000 professionals, sponsored by eGain, found that 92 percent of firms do not consistently capture knowledge from staff about to retire. Eighty-five percent have not put AI to work on knowledge management. Fifty-eight percent of C-suite respondents were very worried about the loss. That is a rising exposure by any reading, and it has no indicator.
Four indicators you could add next quarter
None of these needs new theory. Each one is countable, each supports a threshold, and each can be owned by the team closest to the content.
1. Share of approved entries past review date
Every governed entry carries a review date. The share sitting past it is the cleanest leading indicator available, because staleness comes before a wrong answer rather than after it. Set the threshold where your own policy sets the review cycle.
2. Duplicate and near-duplicate rate
Every surviving copy of a procedure is a chance for a retrieval system to return the wrong one. Count near-duplicates as a share of the library. A rising number means your answers are becoming a coin toss, whatever the retrieval scores say.
3. Open conflicts between approved entries
Two approved entries that contradict each other are a defect with a date. Count them, age them, and escalate on age rather than volume. A conflict left open for a month is a question your firm cannot answer the same way twice.
4. Lag from source change to approved revision
A policy changes, or a rate table, or a rule. How long until every entry that depends on it is revised and approved again? That gap is your real exposure window. It is measurable in days. It is also the indicator an examiner would find most interesting.
What a governed knowledge base makes countable
None of these numbers exist if your knowledge sits in drives, inboxes and people's heads. Cognatum is built to produce them. It scores the knowledge base and splits healthy entries from at-risk ones. At risk means more than overdue for review. It also covers duplicates, conflicting entries and plain gaps. Its Clean step checks each entry against its sources. Near-duplicates merge into one entry. Contradictions get detected rather than left to surface later. Gaps are recorded as gaps, so the assistant says there is no approved answer instead of writing one that sounds right.
Notification, not self-correction
When a source changes, every entry that depends on it is flagged and sent for re-check. The revision queues for a human approval gate. The last approved version stays live until a person signs off on the new one. Cognatum notifies. It does not quietly rewrite your approved answers, and it is not always current by magic. That is what makes the signal usable. A flag has an owner, a date and an age.
Where the regulators are already pointing
Article 9 of the EU AI Act asks for a risk management system for high-risk AI. It calls that system a continuous iterative process, run across the whole lifecycle, reviewed and updated on a regular basis. Paragraph 8 goes further. Testing has to run against metrics and thresholds set in advance. Metrics and thresholds, for the system. No framework yet sets either one for the content the system reads. So the indicator is yours to define, before somebody asks for it.
A test you can run this week
Open your KRI inventory and count the indicators whose data source is your knowledge base. Then take a question from your last examination. Put it to the assistant your team already uses. Ask for three things: the approved entry behind the answer, the person who approved it, and the date. If nobody can produce them inside a minute, you are carrying an exposure with no warning light on it.
Your company's knowledge isn't missing. It's unusable. Cognatum changes that.
Sources
- Key Risk Indicators (KRIs): A Complete Guide for 2026 (metricstream.com)
- Key risk indicators (KRIs): An overview (thomsonreuters.com)
- Key Risk Indicators, the right metrics for management (riskonnect.com)
- Key risk indicator (wikipedia.org)
- Revisions to the principles for the sound management of operational risk (bis.org)
- EU AI Act Article 9, risk management system (europa.eu)
- APQC study on the Great Retirement and the role of knowledge management (apqc.org)
- Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (arxiv.org)
- The Cognatum Knowledge Loop (cognatum.ai)
Common questions
Questions this raises.
What is a key risk indicator?
A quantifiable metric that gives early warning that risk exposure is rising, before a loss or an incident. Unlike a performance indicator, which reports on what already happened, a KRI is forward-looking and carries a threshold plus an escalation path naming who acts.
Can a knowledge base be a KRI data source?
Yes, provided the knowledge is governed. A governed entry carries a review date, a named approver and a source, which makes overdue review, duplicate rate, open conflicts and revision lag all countable. Loose files on shared drives emit nothing to count.
How is this different from AI observability?
Observability watches the system: latency, cost, traces, prompt and response logs. It can show that an assistant answered quickly and name the document it used. It cannot tell you that the document was superseded in March, because approval state is not a field it reads.
What threshold should we set?
Start from your own policy rather than a benchmark. If your document control procedure sets a two-year review cycle, then entries past review date above a low single-digit percentage already breaches your own rule. Thresholds should cascade from the risk appetite statement, as they do for every other indicator.
Does Cognatum keep the knowledge current automatically?
No. It detects that a source has changed, flags every entry that depends on it, and routes a drafted revision to a named approver. The last approved version stays live until a person signs off. Notification and a human gate, not silent self-updating.