Incident Closure Validation · Data Defence Consulting

The pain

Every closure is a decision. Almost none of them are checked.

The detection fired. Somebody, or something, decided it was nothing and closed it. From that moment the incident record says the threat was handled, your metrics agree, and nobody looks at it again. The detection was right. The decision may not have been.

Automation closes at machine speed, with no second pair of eyes

A playbook that closes an account-compromise incident as benign keeps the queue moving. It also makes a security decision that nothing reviews, and it makes the same decision on the same account again next week.

A wrong closure is an active threat that nobody is watching

A real compromise closed as harmless does not go away. It sits behind a green tick while the attacker keeps their access. It is the one incident nobody is looking at, and that is the worst place for it to be.

The account gets fixed later, and the record never learns

Often the compromise is quietly handled off the back of the incident by a separate remediation process. The account is reset, the incident stays labelled benign, and the gap never shows up anywhere you would measure it.

The honest problem

This is not a story about a bad team. In the engagement below, the people were right 80% of the time and missed nothing. The automation was wrong 42% of the time. That is a specific, fixable finding, and it is invisible until somebody re-reads the closures, because the only record of the decision is the decision itself.

The promise

We re-check every closure, whoever made it.

An independent, read-only re-investigation of your already-closed Sentinel incidents. We start from the raw events, not from the closing note, and we reach our own verdict on each one. Then we tell you where the recorded classification does not match what the evidence shows.

Every closure re-investigated from the evidence

What the alert actually fired on, what the raw events show about reach, success and source, and whether the activity landed. The closing summary is the last thing we read, not the first.

Both closure paths, and we tell you which was which

Analyst closures and playbook closures are validated the same way, and every finding says who or what made the call, so the fix lands in the right place: a person's judgement or a closure rule.

Confirmed mis-closures, not suspicions

Where a true positive was closed as benign we show it from the evidence, corroborated by any containment that was taken on the account or device. Where nothing was contained, the evidence still stands on its own, and we say so.

Did the threat persist after closure?

For every confirmed mis-closure we check whether the activity continued after the incident was shut. A wrongly closed incident can leave a live compromise running, and that is the finding you need first.

What this is not

It is not an audit of your analysts. In every run so far the human path has been the reference standard, and the findings point at the automation and the process around it. It is not a change to anything: the whole review is read-only, we never modify a workspace, a rule or an incident, and every flag comes back to your team as a candidate to re-review, not a verdict imposed on you.

The proof

Evidence, not adjectives.

One engagement, anonymised. We re-checked every High and Medium incident one estate had closed in a single week, so nothing below was selected to look good.

13 of 31

incidents an automation had closed as benign were genuine compromises.

17+

genuine account compromises recorded as harmless in one week, found by re-reading every closure.

80% vs 42%

the human path was right 80% of the time and missed nothing. The automated path was wrong 42% of the time.

What the week looked like

Every High and Medium incident closed in seven days was re-investigated, split by who or what closed it. The analysts' closures held up: 80% were right, and not one real threat was missed. The automation's closures did not: 13 of 31 were genuine compromises marked benign.

Why it matters this year

An incident closed as benign never starts the clock.

Under UK GDPR a personal data breach has to be reported within 72 hours of becoming aware of it. An account compromise that was closed as harmless may be a reportable personal data breach that nobody is aware of, so the clock never starts, and the first time anyone finds out is when something else goes wrong. The same logic runs through the assurance regimes that ask you to show your incidents were handled: the evidence has to exist, and a closure with no rationale is not evidence.

Direction of travel

The Cyber Security and Resilience Bill is expected to widen reporting duties and bring managed service providers into scope directly. When it passes, the question "how do you know your closed incidents were really closed?" stops being a good practice question and becomes one you will be asked. None of these regimes names closure validation. They require the outcome and the evidence, and this is how you get both.

This section is general information, not legal advice. Take your own legal advice on your reporting obligations.

How it works

One week of closures. Every one re-read.

Scope

The standard run takes the High and Medium incidents closed in the last seven days, typically 40 to 50, with read-only access.

Re-investigate

Each incident is reconstructed from the raw events, compared against a second automated risk check and any containment taken, and traced forward to see whether the threat persisted after closure.

Return three lists

Confirmed mis-closed, to re-open now. Needs review, flagged with no containment seen, worst first. Looks correct, with the assurance recorded. Every finding names who or what closed it.

Request a validation run

Find out what your closures have been hiding.

Tell us what you run and who closes your incidents, and we will come back to arrange a short, no-cost call. If we do not think a run would tell you anything, we will say so.

A short call to confirm fit, at no cost and no obligation.
Read-only throughout. Nothing in your workspace is changed.
Your details are used to contact you about this enquiry only.

Full name *

Please tell us your name.

Work email *

Please enter a valid work email address.

Organisation *

Please tell us where you work.

Job title

Who closes most of your incidents?

Not sure is a perfectly good answer. Working out who closed what is part of the run.

What would you like to achieve?

We reply within one working day. A 90-day look-back or a standing monthly check is available once fit is confirmed.

Mock-up only, nothing was sent

This is the payload the page would POST to the Azure Function once wired up:

Thank you, that is with us.

We will come back to you within one working day to arrange a short call.

Incidents were closed as safe. Nobody checks if they were right.