The debate about AI in security quickly slides into two extremes: that it changes everything, or that it changes nothing. It is more useful to ask where a statistical model has an advantage over a rule, and where it does not.
Where it delivers measurable benefit
Reducing noise
Grouping similar alerts, removing duplicates and ranking by likelihood of a genuine finding. That is a task with many examples and a clear outcome, and models do it well. The benefit is not that the model detects attacks but that the analyst does not spend a day on a thousand alerts of which nine hundred are the same event.
Detecting deviation from normal
A login at three in the morning from a country where the organisation does not operate, or an account suddenly accessing ten times more files than usual. The model learns what is normal and flags what is not. It works well where "normal" is stable and poorly in environments that change constantly.
Accelerating comprehension
Summarising logs, explaining an unfamiliar command, proposing a query over the data. Here the model does not decide but shortens the time to understanding, with a human verifying.
Preparing documentation
A draft incident report, a proposed policy structure, a comparison of an existing document against a standard's requirements. With mandatory review, this is currently the highest-return application in compliance work.
The model shortens the path to an answer, but a human confirms the answer. The moment that confirmation is removed for speed, the benefit turns into a risk - and one that stays invisible until it materialises.
Where it moves the problem instead of solving it
When the cause is organisational
A system that ranks alerts does not help an organisation where nobody looks at alerts outside working hours anyway. Detection without response is not defence, and a tool does not create an on-call rota.
When there is no data
Models learn from records. An organisation not collecting logs from all key sources will not get useful results - it will get convincing results on incomplete data, which is worse.
When output is taken as fact
Language models produce convincing text even with no basis for it. A vulnerability name that does not exist, an invented article of a regulation, a configuration parameter that never existed - all of it arrives in the same tone as the correct answer.
When sensitive data leaves
Analysing an incident in an external tool means logs, system names and sometimes client data have left the organisation. That is processing with its own legal consequences, and it happens most often under the pressure of an incident.
The attacker's side
The same technology lowered the bar for attackers in three concrete ways: phishing without language errors and tailored to the recipient, faster preparation of malware variants, and convincing voice impersonation in payment fraud.
None of these is a new class of attack. All are existing attacks executed more cheaply and more convincingly. Which is why the defence is not a new tool but tightening controls that already exist - out-of-band payment confirmation, multi-factor authentication, and awareness training that can no longer teach people to spot phishing by its bad grammar.
What to put in place before adopting
- Enter the system in the asset register, with an owner and a use case.
- Define what must not go in. Personal data, client data, configurations, logs containing system names.
- Keep a human in the decision everywhere the output affects people or service availability.
- Record what the model proposed and what the human decided. Without that record there is neither review nor learning from mistakes.
- Measure. If time to detection or time to response does not improve after adoption, the tool did not deliver what it was bought for.
The AI Act applies the same logic to security tools as to everything else: obligations follow the application. A tool that ranks alerts is generally low risk, but a tool that automatically blocks a user is making a decision about a person - and that changes the assessment.
Sources
Not sure where you stand?
Half an hour of conversation, with no obligation. By the end you know what needs doing and in what order.