By Lionel Snell
Editor, NetEvents
Call it alert fatigue. Call it information overload. Call it mind-killing and soul-destroying. The sheer number of alerts coming into a modern security operations center (SOC) can overwhelm even the most dedicated security analysts.
Alerts pour in from many dashboards and security information and event management (SEIM) platforms, with some focused on the network, others on endpoints, some on the firewall and outside-facing servers, and others on critical infrastructure. And with the vast majority of alerts being (fortunately) false alarms, it can be easy to overlook the real warning signs… which may be subtle indications of malicious reconnaissance or an actual breach.
As SC Magazine’s Greg Masters writes in “Crying wolf: Combatting cybersecurity alert fatigue,” nearly three-quarters of security teams stated they were overwhelmed by the volume of vulnerability maintenance work assigned to them. When security teams were queried about contending with threat alerts, 79% said they were overwhelmed by the volume.
And according to Ryan Francis in “False positives still cause threat alert fatigue,” published in CSO, “The Cisco 2017 Security Capabilities Benchmark Study found that, due to various constraints, organizations can investigate only 56 percent of the security alerts they receive on a given day. Half of the investigated alerts (28 percent) are deemed legitimate; less than half (46 percent) of legitimate alerts are remediated. In addition, 44 percent of security operations managers see more than 5000 security alerts per day.”
What can you do? What must you do? Reinvent the SOC. Business as usual simply can’t cut it. Fortunately, there are companies working on this very challenge. Cylance pioneered the application of artificial intelligence (AI), algorithmic science, and machine learning to prevent the most sophisticated security threats. Demisto’s security operations platform combines security orchestration and incident management with machine learning from analyst activities, and interactive investigation. JASK too applies enhanced AI and machine learning to automate the correlation and analysis of threat alerts.
Other companies like CA Technologies have specialist departments addressing these issues. CA’s SVP Central Software Group, Dr Vinod Peris, points out that data has typically been something to look back on with hindsight: “What we are doing with AI is to be more predictive. We’re looking not just at what you’ve missed as red flags, but alerting you that you’re likely to miss”. In the case of card payment security, they use behavioural analytics to assess the gap between the transaction and expected behaviour and warn the bank.
People first
Neither Demisto nor JASK make alert fatigue their starting point. Their first concern is human resources – the lack of qualified security analysts, and a company’s sheer inability to recruit, retain, and afford them. And of course, keep them from burning out.
“The biggest problem that SOCs are having right now is talent,” says Greg Fitzgerald, Chief Marketing Officer at JASK. “One is recruiting just people. Second of all is having the skillsets to just place in those jobs, and then the third piece is the experience, those that actually know what to do when they find something inside the SOC.”
Demisto’s CEO, Slavik Markovich, agrees. “When you talk with analysts and you see them day in and day out, just handling all those incoming alerts, and going through, like, tens of different tools, it burns them out.”
Markovich continues, “We looked at how analysts are working, and man, they’re not happy. After six months, they’re ready to run away. The average, probably, for an analyst is less than two years. The reason is that, because they’re doing the same thing over and over again. Just, nobody wants to operate like that.”
In addition to the tedium, says Fitzgerald, is the lack of opportunity in many organizations. While there are many analysts, there aren’t many spots for promotions. “What needs to happen is the same thing that would happen in any job, which is they want career advancement.,” he says. “What we are seeing today is that the security operations person who has that initial job, once they get educated to understand both the process and the experience, even with a year or two, quickly leave the company. So, organizations spend a lot of time and effort getting a person up to speed, and then they leave.
The solution there, Fitzgerald says, “Make it so they have an upward career path within where they are so they can get out of the mundane job, and start doing something much more proactive about threat hunting, or actually just seeking resolution to the problem they have, or being a part of an incident response team. It’s much more like the elite staff that any IT and security personnel wants to do.”
Addressing Alert Overload
You’ve got to address alert fatigue. Before enterprises can offer more interesting and challenging projects for security analysts, that fire hose of SEIM notices and log anomalies must be made more manageable – both in quantity and in the ratio of false alerts to real incidents.
In the words of Greg Martin, JASK CEO and Co-Founder we need: “to filter the advanced attacker from all of the noise of automated lower-level cybercrime attacks. This is where the industry is really struggling right now: how do I identify what I should care about versus the malware that I see every Monday?”
Cylance’s Kumad Kalia pointed out that, despite the publicity about sophisticated attack innovations, the more common tactic is simply to overwhelm security with a flood of more basic attacks: “Multiple exploits put together so, even if you detect one, you might not think to look in the other place. Sometimes, one attack will be used to overwhelm some resources to hide another stealthier attack underneath”.
Such automated attacks are best dealt with by automated response: “The future is going to be where AI is at the heart of the solution so that you’re not being overwhelmed by that amount of information, that the AI engine in the prevention tool is doing all that heavy lifting.”
“Technologies for preparing and triaging and responding automatically,” are key for Demisto’s Markovich. “Those technologies orchestrate and automate across hundreds of different security tools, and bring the data, fully prepared and analyzed, to the analyst.”
With that data, the analyst can review the recommendation from the security tool, and either allow automation to continue to handle the incident, or choose human intervention. “Triage would be look at the threat intelligence info about the incident, look at the file properties, maybe detonate the file, do all of those things,” adds Markovich. “Then the analyst says, okay, yeah, I think it’s malicious, and then the response automation should be, okay, eradicate this email, block this end-point, block this IP, and so on and so forth.”
The upshot: The technology takes boring, tedious manual labor out of the equation, and “and just allows the analyst to focus on what he’s good at, which is the decision-making and the actual smart hunting and thinking about security,” says Markovich.
Smarter tools can also help with a key element of triage: choosing which alerts to focus on first. “Analysts are overwhelmed with what they have to see today, and they need some sort of prioritization,” says JASK’s Fitzgerald. “It’s not just what’s important. It’s also where to start. Because an attack or a compromise can be caught at any point in the sequence, and so they need some guidance to say, help me, and that’s what happening.”
AI to the Rescue
Leading cybersecurity companies are leveraging artificial intelligence and machine learning in their next-generation SOC platforms. These technologies will enable automatic filtering of threat reports, allow correlation of alerts across platforms, evaluate the dangers, present recommendations – and lead to automatic remediation.
Machine learning is a key component, because malware moves too fast to allow security systems to be trained after the event. Kumad Kalia gave the example of a Cylance system that had not been updated for two years yet could still detect the latest attack patterns. “That’s a profound demonstration of the efficacy of AI within cybersecurity… our code had never seen these types of software – probably hadn’t even been written in the combinations that were then released for attack – and the software stopped these on machines.”
Where will this go? To a solution that reinvents the SOC, with triage and front-line reporting done in real time by software – not by burned-out humans.
Imagine, says Markovich, a SOC with a single pane of glass where the analyst gets alerts already ordered in a queue. All the alerts are already processed by AI, and are presented with all the context and data needed for a human judgment. “The analyst makes a quick decision, almost like Tinder: Swipe left, swipe right, block or it’s okay.”
The action is then done by the SOC platform, so the entire response is being done automatically. Goodbye, non-stop information overload. Goodbye, mind-numbing and soul-destroying alert triage. Finally, we can cure the alert-fatigue epidemic.
