Why Security Awareness Training Fails

I remember being a low-level employee at a huge company. Of course, there was annual security awareness training that you had to sit through. I despised it. First of all, it was a boring-ass videos that were done in the form of a kid’s cartoon. Secondly, I had to watch it in between meetings, answering hundreds of emails (pre-Superhuman), answering calls from distributors, writing reports. Whenever a quiz appeared I wouldn’t even look at what I was clicking, because I didn’t care and I knew that it didn’t really matter what I clicked: there would be another try, or nobody would actually care to do anything even if I failed. Or tests were so goddamn obvious that you’d have to fail on purpose or have an IQ below 50.

Sounds familiar? Well, at least it should be familiar to about 75% of employees. Researchers from UC San Diego tracked how employees actually interact with training materials and found, to no one’s surprise, that employees spent less than a minute interacting with learning material. Some would close training materials immediately. The whole system is kind of a failure.

The question any sane person would ask is, well, why don’t employees try just a little bit to learn and engage with awareness training? Don’t they understand that it’s for the good of everybody?

And they don’t, but it’s not their fault, because the training itself is designed to be a regulatory checkbox (sorry for using such a cliche). Training material assumes employees are almost like kids, or is written in a way that just wastes time. I kid you not: I remember watching a video about phishing, where the first minute and a half was wasted on a bunch of managers responsible for creating this curriculum introducing themselves. You don’t need to conduct scientific research to tell me that training like that will fail to teach anyone.

Decades of studies that tell us what we already know

Let’s get back to that UC San Diego study. I believe it covered more than 19,000 employees over eight months and ten simulated phishing campaigns, and there was no correlation between how recently an employee completed their annual training and whether they fell for phishing. Embedded training was a bit better, with a 2% reduction in failure rates. But that’s not all. In the first month about 10% of employees clicked a phishing link; by month eight, more than half.

The conclusion is that anti-phishing training, at least in its current form, doesn’t work. Like at all. Researchers at ETH Zurich arrived at the conclusion that embedded training might actually make things worse by making employees overconfident. Which is kind of hilarious. I guess if you run quizzes that are very easy to pass, people think that those are the threats they will face in real life, which would explain this result. But what they did find is that regular nudges, small reminders that phishing exists and reporting matters, actually helped a bit.

There also were studies done in the Netherlands and by Harvard that basically found that training didn’t help. It either was short-term or didn’t help at all. Which, again, is not that surprising at all. You can’t learn how to play a guitar if you practice once a year, and it’s actually more effective to practice 15 minutes a day, but every day.

This is purely anecdotal, but after my wife’s Instagram account was taken over, what I did was come up with short, tweet-long educational materials that I would send her every day before work, first thing in the morning. That actually helped, not to the extent I hoped, but it was not nothing. At least some basics of 2FA, password management and account management sank in, which is better than nothing.

There are reports from huge vendors like KnowBe4 that show meaningful improvement, but it’s not a controlled study, and obviously there’s some marketing incentive to show that their training works. Nothing wrong with that, data is data, but healthy skepticism is warranted. The same goes for IBM’s Cost of a Data Breach report, which also shows that there’s a correlation between training and lower breach costs, roughly a million dollars of difference between high-training and low-training organizations.

Build for auditors

So what’s the deal? Based on some research, overall global trends in increased regulation of everything and good old intuition, I think it’s just that the training is designed to satisfy regulatory demands and that’s it.

I know a couple of smaller companies and startups, where my friends work, that initially, even at top level, would just blatantly laugh at the idea of awareness training. Like, the top brass was so overconfident that the whole idea of training seemed stupid to them. And as far as I know, they didn’t care about endpoint security, password management, etc. Of course, an incident happened: the CEO fell for a phishing email. Then a whole process started to make things more secure, with various degrees of success. Nobody was happy with the actual awareness training, but if you want insurance, you have to have at least something to show as proof. A CISO needs a green flag to show the board, a vendor sells a platform that has courses and produces reports, and an auditor accepts such reports. Every actor in that chain gets exactly what they need. The point of all this is not behavior change, but documented evidence that training took place.

But don’t get me wrong, I don’t think anybody in this chain actually wants to have a cybersecurity incident. It’s just that the reality of the whole situation is that it’s much easier to get a compliance check than to actually figure out how to effectively train employees. And hopefully it will have some positive impact.

But things start to change. With the introduction of NIS2, organizations are required to show not only that training happened, but that it was effective.

One does not simply find time for awareness training

This issue is kind of obvious, maybe even more obvious than the first one. Where the hell do you even find time for awareness training? Again, purely anecdotal, but I remember my managers breathing down my neck with all kinds of stuff, and when it was time for annual awareness training, we would get an email telling us that we needed to go through awareness training, and some IT guy would also remind you if you didn’t do it. Top management didn’t really care; awareness training was something required by someone who was not a direct manager and didn’t interact with employees day to day. Leadership was either busy with leadership stuff or sitting in endless meetings, even before COVID. There was no buy-in at all. And if they didn’t care, why should employees?

And also, we already work with a high cognitive load, having to juggle multiple tasks, and imagine another one landing in your inbox. The human brain actually is bad at quickly switching tasks, and if you quickly switch from one task to another in the middle of the process, your brain keeps running the previous task in the background.

“Lack of time” has been the number one barrier to training for three consecutive years in TalentLMS’s surveys. Gallup found that 41% of employees name time demands as the biggest obstacle to learning, and 89% of chief HR officers agree that time is the core problem.

Let’s be honest: an overloaded employee will at best just multi-task through training. If it’s a video, it will be sped up and played in the background. I believe there’s data that shows activity hits a peak during mandatory campaigns, but once they’re done, employees never even log in to LMS platforms. Completion under obligation is nothing more than coercion.

There are also cases where companies demand high performance where employees cannot deal with the workload without resorting to multi-tasking, force employees to finish training, and then punish them when they find out that employees multi-tasked and combined training with work, or went through several courses at the same time!

Employees don’t reject awareness training because they’re lazy or stupid; they reject it because the effort required to engage with it at best has speculative returns.

Okay-okay, here’s what actually might work

Cognitive research going back to Ebbinghaus tells us what might actually work. Remember my example with a guitar? If not, let me repeat: it’s better to practice 15 minutes a day, every day, than to practice once a year.

You probably already forgot about this example, but once I brought it up again you remembered that I used it in the beginning. And it’s short and simple enough to remember. So… I think that’s the way to go: short bite-sized training with spaced repetition. There are even studies that prove that, but I don’t think we need them; just think about EVERY SKILL YOU HAD TO LEARN. I’m willing to bet money that you only got good at any particular skill through repetition, doing small tasks, repeating them and moving on to bigger tasks.

Another area where we can improve things is targeting and specificity. I mean, a person who builds infrastructure that handles user data in the EU probably should be intimately familiar with GDPR, but maybe not every employee at a company needs to know all of the details and sit through multiple boring videos.

There are also other things, like delivering training not through another app, but maybe through Slack or Teams, or even email. Basically any tool an employee is familiar with and is already using. Or maybe try rewarding correct behavior like reporting, even for false positives, and generally focusing on behavior rather than vanity metrics.

Sounds simple enough

Implementing something like this will probably not be easy. I focused on obvious things, but if we’re being honest, there are so many other things, like maybe making top management lead by example and trying to develop a culture of security. Finding the right SAT vendors is also not that easy (maybe we’ll do a rundown of vendors we think are decent or even good). There are a lot of things that require thought.

We’d like to give all of the answers, but the truth is, we’re kind of at the stage of just acknowledging the problem, and we haven’t even talked about AI…