Security Experts:

Connect with us

Hi, what are you looking for?


Application Security

Research: Simulated Phishing Tests Make Organizations Less Secure

A large-scale, long-term phishing experiment conducted in a 56,000-employee organization has come to a startling conclusion: Those simulated phishing tests commonly seen in corporate user-education campaigns are actually making things much worse.

A large-scale, long-term phishing experiment conducted in a 56,000-employee organization has come to a startling conclusion: Those simulated phishing tests commonly seen in corporate user-education campaigns are actually making things much worse.

After a 15-month phishing experiment done in partnership with an unnamed publicly traded global company, researchers at ETH Zurich found that embedded training during simulated phishing exercises did not make employees more resilient to e-mail malware lures and, worse, “can have unexpected side effects that can make employees even more susceptible to phishing.”

The results of the landmark experiment was published this week by the Department of Computer Science at ETH Zurich, a public research university based in Switzerland. It calls into question the expanding corporate spend on user education campaigns that combine simulated phishing attacks with training videos and mandatory quizzes.

Even as investors pour money into startups promising respite from phishing attacks, corporate defenders struggle to block sophisticated email lures that serve as the initial entry point debilitating malware and ransomware extortion attacks. For decades, businesses added user awareness training to cybersecurity budgets in attempts to help employees to spot suspicious links or malicious email attachments.

[ PREVIOUSLY:  Investors Race to Find Profits in Securing Corporate Email ]

There has been active debate over the years about the effectiveness of security awareness training but that has not stopped CISOs (Chief Information Security Officers) from flagging user education as a major priority for cyber defense spending.

The ETH Zurich university study is sure to rekindle that debate with the researchers calling for caution in the deployment of methods like embedded phishing exercises and training, warning of “potential negative side effects.” 

The researchers ran the experiment for 15 months (July 2019 through October 2020) where simulated phishing emails were sent to a quarter of the partner company’s workforce during their normal work flow and context. 

The researchers carefully measured click rates from the phishing lures, submission of credentials, and the enabling macros on attachments.  The team also deployed a reporting button to the corporate email client to allow the employees to click and report suspicious emails.

The raw data from the study tell an interesting story:

  • Overall, the study participants clicked on 6,680 out of 117,864 simulated phishes (5.67%). During the 15 months, 4,729/14,733 participants (32.10%) clicked on at least one phish. 
  • The trend for dangerous actions is similar, with the numbers slightly lower: participants fell for 4,885 simulated phishing emails (4.14% of the total sent emails, and 73.13% of all the clicked simulated phishes), and 3,747/14,733 participants (25.43%) users did at least one dangerous action. 
  • There were 4,260 study participants that reported at least one email. In total, the participants reported 14,401 emails, of which 11,035 were our simulated emails. The button to report phishing was also deployed to 6300 employees that were not part of the experiment but could report phishing: 1,543 of them reported at least one suspicious email, and they reported 4,075 emails. Thus, the total number of reported emails we received during the 15 months was 18,476.

[ READ: Security Awareness Training Debate: Does it Make a Difference? ]

The researchers also found “repeated clickers” who fell victim to multiple lure mails and concluded that many employees in an organization “will eventually fall for phishing if continuously exposed.”

In the experiment, the researchers found that 4,729 out of 14,733 (32.10%) participants clicked on at least one link or attachment in simulated phishing emails. A similar high number applies to dangerous actions: 3,747 out of 14,733 (25.43%) performed at least one. 

“These results indicate that a rather large fraction of the entire employee base will be vulnerable to phishing when exposed to phishing emails for a sufficiently long time. We are the first to show such results at scale,” the researchers added.

[ SEE: Report: Security Awareness Training Top Priority for CISOs ]

The study, which was supported by the Zurich Information Security and Privacy Center (ZISC), also found that crowd-sourced phishing detection can be effective and practical in large organizations. 

“Our experiment shows that crowdsourced phishing detection enables organizations to detect a large number of previously unseen real phishing campaigns with a short delay from the start of the campaign,” the team said, noting that the operational load of phishing report processing can be kept small, even in large organizations. 

“Our study also demonstrates that a sufficiently high number of employees report suspicious emails actively over long periods of time. In summary, we show that crowd-sourced phishing detection provides a viable option for many organizations.”

Related: Investors Race to Find Profits in Securing Corporate Email

Related: Security Awareness Training Debate: Does it Make a Difference?

Related: Report: Security Awareness Training Top Priority for CISOs

Written By

Ryan Naraine is Editor-at-Large at SecurityWeek and host of the popular Security Conversations podcast series. He is a security community engagement expert who has built programs at major global brands, including Intel Corp., Bishop Fox and GReAT. Ryan is a founding-director of the Security Tinkerers non-profit, an advisor to early-stage entrepreneurs, and a regular speaker at security conferences around the world.

Click to comment

Daily Briefing Newsletter

Subscribe to the SecurityWeek Email Briefing to stay informed on the latest threats, trends, and technology, along with insightful columns from industry experts.

Join this webinar to learn best practices that organizations can use to improve both their resilience to new threats and their response times to incidents.


Join this live webinar as we explore the potential security threats that can arise when third parties are granted access to a sensitive data or systems.


Expert Insights

Related Content

Application Security

Cycode, a startup that provides solutions for protecting software source code, emerged from stealth mode on Tuesday with $4.6 million in seed funding.

Management & Strategy

SecurityWeek examines how a layoff-induced influx of experienced professionals into the job seeker market is affecting or might affect, the skills gap and recruitment...


Less than a week after announcing that it would suspended service indefinitely due to a conflict with an (at the time) unnamed security researcher...

Data Protection

The CRYSTALS-Kyber public-key encryption and key encapsulation mechanism recommended by NIST for post-quantum cryptography has been broken using AI combined with side channel attacks.

Data Protection

The cryptopocalypse is the point at which quantum computing becomes powerful enough to use Shor’s algorithm to crack PKI encryption.


The changing nature of what we still generally call ransomware will continue through 2023, driven by three primary conditions.

Data Breaches

LastPass DevOp engineer's home computer hacked and implanted with keylogging malware as part of a sustained cyberattack that exfiltrated corporate data from the cloud...


WASHINGTON - Cyberattacks are the most serious threat facing the United States, even more so than terrorism, according to American defense experts. Almost half...