This article explains why false positives occur in Phishing, Smishing, and Ransomware simulations, what their most common causes are, and how SMARTFENSE allows detecting and managing them to obtain reliable results.
Introduction
False positives in Phishing, Smishing, and Ransomware simulations are a permanent reality in practically all organizations.
This problem is independent of the tool used to simulate attacks. The difference is not whether they occur or not, but how they are detected and managed.
How False Positives Are Generated
Simulation emails include unique links that uniquely identify a user within a campaign.
These links allow recording the user's interactions and measuring their behavior.
When software accesses these links one or more times, false interactions are generated on behalf of the recipient user, resulting in false positives.
| Note: this phenomenon is not exclusive to security simulations. It also occurs in marketing tools and any system that measures human behavior through links. |
Which Tools Cause False Positives?
Short Answer
Almost all.
Detailed Answer
The origin of false positives is broad and dynamic, so it is not possible to maintain an exhaustive list. Moreover, this behavior applies to both corporate devices and personal devices.
Among the main sources are:
Security Solutions
Anti-spam and anti-phishing filters
IDS / IPS
DLP
Email security gateways
Antivirus
Continuous analysis and monitoring solutions
Archiving and Discovery
Threat Intelligence
User Environment Software
Desktop, web, or mobile email clients
Browser extensions
Inbox plugins
Link preview tools
Other web or mobile applications
How to Avoid False Positives?
In the Corporate Environment
In tools, devices, and applications under the organization's control, it is possible to try to mitigate them through Whitelisting.
It is important to clarify that the main goal of Whitelisting is to ensure the delivery of the simulation to the user.
Although it can help, it does not eliminate the underlying problem, since many tools continue analyzing emails even when whitelisted.
Furthermore, implementing Whitelisting on all involved technologies is often complex or simply impossible.
In the Personal Environment
When the user uses personal devices, the margin for action is minimal.
The only alternative would be to intervene directly on the user's device to clean or configure tools, which is neither realistic nor scalable.
How to Detect False Positives?
Option 1: Automatic Detection
The best alternative is to use a simulation tool that automatically detects false positives and alerts when a campaign is affected.
Ideally, this tool should offer:
Source IP addresses
Whois information
User-Agent of the interactions
HTTP trace that generated the false positive
Even better: having a partner who manages SMARTFENSE and handles all this management.
Option 2: Manual Detection
If you do not have a tool that detects false positives, manual detection is possible, although unreliable.
When the Problem Is Obvious
Example:
Emails sent: 1000
Emails opened: 900
Clicks on links: 900
In these cases, it is clear that something is not right.
Manual strategies can be applied such as:
Discard interactions that occurred immediately after sending.
Identify openings and clicks with minimal differences of seconds.
Even so, the campaign remains contaminated and the results are not reliable.
When the Problem Is Not Obvious
This is the most dangerous scenario.
Results seem valid but contain invisible false positives.
The organization makes decisions based on incorrect data and the problem is usually detected too late, when a user complains about being unfairly accused.
Final Thoughts
In 2019, a SMARTFENSE client detected that some users denied having interacted with simulations, even though records indicated openings and clicks.
The cause: security solutions that interacted with emails before reaching the user.
Faced with this, there were two paths:
Blame the client and recommend Whitelisting.
Develop a real solution to obtain reliable results.
SMARTFENSE chose the second path.
Thus was born the false positive detection algorithm, now robust, customizable, and constantly evolving.
Without such a feature, it is not possible to obtain reliable results in Phishing simulations, regardless of the organization.
💡 Best Practices
Assume that false positives will always exist and manage them properly.
Avoid making critical decisions based on uncleaned campaigns.
Customize the detection algorithm according to the organization's technological reality.