How to Manage False Positives with SMARTFENSE

False Positives

This article explains how SMARTFENSE detects, manages, and hides software-generated statistics (false positives) within campaigns. It also details the available reports and settings to ensure result accuracy and maintain the reliability of user metrics.

Software-Generated Statistics

Software-generated statistics are a phenomenon independent of the simulation tool used. They are present in almost all current simulations, as most organizations use email security tools.

A false positive is a statistic automatically generated by software but recorded under a user's name. This situation is common and can affect the accuracy of campaign results. If your simulation tool does not detect these cases, reports may include actions users did not actually perform.

SMARTFENSE incorporates robust and proven algorithms to detect these statistics and provide reliable results in campaigns, constituting a differential advantage over other solutions.

Statistics Analysis

Audit - Campaign Detail

Any campaign affected by software-generated statistics is highlighted so that the administrative user can identify this situation. This can occur in both Newsletter campaigns and Phishing or Ransomware simulations.

Summary

False Positives

From the False Positives tab, resources and reports are presented that allow analyzing and learning more details, facilitating the configuration of hiding. This view shows whether false positives are hidden or visible, with direct access to modify the setting.

Additionally, from this same section, it is possible to reprocess the statistics after adjusting the hiding settings, obtaining updated and reliable data showing only the users' real actions.

List of Software-Generated Interactions

This report lists the false positives recorded in the campaign. The information includes:

• Number of false positives detected.

• Users affected by each case.

• Origin of each false positive.

The data can be exported to Excel or CSV for analysis, facilitating duplicate removal and identification of tools generating false positives.

Number of Interactions per User

This report shows all users assigned to the campaign along with the number of interactions registered under their name.
SMARTFENSE automatically detects most false positives; however, each organization may present particular cases not automatically detected.

If a user has an unusual number of interactions, they will be highlighted, as they are likely affected by false positives not automatically identified.
The report can also be exported to Excel or CSV.

IP Addresses that Interacted in the Campaign

This report shows the IP addresses from which interactions with the campaign have been generated. For each IP, it details:

• If it is related only to user interactions.

• If it is related only to false positives.

• If it is related to both types of interactions.

The information can be exported to Excel or CSV to facilitate analysis and identification of tools generating false positives.

How SMARTFENSE Identifies Software-Generated Interactions

This online resource provides clear information about:

• Whitelist Process

• Software-Generated Statistics

• How does SMARTFENSE detect the interaction of a security tool?

• Interactions coming from IPs or user-agents configured in Whitelist

• Activation of special links

• Analysis of previous interactions 

Reprocess Campaign Statistics

If the filters for hiding software-generated statistics are modified in the False Positives section, the changes only affect statistics recorded after the modification.
However, it is possible to reprocess the current campaign's statistics to:

1. Mark as “Software-Generated Statistics” all statistics received from the currently configured User Agents and IP ranges.

2. Mark as “Software-Generated Statistics” all statistics occurring 3 seconds before those interactions.

Please note that the risk scoring and the users' heat map may change after performing this action.

To reprocess the statistics, press the blue button Reprocess this campaign's statistics and then Accept.
When finished, the platform will automatically return to the Summary view with updated data in the funnel chart and hidden false positives.

View of Users Assigned to the Campaign and Their Actions

From the Users Assigned to the Campaign view, you can consult the table with users and their interactions.

By clicking on a specific user, their interaction history is displayed, indicating whether they are of user or software type.

If hiding is configured in Settings > Security > False Positives, only user type actions will be shown.
To view all interactions, configure that false positives, i.e., software actions, are shown in that section.

In each user's history, the platform can show the status in green No warnings, or in red with the warning that the user has been affected by false positives.

Example without warnings:

Example with false positive warning:

For each user interaction, the following is shown:

• Type of Action (User or Software). 
• IP address.
• Whois (accessed via the Whois Query button).
• User Agent.
• Actions, with the Mark as false positive button.

When pressing this Mark as false positive button, you must choose one of these options:

1. Mark only that interaction as a false positive.

2. Mark all interactions from a specific IP address (x.x.x.x) as false positives, both in the current campaign and future ones.

3. Mark all interactions from a specific User Agent (x) as false positives, both in the current campaign and future ones.

Settings

From the Settings Menu > Security > False Positives, you can define whether to show or hide false positives in the platform's various reports.

Options:

• Show false positives
• Hide false positives

Seconds Range

The seconds range defines the tolerance level for a false positive.
When a software-generated statistic is detected, interactions that occur within the configured range will also be marked as false positives.

Important: if this range is modified, the change will only affect future interactions.
To apply it to already recorded statistics, the corresponding campaign must be reprocessed.

Options:

• Activate seconds range (recommended)

• Deactivate seconds range

Simulation Reporting via the Phishing Report Button

In Phishing or Ransomware simulation campaigns, after a user makes a report using the Phishing button, it is possible to:

• Continue analyzing all interactions after the report.

• Mark all interactions occurring after the report as false positives.

This setting ensures that once a user identifies and reports a simulation, interactions automatically generated by software are treated as false positives.
The logic is applied individually: if a user reports a simulation, this does not affect the collection of statistics for other users.

User Agent Configuration

This setting allows specifying which User Agents should always be marked as false positives.
Each time an interaction is recorded, it will be checked whether the User Agent matches any of those entered.

It is not necessary to enter the full User Agents.
For example: if the string Chrome 83.0 is recorded, an interaction with the User Agent “PC / Mac OS X 10.15.4 / Chrome 83.0.4103” will be marked as a false positive.

IP Configuration

Allows specifying which IP address ranges should always be marked as false positives.
Each time an interaction is recorded, it will be checked whether its IP address belongs to any of the defined ranges.

💡 Recommendations

• Regularly review False Positives reports to maintain data integrity.

• Use statistics reprocessing after modifying IP or User Agent settings.

• Activate the seconds range to improve accuracy in detecting false positives.

• Export reports to Excel or CSV for deeper analysis and internal audits.