How to Manage False Positives with SMARTFENSE

This article explains how SMARTFENSE detects, manages, and hides software-generated statistics (false positives) within campaigns. It also covers the available reports, configuration suggestions, and options to ensure result accuracy and maintain the reliability of user metrics.

Software-generated statistics

Software-generated statistics are a phenomenon independent of the simulation tool used. They are present in almost all current simulations, since most organizations use email security tools.

A false positive is a statistic generated automatically by software but recorded under a user's name. This situation is common and can affect the accuracy of campaign results. If your simulation tool does not detect these cases, reports may include actions that users did not actually perform.

SMARTFENSE incorporates robust, proven algorithms to detect these statistics and deliver reliable campaign results.

Audit – Campaign detail: False positives tab

Any campaign affected by software-generated statistics can be analyzed in detail from the False positives tab within Audit > Campaign detail. This view is available for both Newsletter campaigns and Phishing or Ransomware simulations.

False positives status

This section provides a summary of the current false positive status for the campaign:

• False positives detected: total number of interactions identified as software-generated in this campaign.
• Last reprocessed: date and time of the last statistics reprocess. If never reprocessed, it shows "No reprocesses recorded."

The Reprocess statistics button is also available in this section, allowing you to apply the current false positive configuration to all statistics already recorded in the campaign (see below).

Software impact

This section shows how many software interactions were detected per activity type, and how many users were excluded as a result:

• For each campaign metric (Opens, Clicks, Data entries, Reports):
  • The number of recorded interactions.
  • If software is detected: how many users excluded by software (indicated with ⚠️ and a red arrow).
  • If no software is detected: shows No software detected (✅).

From this section you can access:

• Interaction list: opens the Software-generated interactions report, which lists all false positives recorded in the campaign. The information includes: the number of false positives detected, the users affected by each one, and the origin of each false positive. The table shows the columns Date, User, Email, Action, IP, and User Agent, and can be exported to Excel or CSV.
• How they are identified: informational resource explaining how SMARTFENSE detects software-generated interactions. It covers the following detection mechanisms:
  • Whitelist process: the importance of running the whitelist process on security tools before launching a simulation, so the email reaches the user's inbox and no false statistics are generated.
  • Software-generated statistics: how security tools interact with the unique links in simulation emails and generate interactions under users' names.
  • Interactions from IPs or User Agents configured in False Positives: when SMARTFENSE receives an interaction, it checks whether it comes from an IP or User Agent configured in Settings > Security > False Positives and automatically catalogs it as software-generated.
  • Activation of special links: simulation emails contain special links that are not activated by real users. If an interaction from one of these links is recorded, it is automatically cataloged as software-generated.
  • Analysis of previous interactions: when a new interaction is received, SMARTFENSE checks whether any software-generated interactions occurred within the seconds range configured in Settings > Security > False Positives. If at least one exists, the new interaction is also cataloged as software.
  • Interactions after the simulation report: if the simulation report option is active, interactions recorded after a user reported the simulation are cataloged as software-generated (see section below).

IP addresses that interacted

This table lists all IP addresses from which interactions were recorded in the campaign. For each IP, the type of interactions generated and a recommendation based on its classification are shown:

The table can be filtered by classification and exported to Excel or CSV.

Users and their status

This table lists all users assigned to the campaign with details of their interactions and false positives. The available columns are:

• Total interactions: total number of interactions recorded under the user's name.
• False positives detected: number of interactions identified as software-generated.
• User interactions: number of interactions considered real.
• Status: user classification based on false positive analysis:
  • ✅ No warnings: the user has no relevant false positives.
  • ⛔ Affected by false positives: most of the user's interactions were software-generated.
  • 🔍 Requires attention: the user has an unusual number of interactions that may indicate false positives not detected automatically.

Clicking View detail opens the user's interaction history, showing for each interaction whether it is of type User or Software, along with the IP address, User Agent, and the option to Mark as false positive (irreversible action).

When marking an interaction as a false positive, you can choose to:

1. Mark only that specific interaction.
2. Mark all interactions from a specific IP address, in the current campaign and in future ones.
3. Mark all interactions from a specific User Agent, in the current campaign and in future ones.

The table can be filtered by status and exported to Excel or CSV.

Configuration suggestions

This section displays recommendations automatically generated by the platform based on the campaign analysis. Each suggestion includes a direct action button to facilitate management.

The types of suggestions are:

• User agent not configured: SMARTFENSE detected software interactions from a User Agent not yet registered in the configuration. It is recommended to configure it so its interactions are always marked as software, without relying on automatic detection. Action: Manage.
• IP not configured: SMARTFENSE detected software interactions from an IP not yet registered. It is recommended to configure it to ensure its classification in this and future campaigns. Action: Manage.
• IP with mixed interactions: SMARTFENSE detected both software and real user interactions from the same IP. This case should be reviewed carefully, as it may be a source of false positives. Action: Manage.

• Interactions after the report: interactions were detected after users reported the simulation using the phishing report button, which the platform did not automatically mark as software. This is a typical sign of automatic post-report analysis. SMARTFENSE can mark all post-report interactions as false positives. Action: Activate and reprocess. This setting can be configured permanently from Settings > Security > False Positives, where you can choose between:

  • Continue analyzing all interactions that occur after the report.
  • Mark all interactions that occur after the report as false positives.

  The logic is applied individually: if a user reports a simulation, this does not affect the collection of statistics for other users.

Charts

The tab includes visualizations for analyzing the software impact on the campaign:

• Software detection impact: total software interactions detected and excluded from campaign metrics.
• Real vs. software: donut chart showing the proportion of real and software interactions out of the total.
• Interactions by activity: software vs. user: bar chart comparing software and user interactions for each activity type (Opens, Clicks, Data entries, Reports).
• Interactions over time: software vs. user: line chart showing the evolution of both types of interactions throughout the campaign.

Reprocessing campaign statistics

If the hiding filters in Settings > Security > False Positives are modified, the changes only apply to statistics recorded after the modification. To apply them to existing statistics, the campaign must be reprocessed.

When pressing Reprocess statistics and confirming with Accept, the platform:

1. Marks as "Software-generated statistics" all interactions received from the currently configured User Agents and IP ranges.
2. Marks as "Software-generated statistics" all interactions that occurred 3 seconds before those interactions.

When finished, the platform automatically returns to the Summary view with updated data.

Warning: reprocessing does not modify statistics already marked as "Software-generated statistics." This operation is irreversible: if an IP or User Agent is later removed from the configuration and reprocessed again, interactions already marked as software do not revert to their previous status as user actions. No new interactions will be created under users' names.

The risk scoring and user heat map may be modified after this action. This update is not immediate: the platform runs a recalculation process using the new information, and changes may take up to 24 hours to be reflected.

Users assigned to the campaign view

From the Users assigned to the campaign tab you can view the table of users and their interactions. If hiding is configured in Settings > Security > False Positives, only user type actions are shown. To view all interactions, configure the platform to show false positives.

In each user's history, the platform may show:

• No warnings (green): the user is not affected by false positives.
• Has been affected by false positives (red): software-generated interactions were detected under this user's name.

Settings

From Settings > Security > False Positives you can define whether to show or hide false positives in the platform's various reports.

Options:

• Show false positives
• Hide false positives

Seconds range

The seconds range defines the tolerance level against a false positive. When a software-generated statistic is detected, interactions that occur within the configured range are also marked as false positives.

If this range is modified, the change only applies to future interactions. To apply it to already-recorded statistics, the corresponding campaign must be reprocessed.

Options:

• Enable seconds range (recommended)
• Disable seconds range

Simulation reporting via the Phishing report button

In Phishing or Ransomware simulation campaigns, after a user submits a report using the Phishing report button, you can configure how subsequent interactions should be handled:

• Continue analyzing all interactions that occur after the report.
• Mark all interactions that occur after the report as false positives.

This configuration ensures that once a user identifies and reports a simulation, interactions automatically generated by software are treated as false positives.

The logic is applied individually: if a user reports a simulation, this does not affect the collection of statistics for other users.

This option is configured in Settings > Security > False Positives, under the Simulation reporting section.

User Agent configuration

Allows you to specify which User Agents should always be marked as false positives. Each time an interaction is recorded, the platform checks whether the User Agent matches any of the configured entries.

It is not necessary to enter complete User Agents. For example: if the string Chrome 83.0 is registered, an interaction with the User Agent "PC / Mac OS X 10.15.4 / Chrome 83.0.4103" will be marked as a false positive.

Modifications only apply to future interactions. To apply them to previous campaigns, the campaign statistics must be reprocessed.

Important: marking interactions as software through reprocessing is an irreversible operation. If the User Agent is later removed from the configuration and reprocessed again, interactions already marked as software do not revert to being considered user actions.

IP configuration

Allows you to specify which IP address ranges should always be marked as false positives. Each time an interaction is recorded, the platform checks whether its IP address belongs to any of the defined ranges.

Modifications only apply to future interactions. To apply them to previous campaigns, the campaign statistics must be reprocessed.

Important: marking interactions as software through reprocessing is an irreversible operation. If the IP is later removed from the configuration and reprocessed again, interactions already marked as software do not revert to being considered user actions.

💡 Best Practices

• Review the Configuration suggestions section in each campaign: the platform automatically identifies User Agents and IPs that should be configured as software.
• Use Manage from the suggestions to configure IPs and User Agents directly, without navigating to Settings > Security manually.
• Enable the seconds range to improve false positive detection accuracy.
• Use Reprocess statistics after modifying IP or User Agent configurations to apply changes to already-completed campaigns.
• Export the Users and their status and IP addresses that interacted reports to Excel or CSV for internal audits.
• Pay attention to users with Requires attention status: they may be affected by false positives not detected automatically.