Skip to main content

How to configure a custom mail server in SMARTFENSE

  • Updated

This article explains how to configure email sending from a custom mail server in SMARTFENSE. It brings together in one place all the configuration methods supported by the platform, so you can choose the one that fits your organization's mail server infrastructure.

When to use your own mail server

SMARTFENSE can use your own mail server to send welcome emails, content assignment notifications, reminders, badges, reports, and system notifications.

Phishing and Ransomware simulation emails are always sent from SMARTFENSE servers to protect your server's reputation. This configuration applies to all other platform communications.

Available configuration methods

Select the method that matches your infrastructure to go directly to the instructions:

  1. Through password
  2. Through OAuth 2.0 for Microsoft Exchange Online
  3. Via OAuth 2.0 for Google
  4. Google as SMTP mail server
  5. Office 365 as SMTP mail server
  6. On-premise Exchange Server via SMTP

1. Through password

Configuration for sending emails from a custom server using username and password authentication. The platform connects to your organization's SMTP server to send notifications.

Access path

  1. Log in to SMARTFENSE with an administrator account.
  2. Go to Settings > Email server.
  3. Select the Own email server option.
  4. Choose Through password as the type of authentication.

Email server data

Fill in the following fields with your organization's SMTP server details:

  • Email server: domain name or IP address of the server that will handle sending.
  • Port: port through which the platform will connect to the server.
  • Encryption protocol: defines whether the connection will use encryption. The platform supports TLS or SSL.

Using encryption is recommended in all scenarios, as many emails sent by SMARTFENSE contain private user data.

Authentication data of the email account

The platform supports authentication with an email address and password. If the server does not require authentication, set Authentication method to No Authentication.

Fill in the following fields:

  • Email address or Username: enter an account authorized to log in to the configured mail server.
  • Password: password corresponding to the Email address or Username entered.

FROM field

This setting determines the address used in the FROM field of emails sent by the platform, except for Phishing and Ransomware simulations.

Available options:

  1. Default: uses the address loaded in the Email account authentication data.
  2. Custom: select Use a different email address and enter the desired address.

Emails per minute

Determines the maximum number of emails sent per minute with the configured account.

  • Must be an integer between 1 and 500.
  • Leave the field empty to apply no limit.

The limit applies per campaign. If you run two or more campaigns in parallel, the emails per minute are multiplied by the number of active campaigns.

Check connectivity

Before saving the configuration, you must validate the connection with the server.

  1. Click Check connectivity.
  2. If the connection is successful, you will receive a test email sent through your own server.
  3. If it fails, modify the data entered or cancel the action.

The system does not allow saving a configuration that has not passed the connectivity check.

Configuration example

Below is a typical configuration example for an SMTP server with TLS:

  • Email server: mail.organization-domain.com
  • Port: 587
  • Encryption protocol: TLS
  • Email address: name@organization-domain.com
  • Password: *************
  • Maximum number of emails that can be sent per minute: 30

Back to index

2. Through OAuth 2.0 for Microsoft Exchange Online

Configuration for sending notification emails using OAuth 2.0 authentication through Microsoft Entra ID. Avoids the use of passwords, reduces authentication failures, and is compatible with Microsoft Exchange Online environments that restrict basic SMTP access.

Access path

  1. Log in to SMARTFENSE with an administrator account.
  2. Go to Settings > Email server.
  3. Select the Own email server option.
  4. Choose Through OAuth 2.0 for Microsoft Exchange Online as the type of authentication.

Prerequisites

  • An account with administrator permissions in Microsoft Entra ID.
  • An application registered in Microsoft Entra ID with the Mail.Send API permission of type application, with consent granted.
  • The Configuration instructions available for download in the platform, to share with Microsoft Entra ID administrators.

Email server data

There are two scenarios for filling in the server data:

Scenario 1: reuse the user import configuration

If you have already configured user import and synchronization from Microsoft Entra ID, you can reuse that data:

  1. Click Use configured data in user import.
  2. The platform will auto-fill the Domain, Application ID, and Application secret key fields.
  3. Fill in the Sender user UPN field. This is the UPN of the user that will be used as the sender of emails sent from SMARTFENSE.

When using this option, you must add the Mail.Send API permission of type application to the registered application in Microsoft Entra ID and grant consent. Details are in steps 14 to 19 of the Configuration instructions.

Scenario 2: manual configuration

If you do not have an existing user import configuration, follow the Configuration instructions to register the application in Microsoft Entra ID and obtain the secret key. The instructions can be downloaded from the platform to share with Microsoft Entra ID administrators.

Required fields

Fill in the following fields with the data obtained from the Configuration instructions:

  • Domain: custom domain name of Microsoft Entra ID, as described in points 3 and 4 of the instructions.
  • Application ID: identifier of the application created in Microsoft Entra ID, as described in points 6 and 7 of the instructions.
  • Application secret key: string generated in point 25 of the instructions.
  • Sender user UPN: UPN of the user that will be used as the sender of emails sent from SMARTFENSE.

Emails per minute

Determines the maximum number of emails sent per minute with the configured account.

  • Must be an integer between 1 and 500.
  • Leave the field empty to apply no limit.

The limit applies per campaign. If you run two or more campaigns in parallel, the emails per minute are multiplied by the number of active campaigns.

Check connectivity

Before saving the configuration, you must validate the connection with the server.

  1. Click Check connectivity.
  2. If the connection is successful, you will receive a test email sent through your own server.
  3. If it fails, modify the data entered or cancel the action.

The system does not allow saving a configuration that has not passed the connectivity check.

Back to index

3. Via OAuth 2.0 for Google

Configuration for sending notification emails using OAuth 2.0 authentication with Google Workspace.

How email fields are assigned

Once OAuth 2.0 for Google is configured, the email fields are taken from the following sources:

  • Email of the Google Workspace account with which the project will be set up: uses the email of the Google Workspace account with which the project is configured.
  • Reply to: uses the Reply to field from the mail server configuration in SMARTFENSE.
  • JSON Private Key: text in JSON format from the downloaded OAuth 2.0 Clients ID.

You can set a From Name: uses the organization name field loaded in Settings > Organization > Organization data, field: Name to display as the sender of emails.

Prerequisites

  • Google Workspace account with permissions to create projects in Google Cloud. This is the same account you must enter in the Email of the Google Workspace account with which the project will be set up field, and it will be used to send emails.
  • Administrator access to SMARTFENSE.
  • Access to Settings > Organization > Organization data to verify the organization name.

Step 1 — Prepare the configuration in SMARTFENSE

  1. Log in to SMARTFENSE with an administrator account.
  2. Go to Settings > Email server and select the Own email server option.
  3. Choose the Via OAuth 2.0 for Google type of authentication.
  4. Leave the screen open. The fields to fill in are Email of the Google Workspace account with which the project will be set up, Reply to, and JSON Private Key.
  5. Use the Configuration instructions available in the platform to follow Steps 2, 3, and 4 below. The instructions can be downloaded to share with the Google Cloud administrator.

Step 2 — Create the project in Google Cloud

  1. Go to the Google Cloud console at https://console.cloud.google.com/welcome/new to create a new project.
  2. Assign a descriptive name and ID to make future management easier.
  3. Click the Create button.

Step 3 — Enable the Gmail API

  1. Select the project you just created.
  2. From the sidebar, go to APIs & Services > Library.
  3. Search for Gmail API.
  4. Click Enable to enable the Gmail API in the project.

Step 4 — Create OAuth 2.0 credentials

  1. Go to APIs & Services > Credentials and click the Create credentials button.
  2. Under Credential type, select User data and click Next.
  3. Configure the Consent screen by filling in the required application details: App name, User support email, and Email addresses, then click Save and continue.
  4. Configure credential scopes: click Add or remove scopes.
  5. Add the following scope: https://mail.google.com/ and click Update.
  6. Click Save and Continue.
  7. For the OAuth Client ID application type, select Web application.
  8. Under Authorized redirect URI, click ADD URI and add the following callback URL:
 
   https://instance.takesecurity.com/custom-email-server-google-oauth2callback/

Replace instance with the name corresponding to your SMARTFENSE instance URL. 9. Click Create. 10. Download the credentials in JSON format by clicking the Download button.

The callback URI must be entered exactly in that format. Any variation will cause the OAuth handshake to fail when saving the configuration in SMARTFENSE.

Step 5 — Complete the configuration in SMARTFENSE

  1. Return to the Settings > Email server > Own email server > Via OAuth 2.0 for Google screen in SMARTFENSE.
  2. Fill in the fields:
    • Email of the Google Workspace account with which the project will be set up: enter the account that will be used for sending.
    • Reply to: enter the address to be used in the reply-to field of emails.
    • JSON Private Key: paste the complete contents of the JSON file downloaded in the previous step.
  3. If applicable, fill in the Maximum number of emails that can be sent per minute field with an integer between 1 and 500. Leave it empty if you do not want to apply a limit.
  4. Click Save and check.

The emails per minute limit applies per campaign. If you run two or more campaigns in parallel, the amount per minute is multiplied.

Step 6 — Authorize the application

  1. When you click Save and check, a Google window will open.
  2. Select the Google Workspace account with which the project was created.
  3. Click Allow to authorize the application.
  4. Verify that SMARTFENSE displays a success message.

The account that authorizes the OAuth consent must be the same one used to create the project in Google Cloud. Otherwise, the validation will fail.

Back to index

4. Google as SMTP mail server

Configuration of a Google Workspace or Gmail account as an SMTP server for sending notifications from SMARTFENSE. Applies when the organization wants to use its own Google domain.

Preliminary considerations

Google imposes a limit of 10,000 emails per day for SMTP sending. Once exceeded, the server stops accepting new messages for 24 hours.

Configuration parameters

  1. Log in to SMARTFENSE with an account with administrator permissions.
  2. Go to Settings > Email server.
  3. Select the Own email server option.
  4. Under Type of authentication, choose Through password.
  5. Fill in the fields with the following values:
    • Email server: smtp.gmail.com
    • Port: 587
    • Encryption protocol: TLS
    • Email address: user@organization-domain.com
    • Password: password for the indicated account
    • Maximum number of emails that can be sent per minute: 25

Accounts with two-factor authentication enabled

If the Google account has two-factor authentication enabled, the regular password cannot be used. You must generate an app password:

  1. Sign in to your Google account.
  2. Go to the App passwords page.
  3. Under Select app, choose Mail.
  4. Under Select device, choose Other (custom name).
  5. Type SMARTFENSE.
  6. Click Generate. Google will confirm the password creation.
  7. Copy the 16-character key generated by Google.
  8. Enter that key in the Password field of the mail server configuration in SMARTFENSE, instead of the regular password.
  9. Click Check connectivity in the platform to validate sending.

Accounts without two-factor authentication

Google blocks sign-in by default from apps it considers less secure. To allow sending:

  1. Enable less secure apps in the Google account settings.
  2. Return to SMARTFENSE and click Check connectivity.

If the account belongs to a managed domain (Google Workspace in an organization), the domain administrator may need to enable access for less secure apps. In that case, coordinate with the domain administrator before proceeding.

FROM field rewriting

Google automatically rewrites the FROM field of emails sent through its SMTP server with the account's default address. This may change the visible address and the Reply-to field in some email clients.

Workaround: in the Accounts tab of the Google account, set the default account you want to use as the sender. Google will rewrite the FROM field with that account.

Check connectivity

Before saving the configuration, click Check connectivity. If the connection is successful, you will receive a test email and can save. If it fails, the platform does not allow saving the configuration.

Common errors

535, b'5.7.8 Username and Password not accepted. gsmtp

The credentials provided are not valid for the Google account. Verify the email address and password entered, and review the less secure apps setting or the app password configuration, depending on the account type.

Back to index

5. Office 365 as SMTP mail server

Configuration of an Office 365 account as an SMTP server for sending notifications from SMARTFENSE. Applies when the organization wants to use its own Microsoft domain.

Preliminary considerations

Microsoft is discontinuing basic authentication (username and password) for access to its mail servers. When possible, use the Through OAuth 2.0 for Microsoft Exchange Online method instead of SMTP with password.

If the connectivity error returns 535, b'5.7.139 Authentication Unsuccessful', the cause is that Microsoft no longer accepts basic authentication for that account. The solution is to configure the mail server with Through OAuth 2.0 for Microsoft Exchange Online.

Configuration parameters

  1. Log in to SMARTFENSE with an account with administrator permissions.
  2. Go to Settings > Email server.
  3. Select the Own email server option.
  4. Under Type of authentication, choose Through password.
  5. Fill in the fields with the following values:
    • Email server: smtp.office365.com
    • Port: 587
    • Encryption protocol: TLS
    • Email address: user@organization-domain.com
    • Password: password for the indicated account
    • Maximum number of emails that can be sent per minute: 25

Maximum number of emails per minute

The recommended value for Office 365 accounts is 25 emails per minute. If unexpected blocks or delays are observed during sending, reduce the value to 15 or another lower value until sending stabilizes.

The limit applies per campaign. If you run two or more campaigns in parallel, the total volume per minute is multiplied by the number of active campaigns.

Check connectivity

Before saving the configuration, click Check connectivity. If the connection is successful, you will receive a test email and can save. If it fails, the platform does not allow saving the configuration.

Common errors

535, b'5.7.3 Authentication Unsuccessful

The credentials provided are not valid for the Office 365 account. Verify the email address and password.

535, b'5.7.139 Authentication Unsuccessful

Microsoft rejects basic authentication for the account. Switch the configuration to the Through OAuth 2.0 for Microsoft Exchange Online method.

451 4.7.500 Server busy

Exchange Online is temporarily rejecting emails due to protection policies (greylisting, new sending patterns, or source reputation). This is a behavior of the client's Microsoft environment, not a SMARTFENSE issue. To resolve it:

  1. Coordinate with the organization's IT team.
  2. Configure an Exchange Online connector to accept SMARTFENSE sends.
  3. Review filtering policies and transport rules that may be blocking sends.
  4. Implement the recommended whitelist for Microsoft, including the fixed SMARTFENSE IPs: 54.194.86.204 and 54.194.98.136.

Back to index

6. On-premise Exchange Server via SMTP

Configuration of an on-premise Microsoft Exchange server as an SMTP server for sending notifications from SMARTFENSE. Applies when the organization operates its own Exchange infrastructure.

Prerequisites

  • Access to the Exchange Admin Center (EAC) or Exchange Management Shell (EMS).
  • Permissions to create and modify receive connectors in Exchange and firewall rules in the organization's network.
  • IP address or domain name of the Exchange server and the SMTP port to use.
  • The fixed SMARTFENSE IPs from which sends will originate: 54.194.86.204 and 54.194.98.136.

Step 1 — Create the Receive Connector

  1. Open the Exchange Admin Center (EAC) or Exchange Management Shell (EMS).
  2. Go to Mail Flow > Receive Connectors.
  3. Create a new connector with the following values:
    • Name: for example, "SMTP Connector for SMARTFENSE".
    • Role: Frontend Transport or Hub Transport depending on your configuration.
    • Type: Custom, to enable advanced permissions.
    • Local IP address (Bindings): the IP of the Exchange server that will receive connections from SMARTFENSE.
    • Remote IP ranges: 54.194.86.204 and 54.194.98.136.

Step 2 — Configure connector permissions

  1. Select the connector you created and open Properties.
  2. Under Security, enable the authentication methods your environment requires (for example, basic authentication or TLS).

Step 3 — Configure firewall rules

Allow SMTP traffic (port 25 or whichever your organization has defined) from 54.194.86.204 and 54.194.98.136 to the Exchange server.

Step 4 — Configure SMARTFENSE

  1. Log in to SMARTFENSE with an administrator account.
  2. Go to Settings > Email server and select the Own email server option.
  3. Under Type of authentication, choose Through password.
  4. Fill in the fields with the Exchange server data:
    • Email server: IP address or domain name of the Exchange server.
    • Port: 25 or 587 depending on your server configuration.
    • Encryption protocol: TLS if the receive connector requires it.
    • Email address or Username and Password: account authorized to authenticate sends, if the connector requires it.
    • Maximum number of emails that can be sent per minute: integer between 1 and 500, or empty to apply no limit.

If your on-premise Exchange server has two-factor authentication, use an app password and enter it in the Password field instead of the regular password.

Step 5 — Check connectivity

  1. Click Check connectivity.
  2. If the connection is successful, you will receive a test email sent from your Exchange server.
  3. If it fails, review the data entered, the receive connector configuration, and the firewall rules before saving.

The system does not allow saving a configuration that has not passed the connectivity check.

Diagnosis with an external client

If issues arise during the connectivity check, first validate sending from an external SMTP client such as Mozilla Thunderbird, outside the corporate network and without VPN. See the following image as a configuration reference.

Back to index

💡 Best practices

  • Always validate the configuration with Check connectivity before saving, for any of the six methods.
  • Use TLS or SSL as the encryption protocol when applicable, as many emails sent by SMARTFENSE contain private user data.
  • Prefer OAuth 2.0 over basic authentication in Google and Microsoft environments, as both providers are progressively deprecating password-based authentication.
  • Use an account or UPN dedicated exclusively to SMARTFENSE to make tracking and configuration control easier.
  • Adjust the Maximum number of emails that can be sent per minute according to the server's actual capacity, taking into account the cumulative impact of parallel campaigns.
  • Monitor the expiration of passwords, app passwords, and application secret keys, and plan their rotation before they expire to avoid disruptions in sending.
  • Coordinate with the client's IT team any changes to connectors, firewall rules, transport policies, or authentication methods.
  • Run a test campaign with a small number of users before launching mass sends from a new account or configuration, to validate the reputation and policies of the mail server.

Related to

Related articles