Keyword filtering

 

Applies to: Forefront Security for Exchange Server

Keyword Filtering helps you identify unwanted e-mail messages by analyzing the contents of the message body as it is being transported by the Transport scan job. By creating keyword lists, you can filter messages based on a variety of words, phrases, and sentences

Creating new keyword lists

For maximum flexibility, you can create your own lists of keywords to scan for. You can thus maintain individual lists of filters for use by different scan jobs.

To create a new keyword list

  1. In the FILTERING section of the Shuttle Navigator, click the Filter Lists icon.

  2. In the List Types pane, select Keywords.

  3. In the List Names section, click the Add button.

  4. Type a name for the new list, and then press Enter. The empty list appears in the List Names section.

  5. With the new list name selected, click the Edit button. The Edit Filter List dialog box appears. Use it to add content to your filter list.

  6. In the Include In Filter section, click the Add button.

  7. Type a word or phrase to be included in the filter list. Press Enter when you are finished typing. You may have as many words or phrases as you want, but each must be entered separately.

    The Exclude From Filter section is used to enter keywords or phrases that should never be included on the Keyword list. This prevents those words and phrases from accidentally being added when importing a list from a text file. For more information on importing files, see Importing items into a filter list.

  8. When you are finished adding items, click OK. The list of words you just entered appears, alphabetically, in the pane next to List Names.

  9. Click Save.

Configuring keyword lists

After you have created a keyword list, you must configure it.

To configure a keyword list

  1. In the Shuttle Navigator, click FILTERING.

  2. Click the Keyword icon. The Keyword Filtering work pane appears.

  3. Select the Transport scan job. (Keyword filtering only works with the Transport Scan Job.)

  4. In the Keyword Fields section, select Message Body.

  5. Select one of the filter lists you have created.

  6. Using the Filter field, set the filter to Enabled.

  7. Set the Action. For more information, see Keyword filter actions.

  8. Click the General tab.

  9. Indicate if you would like to Send Notifications.

  10. Indicate if you would like to Quarantine identified files. Enabling quarantine causes deleted attachments and purged messages to be stored, permitting you to recover them. However, worm-purged messages are not recoverable.

  11. Indicate what combination of Inbound, Outbound, and Internal mail should be scanned.

  12. Click the Identify tab and indicate whether the filter should look in the subject line, the message header, or both.

  13. Indicate the Minimum Unique Keyword Hits. This setting enables you to specify how many unique keywords must be matched for the action to be taken. The default is one (1). For example, you have set the minimum unique keyword hits value to 3. The word "wonderful", which is in the list, appears three times in the message. However, no other word in the list appears at all. The keyword filter has not been matched, because only one term in the list was matched.

  14. Click Save.

Filters for racial discrimination, sexual discrimination, spam, and any other custom lists must be created individually. For profanity filters, see Example lists.

Keyword filter actions

You must indicate the action that Forefront Security for Exchange Server should take upon detecting a match to your filter criteria.

Note

You must set the action for each content filter you configure. The action setting is not global.

The action choices are:

Skip: detect only

Records the number of messages that meet the filter criteria, but enables messages to route normally. If, however, Delete Corrupted Compressed, Delete Corrupted Uuencode Files, or Delete Encrypted Compressed Files was selected in General Options, a match to any of those conditions will cause the item to be deleted.

Purge: eliminate message

Deletes the message from your mail system. When you select this option, a warning appears informing you that if there is a filter match, the message will be purged and unrecoverable, unless quarantined. Click Yes to continue.

Identify: tag message

The subject line or message header of the detected message can be tagged with a customizable word or phrase, so that it can be identified later for processing into folders by user inboxes or for other purposes identified by the Forefront Server Security Administrator. This tag can be modified by clicking the Tag Text button on the Scan Job Settings work pane and modifying the text. The same tag, however, is used for all filters associated with the particular scan job.

Note

Forefront Security for Exchange Server keyword filtering scans both plain text and HTML message body content. If Forefront Security for Exchange Server finds a match in both the HTML and the plain text, it reports two detections in the Incidents log and the Quarantine database.

Keyword list syntax rules

The following are the syntax rules for a keyword filter list. Be careful to use the appropriate syntax because FSE does not perform validation. If the filtering results are not what you are expecting, it is recommended that you double-check your syntax.

  • Each item (line of text) is considered a search query.

  • Queries use the OR operator. It is considered to be a positive detection if any entry is a match.

  • Queries are comprised of operands (keywords), which are text tokens or a string of text tokens, such as:

    • apple (means that the text contains “apple”)

    • apple juice (means that the text contains “apple juice”)

    • get rich quick (means that the text contains “get rich quick”)

  • Queries may also contain operators that precede or separate operands in an expression.

  • An expression may be comprised of a single operand, an operand preceded by the _NOT_ or _HAS[#]OF_ operators, or two operands joined by the _AND_, _ANDNOT_, or _WITHIN[#]OF_ operators.

    The following logical operators are supported in expressions. There must be a space between an operator and an operand (or another operator), represented in the examples by the • character:

    • _AND_ (logical AND). For example, apples•_AND_•oranges. A filter such as this would be matched if the text contains both “apples” and “oranges”.

    • _NOT_ (negation). For example, _NOT_•oranges. A filter such as this would be matched if the text does not contain “oranges”.

    • _ANDNOT_ (logical AND negation). For example, apples•_ANDNOT_•oranges. A filter such as this would be matched if the text contains “apples” but does not contain “oranges”. _ANDNOT_ is functionally equivalent to _AND_•_NOT_.

    • _HAS[#]OF_ (frequency). Specifies the minimum number of times that the text must appear in order for the query to be considered true. For example, _HAS[4]OF_•get rich quick. If the phrase "get rich quick" is found in the text four or more times, this query is true. This operator implicitly has a default value of 1 when it is not specified.

    • _WITHIN[#]OF_ (proximity). If the two terms are within a specified number of words before or after each other, there is a match. For example, free•_WITHIN[10]OF_•offer. If "free" appears within 10 words before or after "offer", this query is true.

    Multiple operators are permitted in a single query. The precedence of the operators is (from highest to lowest):

    • _WITHIN[#]OF_

    • _HAS[#]OF_

    • _NOT_, _AND_, and _ANDNOT_ (these are at the same precedence level because they are used in conjunction when part of an expression)

    This precedence cannot be overridden with parentheses. Other considerations are:

    • The logical operators must be entered in uppercase letters.

    • Phrases may be used as keywords. For example, apple juice or get rich quick. Quotation marks are not used.

    • Multiple blank spaces (blank characters, line feed characters, carriage return characters, horizontal tabs, and vertical tabs) are treated as one blank space for matching purposes. For example, A••••B is treated as A•B and matches the phrase A•B.

    • In HTML-encoded message texts, punctuation (any non-alphanumeric character) is treated as a word separator similar to blank spaces. Therefore, words surrounded by HTML tags can be properly identified by the filter. However, note that the filter '<html>' will match '<html>', but not 'html'.

Examples (the • character represents a space):

  • apples•_AND_•oranges•_AND_•lemons•_WITHIN[50]OF_•juice

    This expression means that “apples”, “oranges”, and “lemons” all appear at least once, and that “lemons” is within 50 words of “juice”.

  • confidential•_WITHIN[10]OF_•project•_AND_•banana•_WITHIN[25]OF_•shake

    This expression means that “confidential” is within 10 words of “project”, and that “banana” is within 25 words of “shake”.

  • _HAS[2]OF_•get rich•_WITHIN[20]OF_•quick

    This expression means that “get rich” appears at least 2 times within 20 words of “quick”.

Case-sensitive filtering

The General Option Case Sensitive Keyword Filtering setting causes Forefront Security for Exchange Server to use case-sensitive comparisons for all keyword filters. By default, comparisons are not case-sensitive. For more information, see "General Options" in Forefront Server Security Administrator.

Example lists

To aid you in filtering for profanity, example lists in various languages are included with the product. This is an optional component of FSE and must be installed separately.

If you want to install one or more of these lists, follow these steps.

To install the example lists

  1. Find the file called KeywordInstaller.msi in the installation folder and double-click it.

    Note

    The .msi file is not present on any computer which has had an Administrator-only installation or on one that does not contain a Forefront Security product.

  2. You must read and consent to the license agreement/disclaimer.

  3. You are presented with a list of available files. You may select any number of the various language files. The files you select are placed into a folder called Example Keywords in the database directory (which, by default is c:\Program Files(x86)\Microsoft Forefront Security\Exchange Server\Data).

  4. After the files have been extracted, you must import them into your filters. For more information on importing files, see Importing items into a filter list.

Note

It is your responsibility to visually inspect all of the selected files to determine if there are words that are completely harmless in your environment, especially if you are using multiple language files. You must review the imported list and decide if you are going to eliminate any word clashes. If a certain word is unacceptable in one language but harmless in another, you must determine what is more important to you: catching everything (the default, if you accept all the words in all the selected lists) at the risk of false positives or risk not detecting something by deleting words from the list (which avoids those false positives).

Allowed senders lists

Forefront Security for Exchange Server provides allowed senders list functionality so that administrators can maintain lists of safe e-mail addresses or domains that are not subjected to filtering by the Transport Scan Job. (The allowed sender lists have no effect on scanning for viruses.) Forefront Security for Exchange Server checks the sender address or domain against the allowed senders list. If the e-mail address or domain appears on the allowed senders list, Forefront Security for Exchange Server will bypass all filtering that has been enabled for the list.

To create an allowed senders list

  1. In the FILTERING section of the Shuttle Navigator, click the Filter Lists icon.

  2. In the List Types section, select Allowed Senders.

  3. In the List Names section, click the Add button.

  4. Type a name for the new list and then press ENTER. The empty list appears in the List Names section.

  5. With the new list name selected, click the Edit button. The Edit Filter List dialog box appears. Use it to enter e-mail addresses or e-mail domains to include in the allowed senders list.

  6. In the Include In Filter section, click the Add button.

  7. Type an e-mail address or domain to be included in the filter list. Press ENTER when you are finished typing. User addresses should be entered in the format: user@customer.com. E-mail domain names should be entered in the format: *domain. You may have as many allowed senders as you want, but each address or domain must be entered separately.

    The Exclude From Filter section is used to enter addresses or domains that should never be included on the allowed senders list. This prevents those addresses and domains from accidentally being added when importing a list from a text file. For more information on importing files, see Importing items into a filter list

  8. When you are finished adding items, click OK. The list of addresses and domains you just entered appears, alphabetically, in the pane next to List Names.

  9. Click Save.

Enabling allowed senders lists

After you have created an allowed senders list, you must enable it.

To enable an allowed senders list

  1. In the Shuttle Navigator, click FILTERING.

  2. Click the Allowed Senders icon. The Allowed Senders work pane appears.

  3. From the list in the upper pane, select the Transport Scan Job.

  4. In the Sender Lists pane, select the name of the allowed senders list.

  5. Set the List State to Enabled.

  6. In the Skip scanning for section, indicate if the allowed senders list should apply to Keyword Filtering, File Filtering, or both. You can click All Types to have all the choices selected. If none of the check boxes are selected, the filter is effectively disabled.

  7. Click Save.

Importing items into a filter list

Data for filter lists may be created offline in Notepad or a similar text editor and then imported into the appropriate filter list using the Forefront Server Security Administrator. Note that Forefront Security for Exchange Server can only import lists that are UTF-16 or ANSI files. Other Unicode types will not be properly imported.

To create and import entries into a filter list

  1. Create a list and save it as a text file. Place each filter on its own line in the file.

  2. In the FILTERING section of the Shuttle Navigator, click Filter Lists.

  3. Select the filter list into which you will be importing data.

  4. Click Edit. The Edit Filter List dialog box appears.

  5. Click the Import button. A File Explorer window opens. Use it to navigate to the text file you created in step 1.

  6. Select the file and click Open.

  7. The file is imported into the middle pane of the Import List editor to enable you to select the entries you would like to include in your filter list. Use the <=== button to move all the items into the Include In Filter section or use the <--- button to move single items. You can use the right-pointing arrows to move items into the Exclude From Import section.

  8. When you have moved all the desired items, click OK.

  9. Click Save.