Use Microsoft Sentinel watchlists

Note

Azure Sentinel is now called Microsoft Sentinel, and we’ll be updating these pages in the coming weeks. Learn more about recent Microsoft security enhancements.

Microsoft Sentinel watchlists enable the collection of data from external data sources for correlation with the events in your Microsoft Sentinel environment. Once created, you can use watchlists in your search, detection rules, threat hunting, and response playbooks. Watchlists are stored in your Microsoft Sentinel workspace as name-value pairs and are cached for optimal query performance and low latency.

Important

Noted features are currently in PREVIEW. The Azure Preview Supplemental Terms include additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

Common scenarios for using watchlists include:

  • Investigating threats and responding to incidents quickly with the rapid import of IP addresses, file hashes, and other data from CSV files. Once imported, you can use watchlist name-value pairs for joins and filters in alert rules, threat hunting, workbooks, notebooks, and general queries.

  • Importing business data as a watchlist. For example, import user lists with privileged system access, or terminated employees, and then use the watchlist to create allowlists and blocklists used to detect or prevent those users from logging in to the network.

  • Reducing alert fatigue. Create allowlists to suppress alerts from a group of users, such as users from authorized IP addresses that perform tasks that would normally trigger the alert, and prevent benign events from becoming alerts.

  • Enriching event data. Use watchlists to enrich your event data with name-value combinations derived from external data sources.

Note

  • The use of watchlists should be limited to reference data, as they are not designed for large data volumes.

  • The total number of active watchlist items across all watchlists in a single workspace is currently limited to 10 million. Deleted watchlist items do not count against this total. If you require the ability to reference large data volumes, consider ingesting them using custom logs instead.

  • Watchlists can only be referenced from within the same workspace. Cross-workspace and/or Lighthouse scenarios are currently not supported.

Create a new watchlist

  1. From the Azure portal, navigate to Microsoft Sentinel > Configuration > Watchlist and then select + Add new.

    new watchlist

  2. On the General page, provide the name, description, and alias for the watchlist, and then select Next: Source.

    watchlist general page

  3. On the Source page, select the dataset type (currently only CSV is available), enter the number of lines before the header row in your data file, and then choose a file to upload in one of two ways:

    1. Click the Browse for files link in the Upload file box and select your data file to upload.
    2. Drag and drop your data file onto the Upload file box.

    You will see a preview of the first 50 rows of results in the wizard screen.

  4. In the SearchKey field, enter the name of a column in your watchlist that you expect to use as a join with other data or a frequent object of searches. For example, if your server watchlist contains country names and their respective two-letter country codes, and you expect to use the country codes often for search or joins, use the Code column as the SearchKey.

  5. Select Next: Review and Create.

    watchlist source page

    Note

    File uploads are currently limited to files of up to 3.8 MB in size.

  6. Review the information, verify that it is correct, wait for the Validation passed message, and then select Create.

    watchlist review page

    A notification appears once the watchlist is created.

    watchlist successful creation notification

Create a new watchlist using a template (Public preview)

  1. From the Azure portal, navigate to Microsoft Sentinel > Configuration > Watchlist > Templates (Preview).

  2. Select a template from the list to view details on the right, and then select Create from template to create your watchlist.

    Create a watchlist from a built-in template.

  3. Continue in the Watchlist wizard:

    • When using a watchlist template, the watchlist's Name, Description, and Watchlist Alias values are all read-only.

    • Select Download Schema to download a CSV file that contains the relevant schema expected for the selected watchlist template.

    Each built-in watchlist template has it's own set of data listed in the CSV file attached to the template. For more information, see Built-in watchlist schemas

  4. Populate your local version of the CSV file, and then upload it back into the wizard.

  5. Continue as you would when creating a new watchlist from scratch, and then use your watchlist with queries and analytics rules.

Use watchlists in queries

Tip

For optimal query performance, use SearchKey (representing the field you defined in creating the watchlist) as the key for joins in your queries. See the example below.

  1. From the Azure portal, navigate to Microsoft Sentinel > Configuration > Watchlist, select the watchlist you want to use, and then select View in Log Analytics.

    use watchlists in queries

  2. The items in your watchlist are automatically extracted for your query, and will appear on the Results tab. The example below shows the results of the extraction of the Name and IP Address fields. The SearchKey is shown as its own column.

    Note

    The timestamp on your queries will be ignored in both the query UI and in scheduled alerts.

    queries with watchlist fields

  3. You can query data in any table against data from a watchlist by treating the watchlist as a table for joins and lookups. Use SearchKey as the key for your join.

    Heartbeat
    | lookup kind=leftouter _GetWatchlist('mywatchlist') 
     on $left.RemoteIPCountry == $right.SearchKey
    

    queries against watchlist as lookup

Use watchlists in analytics rules

Tip

For optimal query performance, use SearchKey (representing the field you defined in creating the watchlist) as the key for joins in your queries. See the example below.

To use watchlists in analytics rules, from the Azure portal, navigate to Microsoft Sentinel > Configuration > Analytics, and create a rule using the _GetWatchlist('<watchlist>') function in the query.

  1. In this example, create a watchlist called “ipwatchlist” with the following values:

    list of four items for watchlist

    create watchlist with four items

  2. Next, create the analytics rule. In this example, we only include events from IP addresses in the watchlist:

    //Watchlist as a variable
    let watchlist = (_GetWatchlist('ipwatchlist') | project IPAddress);
    Heartbeat
    | where ComputerIP in (watchlist)
    
    //Watchlist inline with the query
    //Use SearchKey for the best performance
    Heartbeat
    | where ComputerIP in ( 
        (_GetWatchlist('ipwatchlist')
        | project SearchKey)
    )
    

    use watchlists in analytics rules

View list of watchlists aliases

To get a list of watchlist aliases, from the Azure portal, navigate to Microsoft Sentinel > General > Logs, and run the following query: _GetWatchlistAlias. You can see the list of aliases in the Results tab.

list watchlists

Manage your watchlist in the Microsoft Sentinel portal

You can also view, edit, and create new watchlist items directly from the Watchlist blade in the Microsoft Sentinel portal.

  1. To edit your watchlist, navigate to Microsoft Sentinel > Configuration > Watchlist, select the watchlist you want to edit, and select Edit watchlist items on the details pane.

    Screen shot showing how to edit a watchlist

  2. To edit an existing watchlist item, mark the checkbox of that watchlist item, edit the item, and select Save. Select Yes at the confirmation prompt.

    Screen shot showing how to mark and edit a watchlist item.

    Screen shot confirm your changes.

  3. To add a new item to your watchlist, select Add new on the Edit watchlist items screen, fill in the fields in the Add watchlist item panel, and select Add at the bottom of that panel.

    Screen shot showing how to add a new item to your watchlist.

Next steps

In this document, you learned how to use watchlists in Microsoft Sentinel to enrich data and improve investigations. To learn more about Microsoft Sentinel, see the following articles: