MIIS / ILM is slow when running an Import

  • Article

 

Please excuse the dry style of some of this post. I was attempting to write a more official-like technical document and realized that I was doing a "brain-dump" that was more suited to a blog-type posting. It started out when I wanted to write up what is now in item #5, then we ran into some other customer issues with poor performance that I thought I'd throw in for good measure. Performance is a very broad topic, so please view the information here as a collection of pointers rather than an exhaustive list of all things Performance.

Troubleshooting Poor Performance on Import

Poor performance when running Imports can be caused by a number of factors, each of which needs to be investigated. At a high level, these are the factors to consider.

  1. Performance related to the connected data source
  2. Network performance to the MIIS / ILM SQL database (if it is remote)
  3. Operations
  4. General health of the MIIS / ILM SQL database
  5. Additional Considerations for an Active Directory MA

 

Note that these are general guidelines based on previous customer support issues and in some cases more targeted troubleshooting is needed. Items 2, 3 and 4 can impact all ILM runs, not just Import.

Troubleshooting

Here are some suggestions for troubleshooting the various layers:

1. Performance related to the connected data source

For example, if you are having problems with an AD MA - try this.

From the MIIS / ILM machine use one of the standard AD management tools (ADU&C, ADSIEdit, LDP) to connect to the data source and browse to the directory from which you are Importing. If this operation is slow then this needs to be addressed from the Directory Services / Networking perspective.

If you're connecting to a different type of data source, you'll need to find whatever tools are appropriate.

Alternately a network capture might also reveal network latency that could cause the performance degradation.

2. Network performance to the MIIS / ILM SQL database (if it is remote)

MIIS / ILM operations are "chatty" with the backend database. If the SQL database is not on the same server as MIIS / ILM then performance will always be slower.

You can test this by running SQL management tools on the MIIS / ILM server and querying the backend database to test how responsive it is.

IMPORTANT NOTE: You must take care not to change any of the data in the database.

As with #1, a network capture could reveal network latency.

Don't see this as a recommendation to run SQL locally to ILM, there is a tradeoff between performance and scalability, reliability and various other "bilities".

3. Operations. Running multiple MA's at the same time can cause performance degradation. You can look at the Run Histories to see if there was any overlap between previous runs. You can prevent this situation in a number of ways.

  • Place the calls to all runs in the same batch file to guarantee that they run sequentially.
  • Use code to make sure no other MA runs are in progress. Take a look at the MIIS_ManagementAgent.RunStatus Method of the MIIS_ManagementAgent Class in the ILM Developer Reference.

 

4. General health of the MIIS / ILM SQL database - Of course, this will apply to all ILM operations, not just Imports.

Issues that have been observed to adversely impact ILM performance in general are

  • Index fragmentation
  • Database size, usually caused by not clearing Run Histories periodically

For further information on maintaining a healthy database, please refer to the "Care and Feeding of MIIS SQL Server Database" on https://www.ilmbestpractices.com/

(Note that this is a 3rd party site and Microsoft is not responsible for its content)

5. Additional Considerations for an Active Directory MA

The MA configuration of excluded or included OU's as well as the object types in those OU's can have an impact.

Consider the following scenario: You have a parent OU named Departments, and two child OU's named HR and Accounting. You want to import users from Accounting but not from the HR OU. One way to configure this in the MA properties is to select the Departments OU, at the top of the treeview, then deselect HR at the lower level. In this example it is more efficient to deselect  the Departments OU then select the Accounting OU individually.

Another consideration is group membership. Let's modify the example above, and this time you have group objects in the Accounting OU that you want to Import that also contain members from the HR OU (not a great example, but it serves to make the point).

On Import, ILM looks at the member attribute of the group, it sees that it has n members and will create n placeholder objects in the MA's connector space. It cannot resolve the placeholder objects to user objects from the HR OU because they are in an OU that is out of scope or excluded. You should avoid importing groups whose members are in an OU that is not imported.

The question of whether or not to Import and synchronize groups leads to some discussions that I'm not going to get into here, just a couple of quick tips. Obviously if you're not interested in synchronizing groups then don't select "group" in the Object types MA property. If you are synchronizing groups then you'll probably notice that as your groups get larger the synchronization performance gets exponentially worse, usually once the group membership gets up into the thousands. Our recommendation is to limit the number of users in your groups; there are more sophisticated approaches that we might tackle on this blog some time in the future.

 If you're a savvy user reading this - and you've made it this far - you might wonder why I'm bringing up such obvious points, but remember, the Identity support team gets tickets from the entire spectrum.