How to detect the performance issue:
Check queue length for each repo from: https://capsinsight.azurewebsites.net/Charts/StorageQueue.aspx?env=prod&comp=OpenLocalization&height=580
Check queue message waiting time from: https://capsinsight.azurewebsites.net/Charts/StorageQueue.aspx?env=prod&comp=OpenLocalization&height=580
- If the queue length or queue message waiting time is much higher than normal, email will be sent to open localization team.
How to deal with the performance issue:
Login to the corresponding machine
- Check queue name(repository id) and find the corresponding repository defined in ambient configuration: http://ambientconfiguration.blob.core.windows.net/openlocalization/openlocalization_prod.json
Find corresponding instance id and login to this machine, user is .\openloc, password: allknow :)
Check the log of the machine, find out where it stuck:
- You can find detail log from D:\temp(latest 1 minute log) and D:\commited\traceex(latest 7 minutes log) folders
- Find more log from query portal: http://capsinsight.azurewebsites.net/Charts/AsyncUserQuery.aspx
- Restart this machine or restart the worker role host process if need.
Scale out to improve performance:
Currently Open Localization can Resize the instance size(small -> media -> large..) to improve performance.
Separate the repositories to more instances to improve performance(change the instance configuration in our ambient configuration)