Thoughts on improving performance of the Logging Application Block
EntLib5 team is working on improving performance of the logging block (story LAB02 in our product backlog). We’ve identified that message formatting in particular takes substantial amount of time based on our own performance tests and customer reports. So, we are considering alternatives. One of them is improving performance of the formatting operations and another one is doing formatting and logging asynchronously, thus offloading the formatting work to a separate thread and improving perceived performance. Note, this discussion is not about optimizing the message formatting mechanism, which we may address separately.
Our intention is to implement these improvements on the opt-in basis to preserve the default behavior of the Logging Block consistent with the previous versions of the EntLib.
Here are the possible limitations we’ve identified when doing formatting and logging asynchronously:
1. When using the Logging Block, it is possible to utilize non-EntLib-specific .NET trace listeners (such as console, debug, eventlog, etc.). These trace listeners use TraceEventCache object to obtain context information, including thread-specific information, which we cannot control. This means that performing the logging operation in a thread different than the originating one may result in the log entry that has inaccurate context information.
2. Besides, the TraceEventCache also supplies the call stack, which we can force to be pre-computed in the originating thread. However, this turns out to be very expensive and likely be a waste since not all trace listeners would require it. Options include:
a. Pre-compute always
b. User elects to pre-compute via configuration
c. Add logic to the block to determine whether pre-compute is necessary based on the configuration of the trace listeners
Note, this is not the call stack you would care about when logging exceptions through the LoggingHandler of the Exception Handling Application Block. In this scenario, the actual message (together with the call stack) gets formed before a call to the Logging Block is even made.
3. With this approach, there is no guarantee that all log entries are effectively logged. For example, if a log entry posted just before the application terminates, it’s possible that the logging thread may not complete its processing.
We’d like to understand which logging characteristics you care about. Among the following, please select top 3 that matter to you:
A. Maintain the chronological order of log entries
B. Avoid depletion of the thread pool
C. Minimize the lag time between post of the log entry and its persistence (this would reduce the risk of losing posted entries)
D. Start throttling to avoid unbound memory consumption
E. Ability to provide custom logging strategies (such as async logging) to be plugged-in
F. Ability to elect which categories/trace listeners use async or sync logging mechanism
Based on relative importance of these choices, we will select a design direction ranging from using the thread pool to implementing more sophisticated thread and data management.
With your votes (feel free to reply to this post or send Grigori an email), please describe the context driving your choices and any other comments or concerns we may have overlooked.
We would appreciate your rapid feedback – if you care for this story, please do not wait with your response.