Troubleshooting Pool Leaks Part 4 – Debugging Multiple Users for a Tag
In our previous articles we discussed various techniques for identifying a pool memory leak and narrowing the scope of the leak to an individual pool tag. Knowing the leaking pool tag is often sufficient to identify the cause of the problem and find a solution. However, there may be a scenario where multiple drivers use the same pool tag (such as DDK) or when one driver uses the same tag in multiple places. In this scenario you will need more information to identify the source of the leak. In our next several articles we will present techniques to get this information.
This article will present a basic technique where we modify each pool tag to identify what code in which driver is allocating the memory that gets leaked.
This technique requires a live debug of the problematic system. There are many resources with steps for how to configure a system for a live debug. The debugging tools have instructions in the debugger.chm help file, under Debugging Tools for Windows\Debuggers\Installation and Setup\Kernel-Mode Setup.
Using the same technique as in Part 3, identify where the tag in question is used.
0: kd> !for_each_module s -a @#Base @#End "Leak"
fffff880`0496e3aa 4c 65 61 6b 3b c1 0f 42-c1 41 8d 49 fd 8b d0 ff Leak;..B.A.I....
fffff880`0496e621 4c 65 61 6b 3b c1 0f 42-c1 33 c9 8b d0 ff 15 cc Leak;..B.3......
Next, edit each instance so that they are unique. The ASCII code for numeral 1 is 0x31, and the codes for each numeral increase sequentially. Using this information edit each tag to be Lea1, Lea2, etc.
0: kd> eb fffff880`0496e3aa+3 31
0: kd> eb fffff880`0496e621+3 32
Confirm your edits resulted in the expected tags using the dc command.
0: kd> dc fffff880`0496e3aa l1
fffff880`0496e3aa 3161654c Lea1
0: kd> dc fffff880`0496e621 l1
fffff880`0496e621 3261654c Lea2
Now wait for the leak to happen and repeat the steps from Part 3 to identify which of the tags is leaked. This tells you what code allocates the memory that gets leaked. Below we can see that Lea2 is the tag being leaked.
0: kd> !poolused /t5 2
Sorting by NonPaged Pool Consumed
Tag Allocs Used Allocs Used
Lea2 257 263168000 0 0 UNKNOWN pooltag 'Lea2', please update pooltag.txt
nVsC 664 1531552 0 0 UNKNOWN pooltag 'nVsC', please update pooltag.txt
netv 4369 1172224 1 144 UNKNOWN pooltag 'netv', please update pooltag.txt
Leak 1 1024000 0 0 UNKNOWN pooltag 'Leak', please update pooltag.txt
EtwB 94 945136 4 163840 Etw Buffer, Binary: nt!etw
TOTAL 41296 281814544 44077 68102368
Knowing what code allocates the leaked pool may be very valuable to a driver developer who needs to narrow the scope of the problem. Often this information is sufficient for a developer to code review the use of this memory and identify why it would be leaked.
There are times when more information is needed to determine the cause of the leak. A developer may need the call stacks of memory being allocated and freed. We will capture this information in Part 5.