Crash Course

Analyze Crashes to Find Security Vulnerabilities in Your Apps

Adel Abouchaev and Damian Hasse and Scott Lambert and Greg Wroblewski

Contents

Access Violations
Access Violation When Writing Data
Non-Exploitable Exceptions
/GS Exceptions
NX Exceptions
Putting It into Practice

How can you make sure a crash in your program is not exploitable? The short answer is simple: assume every crash is exploitable and just fix it! At the very least, it is a matter of quality and it is often cheaper—and more practical—to fix the issue before the product has been shipped to customers. It can be rather expensive to perform the analysis necessary to determine exploitability.

Analyzing program failures related to memory corruption in order to understand the security ramifications can be a complex and error-prone task. Several factors must be considered, including the location of the buffer in memory, the possible targets for overwriting, the size of the overwrite, restrictions on the data that can be used during the overwrite, the state of the runtime execution environment, and the ability to bypass any mitigation mechanisms in place. In a nutshell, you must understand the root cause of the failure in order to answer these questions thoroughly.

Keep in mind that not every failure will manifest itself in an observable manner. One example of this is the GDI remote code execution issue discussed in Microsoft Security Bulletin MS07-017. The software responsible for invoking the vulnerable parsing code made use of an exception handler to recover from pretty much any exception that could be generated and continue operating as if nothing had occurred. (Read more about it at blogs.msdn.com/sdl/archive/2007/04/26/lessons-learned-from-the-animated-cursor-security-bug.aspx.) Another example that's less obvious can be found in certain types of stack and heap memory corruptions, where it's possible that failures have occurred but the current state of the program and its execution environment don't show any obvious signs.

This article offers guidance on how to analyze program crashes with regard to possible security implications—such as the case of memory corruption enabling arbitrary code execution or at the very least denial of service. We will enumerate the common hardware and software exceptions you might encounter when looking at these types of issues. We'll also offer some general guidelines you can use during such an investigation. Figure 1, for example, gives you a graphical path of the investigation process to assist you in deciding whether a particular crash is exploitable. It is important to remember that these are just guidelines and that only a full root-cause analysis can ensure that you have correctly diagnosed the crash as not being exploitable. New techniques or variations of existing attack techniques are being discovered all the time.

Figure 1** Access Violation Analysis Path **(Click the image for a larger view)

The most common cause of crashes is hardware or software exceptions. A typical modern processor can generate many different types of hardware exceptions, but in the Windows® environment only some of those create problems related to software security. The most common hardware exception is an access violation. We will first cover how to analyze hardware exceptions, followed by software ones.

Access Violations

Access violation exceptions (0xc0000005= STATUS_ACCESS_VIOLATION) are generated by modern processors when a memory access caused by an instruction or program execution does not satisfy certain conditions defined by the processor architecture or memory management unit structures.

While a pure crash can only result in a denial of service condition, it is not safe to assume that a crash can't be used to achieve more dangerous effects, including code execution. In analyzing crashes, you should assume that the entire body of memory, with some minor exceptions, is under control of a potential attacker; thus an access violation can in most cases lead to data controlled by the attacker. This statement applies to an exception that occurred while an instruction was either reading or writing data.

If the access violation can lead to your data being controlled by the attacker, then each access violation caused by a read from memory turns into loading an attacker's controlled data. The security effects of such an operation are not always easy to determine. You could perform a full data flow analysis on either the binary or the source code to find the scope of source address control and the consequences of feeding the program with random data at certain points of execution. This is a time-consuming and challenging task. In response, we have developed simple heuristics to quickly analyze read-access violation crashes for code-execution potential.

As the following example shows, an invalid memory pointer in the register eax caused a crash. In this case, control of the memory contents gives the attacker full control over program flow:

Application!Function+0x133:
3036a384 eb32            call     [eax]           ds:0023:6c7d890d=??
0:000> ub
mov    eax, [ebx]  ->  eax = invalid memory pointer
...                         (instructions not affecting register eax)
call        [eax]  ->  crash

Situations in which the address being read can't be sufficiently controlled by the attacker can be treated as denial of service conditions. For example, in the typical Windows user mode environment, a crash that occurs at a NULL pointer that was initialized and can't be affected by an attacker can't lead to code execution by itself.

In the following example, you can see that a crash was caused by referencing address zero through value in register eax:

Application!Function+0x133:
3036a384 8b14            mov     ecx, [eax]           ds:0023:00000000=??
0:000> ub
xor    eax, eax
...                         (instructions not affecting register eax)
cmp    ebx, 2
jne        label123
mov    ecx, [eax]  ->  crash, eax = 0 (NULL)

By disassembling (with the ub command in the Visual Studio® command-line debugger), we can track the data flow in this register until we confirm that the value in the register can't be influenced by malicious input. Indeed, in this example, the register was zeroed by XORing with itself and it wasn't used until the crashing instruction was reached.

Sometimes the exploitability of a defect is not immediately visible from the crashing instruction. For example, after disassembling the following instructions, you can see that it is a consequence of the control flow when the failing instruction (described in the previous paragraph) is followed by a critical one:

(1258.1638): Access violation - code c0000005 (second chance)

Application!Function+0x123:
3036a384 8b12            mov    eax, [ebx]           ds:0023:6c7d890d=??
0:000> u
mov    eax, [ebx]  -> crash, ebx points to invalid memory
...             (instructions not affecting regist an example er eax)
call        [eax]  ->  possibility of code execution

This example is similar to the first one, yet this time the crash occurred on the instruction loading data into the register eax. Although this operation by itself does not indicate security problems, the disassembly clearly shows that control over the value of register ebx implies control over the value of register eax, which leads to possible code execution.

The remaining cases can be analyzed by simulating malicious data injection during program run time, either manually under a debugger or automatically through a debugging tool. After reaching the instruction that causes the access violation, we can either change the source address to point to a valid memory address and execute the instruction again or we can put random data into the destination register and skip the faulting instruction, continuing execution. We will repeat this process for each access violation encountered until we reach an exploitable condition.

Let's analyze two examples. First, let's look at tracking data flow when we change the source address:

Application!Function+0xa70:
3036a37e 8b4708          mov     eax,dword ptr [edi+8] ds:0023:040fd004=????????

The crash here is caused by an incorrect value in register edi. We will change it to point to a valid memory region. There are many possible choices, but in practice we often use the current value of register eip. This ensures that a relatively large chunk of memory around the new value of edi is valid and adds the possibility of catching any subsequent writes to the memory block containing code, as it is always marked as read-only:

0:000> r edi=eip
0:000> g

After setting register edi to the current value of register eip, we continue execution, hitting another exception on register esi where we repeat the process:

(1258.1638): Access violation - code c0000005 (second chance)

Application!Function+0xa76:
3036a384 f60601          test    byte ptr [esi],1           ds:0023:6c7d890d=??
0:000> r esi=eip
0:000> g

Continuing execution, we hit a write access violation exception on an instruction trying to write data to the code segment. This means an attacker can also write data to any memory address:

 (1258.1638): Access violation - code c0000005 (second chance)

Application!Function+0xbef:
3036a4fd 894710          mov     dword ptr [edi+10h],eax ds:0023:3036a38e=0c46f60d

As you can see, the initial read access violation turned out to be a real security issue with possibilities for code execution.

Now let's look at tracking data flow by setting destination data and skipping instructions. Analyzing the same crash as before, we will keep changing the destination data. After the initial exception is hit, we set the eax register to any easy-to-track value and increase register eip by the size of the current instruction to skip over it:

Application!Function+0xa70:
3036a37e 8b4708          mov     eax,dword ptr [edi+8] ds:0023:03f93004=????????
0:000> r eax=deadbeef
0:000> r eip=eip+3
0:000> g

Continuing execution, we hit the next exception, which does not have a destination, so we can just skip the instruction:

(1258.1638): Access violation - code c0000005 (second chance)

Application!Function+0xa76:
3036a384 f60601          test    byte ptr [esi],1           ds:0023:deadbefb=??
0:000> r eip=eip+3
0:000> g

Executing further, we reach the same write access violation exception, implying the possibility of exploitable issue:

(1258.1638): Access violation - code c0000005 (second chance)

Application!Function+0xbef:
3036a4fd 894710          mov     dword ptr [edi+10h],eax ds:0023:03f9300c=0c46f60d

While both methods of simulating malicious data injection often yield the same results, from practice we can say that in many cases they cover different code paths, with only one method showing exploitability of an issue. All of the methods discussed so far allow finding potentially exploitable read access violation crashes very quickly, but we must remember that they do not provide a final validation that a certain crash is not an exploitable issue.

Access Violation When Writing Data

Crash Analysis Resources

• MSDN Security Developer Center
• Latest Microsoft Security Bulletins
• Visual Studio Debugger Security
• Debugging Tools for Windows

An access violation during data writing indicates possible memory corruption, which almost always leads to an exploitable condition with potential for code execution. Very often such writes are indicators of buffer overflow conditions present in the crashing program. In practice, a few write access violation crashes can be shown to be not exploitable. Nonetheless, these situations require you to perform a full data flow analysis to understand the root cause of the issue. You need to ensure that the corruption doesn't lead to an attacker being able to overwrite data to arbitrarily influence the flow of execution.

Most access violations with data writes can lead to exploitable cases that will result in malicious code execution. Typically code execution is achieved by overwriting an (arbitrary) piece of memory, either on the stack or on the heap.

In the following example, the crash occurred on a memory copying instruction when the destination register reached the upper boundary of the stack and hit an unallocated area of memory. The size of the copying operation came from the variable at address [ebp-8] and the crash indicates that it is under the attacker's control:

Application!Function+0x143:
3036a384 f3a4            rep     movsb           es:0023:00140000=??   ds:0023:0125432c=41414141
0:000> ub
mov    esi, [ebp-4]
mov    edi, [ebp+4]
mov    ecx, [ebp-8]
rep    movsb  ->  write access violation if value in ecx is big enough

We can say with confidence that a write access violation can't lead to code execution only if the destination addresses controlled by the attacker point to invalid memory or to data that has no influence on program execution. In practice, only a small set of write access violations satisfy this condition. In the Windows user mode environment, a write to a NULL pointer or system address space (usually addresses above 2GB) could be an example of a non-exploitable issue (assuming that the page with address 0x00000000 can't be allocated). Also, in a server scenario, the instruction would lead to a denial of service, which should be considered a security vulnerability if it is triggered by a non-administrator user.

This example is similar to the earlier case discussed regarding read access violations. Quickly disassembling shows that the register eax has been initialized to value zero and has not been changed until the crashing instruction was reached:

Application!Function+0x133:
3036a384 8d14            mov     [eax], ecx           ds:0023:00000000=??
0:000> ub
xor    eax, eax
...                       (instructions not affecting register eax)
cmp    ebx, 2
jne    label123
mov    [eax], ecx  ->  crash, eax = 0 (NULL)

Non-Exploitable Exceptions

In user mode—not in kernel mode—this class of exceptions is unlikely to be used in a single-stage exploit. Usually these exceptions result in the denial of service. Only in some cases, when the exception handler is changed as a result of another vulnerability, can these exceptions lead to malicious code execution. Denial of service is a high-priority bug in the server platform and medium priority in workstations.

An example of these exceptions is static/global dereferencing (read and write). The process in this example does not have read access to page 0x310000 for the read access violation to occur, or write access to the same page to trigger a write access violation.

mov ebp, 310046h
mov eax, [ebp+4h]

inc eax
mov [ebp+8h], eax

The first two exceptions need to be carefully analyzed. If the static/global value points to the address in memory where malicious code can write without limits in length and can overwrite other structures to stage the complex exploit, this case has to be marked as exploitable. If it only allows you to store a single DWORD on the address that is not part of any control structures, it is most likely not exploitable. However, this statement must be verified by running the program and observing the values (also known as runtime analysis). If the value that is stored is not used later in the code, it cannot lead to another exploitable condition and can be ignored. If the value could be used as a memory address or in a memory copy operation, the risk of exploitation is higher.

Code execution on a static/global address in uncontrollable address space (page 0 and similar cases) is another instance where exploitability relies on whether the memory page at the address from the operand is controllable for write:

0040137F B8 DE C0 AD DE    mov     eax, 0DEADC0DEh
00401384 FF D0             call    eax

Analysis of code execution on a static/global address has further implications. If the address belongs to the pages that would never be associated with normal process execution (page 0 or pages with addresses above 0x80000000), then it is not exploitable.

The division-by-zero operation will also trigger an exception, but it is not directly exploitable at this point. This situation needs extra analysis to decide whether the result of this exception can set the CPU to execute code that can lead to a successful exploit:

004013D6 33 C9    xor    ecx, ecx
004013D8 8B C1    mov    eax, ecx
004013DA 40       inc    eax
004013DB F7 F1    div    ecx

Unhandled C++ exceptions can disrupt the execution of the process. During runtime, they will cause termination of the application. Under the debugger, it is possible to continue after the unhandled exception. C++ exceptions occur when the runtime library's exception raise functions are called:

00401902    mov     [ebp+var_4], 1
00401909    push    offset __TI1H
0040190E    lea     eax, [ebp+var_4]
00401911    push    eax
00401912    call    __CxxThrowException@8 ; _CxxThrowException(x,x)

This condition can only be exploitable if the exception handler mechanism is already overwritten. Otherwise unhandled C++ exceptions are not exploitable. The stack trace for this exception is shown in Figure 2.

Figure 2 Stack Trace for an Unhandled C++ Error

CommandLine: test.exe
Symbol search path is: srv*c:\Symbols*\\symbols\symbols
Executable search path is:
ModLoad: 00400000 00405000   test.exe
ModLoad: 7c900000 7c9b0000   ntdll.dll
ModLoad: 7c800000 7c8f5000   C:\WINDOWS\system32\kernel32.dll
ModLoad: 78130000 781cb000   C:\WINDOWS\WinSxS\x86_Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_8.0.50727.762_x-ww_6b128700\MSVCR
80.dll
ModLoad: 77c10000 77c68000   C:\WINDOWS\system32\msvcrt.dll
(1494.14c4): Break instruction exception - code 80000003 (first chance)
eax=00251eb4 ebx=7ffda000 ecx=00000004 edx=00000010 esi=00251f48 edi=00251eb4
eip=7c901230 esp=0012fb20 ebp=0012fc94 iopl=0    nv up ei pl nz na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000202
ntdll!DbgBreakPoint:
7c901230 cc              int     3
0:000> g
1
(1494.14c4): C++ EH exception - code e06d7363 (first chance)
(1494.14c4): C++ EH exception - code e06d7363 (!!! second chance !!!)
eax=0012fee0 ebx=00000000 ecx=00000000 edx=781c3c58 esi=0012ff68 edi=004033a4
eip=7c812a5b esp=0012fedc ebp=0012ff30 iopl=0    nv up ei pl nz na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000206
kernel32!RaiseException+0x53:
7c812a5b 5e              pop     esi
0:000> kb
ChildEBP RetAddr  Args to Child
0012ff30 78158e69 e06d7363 00000001 00000003 kernel32!RaiseException+0x53
0012ff68 00401917 0012ff78 004022b8 00000001 MSVCR80!_CxxThrowException+0x46
0012ff7c 004011b2 00000001 00353098 003530d0 test!wmain+0x27
0012ffc0 7c816fd7 00011970 7c9118f1 7ffda000 test!__tmainCRTStartup+0x10f
0012fff0 00000000 004012fb 00000000 78746341 kernel32!BaseProcessStart+0x23

Note that the stack overflow exception may hide other issues at runtime, divert code flow to a different path, disable compiler generated protection mechanisms such as /GS, or bring inconsistency into the application by freeing memory that is still in use. When stack space is almost consumed, it is possible that an application will fail inside of C++ methods with a stack overflow exception and important pieces of code will not be executed in order. Additional triaging will be required in such conditions.

/GS Exceptions

/GS (0xc0000409=STATUS_STACK_BUFFER_OVERRUN) exceptions are those Windows will throw whenever it detects that the security cookie protecting the return address has been tampered with. Since the goal of /GS is to turn buffer overruns that lead to code execution into denial of service attacks, whenever such a crash is detected you can be certain you have a security bug. (Unfortunately, due to buggy memory, overclocked motherboards, faulty hardware, and other issues, sometimes the code that validates the cookie gets tripped without having an actual buffer overrun.)

In Windows Vista®, the operating system will throw an int 3 exception when a STATUS_STACK_BUFFER_OVERRUN is detected (assuming a debugger is present). In earlier versions of Windows, a breakpoint should be placed in kernel32!UnhandledExceptionFilter to detect whether a security cookie got tampered with (otherwise the process will get terminated and the user will not get notified).

In Figure 3, the function foo will overrun a buffer by copying too much data to a stack buffer, which will cause the /GS cookie to get overwritten.

Figure 3 Overwriting the /GS Cookie

<the /GS cookie is being setup in the function prolog>
0:000:x86> u gs!foo
gs!foo:
010011b9 8bff            mov     edi,edi
010011bb 55              push    ebp
010011bc 8bec            mov     ebp,esp
010011be 83ec18          sub     esp,18h
<global cookie will be moved to eax register>
010011c1 a100200001      mov     eax,dword ptr [gs!__security_cookie (01002000)] 
010011c6 53              push    ebx
010011c7 56              push    esi
010011c8 57              push    edi
010011c9 8b7d08          mov     edi,dword ptr [ebp+8]
<cookie will be placed on the stack (ebp-4)>
010011cc 8945fc          mov     dword ptr [ebp-4],eax 
<content of the source (src) and destination buffers (dst) – before the overrun>
0:000:x86> dv /V
000bfefc @ebp+0x08             src = 0x009d16cb "123456789012345678901234567890"
000bfedc @ebp-0x18             dst = char [20] ""
<value of the security cookie on the stack; note that it is located right after the buffer, before saved ebp (0x000bff24) and the return address (0x0100124a)> 
0:000:x86> dd 000bfedc+0n20 l3
000bfef0  0000b96f 000bff24 0100124a
<value of global cookie>
0:000:x86> dd gs!__security_cookie l1 
01002000  0000b96f
... code runs ...
<after the overrun has happened, the cookie got overwritten (with 0x34333231) as well as the last two bytes of the return address (with (0x3039)>
0:000:x86> dd 000bfedc+0n20 l3
000bfef0  34333231 38373635 01003039
<in the function epilog before it returns the /GS cookie gets checked (i.e. the return address has not been used yet)> 
0:000:x86> u gs!foo+0x54
gs!foo+0x54:
0100120d 59              pop     ecx
<cookie is placed in the ecx register>
0100120e 8b4dfc          mov     ecx,dword ptr [ebp-4]
01001211 5f              pop     edi
01001212 5e              pop     esi
01001213 5b              pop     ebx
01001214 e882020000      call    gs!__security_check_cookie (01002000)
01001219 c9              leave
0100121a c20400          ret     4
<the below functions compares the global cookie with the one that was stored in the stack (currently in ecx register)>
0:000:x86> r
eax=0000000c ebx=7efde000 ecx=34333231 edx=00000000 esi=00000000 edi=00000000
eip=0100149b esp=000bfed8 ebp=000bfef4 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
gs!__security_check_cookie:
0100149b 3b0d00200001    cmp     ecx,dword ptr [gs!__security_cookie (01002000)] ds:002b:01002000=0000b96f
<if a debugger was attached to the process, then an (int 3) will be issued (in Windows Vista) before the process gets terminated>
0:000:x86>
STATUS_STACK_BUFFER_OVERRUN encountered
 (15d0.1708): Break instruction exception - code 80000003 (first chance)
ntdll!DbgBreakPoint:
773e0004 cc              int     3

NX Exceptions

NX (0xc0000005=STATUS_ACCESS_VIOLATION) exceptions are thrown by Windows whenever it detects code executing on a page that is not marked executable (in other words, the page does not have PAGE_EXECUTE, PAGE_EXECUTE_READ, or other relevant flags). NX is enforced in 64-bit versions of the operating system. Some applications like unpackers and digital rights management (DRM) rely on executing code on the heap, so not every NX exception should be considered a security vulnerability. It still makes sense to understand the root cause of the bug to ensure that it does not have security implications.

It's important to note that this type of exception uses the error code 0xc0000005, which is not specific to NX; any application that misbehaves (by, say, reading from unallocated memory) can throw this error. In order to understand whether the exception/error code is indeed NX related, we need to review the protection set on the page. If the page is not marked executable, we have an NX exception; otherwise we have some other type of problem. In Figure 4, for example, a first chance exception was hit, but the instruction seems to be valid and it references valid data.

Figure 4 Potential NX Exception

(1424.a78): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
00000000`003b0020 2801            sub     byte ptr [rcx],al ds:00000000`7702e6aa=c3
0:000> r
rax=0000000000000001 rbx=000000000021fd10 rcx=000000007702e6aa
rdx=0000000000000000 rsi=000000000000000a rdi=0000000000000000
rip=00000000003b0020 rsp=000000000021fca0 rbp=00000000ff130000
 r8=000000000021fc98  r9=00000000ff130000 r10=0000000000000000
r11=0000000000000244 r12=00000000ff131728 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei pl zr na po nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
00000000`003b0020 2801            sub     byte ptr [rcx],al ds:00000000`7702e6aa=c3
<when we execute the actual instruction again, we hit a second chance access violation>
0:000> t
(1424.a78): Access violation - code c0000005 (!!! second chance !!!)
00000000`003b0020 2801            sub     byte ptr [rcx],al ds:00000000`7702e6aa=c3
<a closer look at the protection of the page, shows that it does not have PAGE_EXECUTE set, this explains why the code cannot be executed>
0:000> !address 00000000`003b0020
 ProcessParametrs 00000000003b25c0 in range 00000000003b0000 00000000003e0000
 Environment 00000000003b1370 in range 00000000003b0000 00000000003e0000
    00000000003b0000 : 00000000003b0000 - 0000000000030000
                    Type     00020000 MEM_PRIVATE
                    Protect  00000004 PAGE_READWRITE
                    State    00001000 MEM_COMMIT
                    Usage    RegionUsageHeap
                    Handle   00000000003b0000

Putting It into Practice

As you can see, analyzing program failures for security implications is an involved and potentially error-prone task. We discussed the most common exceptions you are likely to encounter when analyzing crashes, including read/write access violations, division by zero, C++ exceptions, /GS exceptions, and NX-related issues. The key point to remember is that only a full root-cause analysis can ensure that you have correctly diagnosed whether a given crash is exploitable.

In an effort to provide you with some useful guidelines for analyzing your own apps, we have summarized the information into a quick reference you can keep by your workstation. Figure 5 captures the guidelines discussed in the article in table form. For example, if a Write Access Violation occurred as a result of the CPU trying to write data to a memory page without having write access to it, you need to fix the issue.

Figure 5 Triage Fix Criteria

Likewise, Figure 1 (introduced earlier in this article) is intended to provide another view of the process by using a graph structure to help you decide whether a particular crash is exploitable. You start with the top node and ask yourself which node to visit next. You continue this process until you arrive at either the Not Exploitable or Exploitable node.

We hope you find these guidelines useful when triaging program failures. To find more information, follow the links in the "Crash Analysis Resources" sidebar. And remember, if you diagnose a crash as exploitable in a Microsoft product, please send this information to secure@microsoft.com.

Adel Abouchaev (CCIE#12037, MCSE, CISSP) is a Security Software Engineer on the Secure Windows Initiative (SWI) team at Microsoft. He is responsible for fuzzing and runtime analysis tools and works to provide product teams at Microsoft with automated methods of security analysis and security evaluation.

Damian Hasse Lead Security Software Engineer at Microsoft, leads a team of security researchers that investigate vulnerabilities and security threats as part of the Microsoft Security Response Center (MSRC).

Scott Lambert is a Security Program Manager on the Secure Windows Initiative (SWI) team at Microsoft. He is responsible for the enhancement of internal security tools, including various fuzzing tools. Leveraging his industry experience, Lambert works to ensure that SWI tools identify the vast majority of vulnerability classes.

Greg Wroblewski is a Security Software Engineer at Microsoft working in a security response team.