/3GB /PAE and AWE on 32-bit Systems
I was reading a blog post from Chad Boyd - http://blogs.msdn.com/chadboyd/archive/2007/03/24/pae-and-3gb-and-awe-oh-my.aspx. He has very detailed and excellent information about /3GB PAE and AWE. It made my understanding more clear. Here is the summarized version:
For 32-bit OS
Memory Addressability Limitation:
By default, the 32bit OS can only 'see' and use up to 4gb of memory, as it uses a 32bit range to map physical memory space (with a valid range of 0x00000000 up to 0xFFFFFFFF, or 0 thru 4,294,967,295 in decimal format, which correlates to a maximum high range pointer of 4GB. Yes, this means that on a default 32bit system no matter how much physical memory you install on the system, the system would only ever use at most 4 GB of it (kind of a waste if you have a 32bit server with 64GB of memory)
Virtual Address Space (VAS) Limitation:
When a process is created on a 32-bit Windows System, OS allocates Virtual Address Space (VAS) to that process. All user-mode process (such as SQL Server) has its own VAS. Each process in 32-bit system has 4 GB of VAS. VAS is further divided into two equal sections: 1st section which is available to the process and 2nd section which is reserved for kernel/system. So, out of 4 GB VAS, 2 GB is available to the process and 2 GB is reserved for kernel/system. So, though a VAS is 4GB in size, a user-mode process by default only has access to up to 2GB of that space - yes, this means that by default, a 32bit user-mode process can only ever access 2GB of memory at the most using standard API's.
/3GB can address the VAS limitations:
With /3GB you can change the VAS allocation from 2 GB for process to 3 GB availability for the process and reserving only 1 GB for kernel/system. Limitation of /3GB is that it can only be used for the system having memory upto 16 GB. This is because the limited 1GB of kernel space - that is very limited space to be able to store internal memory mapping structures.
/PAE can address the memory addressability limitation of 32-bit system:
With /PAE you can change the addressability of 32-bit system from 2^32 to 2^36 (from 4 GB to 64 GB).
AWE can take the advantage of increased addressability by PAE. With AWE you can increase memory availability to SQL Server from 4 GB (for 32-bit) to up to 64 GB. So, although AWE allows you to access/use memory outside the processes' VAS, the process is still confined by the VAS in terms of being able to read/write data into memory mapped by AWE mechanisms. Only Data Cache can utilize AWE, no other memory consumer within SQL such as Hash cache, Sorting, plan cache, connection cache, etc. can use memory available through AWE.
If your system has < 4 GB - use only /3GB
If your system has > 4 GB and < 16 GB - use /3GB + /PAE + AWE
If your system has > 16 GB - use /PAE + AWE