question

jayrhoades-8813 avatar image
0 Votes"
jayrhoades-8813 asked GaryNebbett commented

createfilemapping returns 1450

using the msdn online example (c++) code
found here: https://docs.microsoft.com/en-us/windows/win32/memory/creating-a-file-mapping-using-large-pages

on a windows 10 version 1909 box with 48GB of memory, the createfilemapping() for a size of 0x200000 returns error 1450 as shown below

the privilege "SeLockMemoryPrivilege" is successfully enabled in the process token.
any suggestions/experience with such would be appreciated;
thanks you;


86325-image.png


windows-api
image.png (12.9 KiB)
· 11
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Is your program compiled as 64-bit?

0 Votes 0 ·

David,
thanks you so much; i check on the c++ solution which use the straight away example;
i believe is 32, then i test
if that is underlying problem, i am so embarrassed;

i know on the c# managed side is "Any CPU" ( i have set the managed side c# mapping for large pages aside for the time being)...

ok; that changes things!!!!
i have an arbitrary setting of size on 6 * the pGetLargePageMinimum whose value is 0x200000) (since when using large pages it say must be a multiple of such;
86346-capture.png

on the c++ tester is NOW setted to x64 - and that markedly changes things, now on the
86353-capture1.png


file handle returned by CreateFileMapping as shown is 0x0 - not good!


then when i step on the GetLastError() it show:

86336-capture2.png


0 Votes 0 ·
capture.png (5.3 KiB)
capture1.png (32.6 KiB)
capture2.png (12.7 KiB)

Is hFileHandle valid?
Have you established whether the code works for small sizes?

0 Votes 0 ·
Show more comments

After researching, this seems to be related to the registry settings, you can refer to this thread.



0 Votes 0 ·

SongZhu-MSFT;
Thanks you for your reply; the thread which you reference does not make solution;

The problems originate with the inclusion of manifest constants:
| SEC_COMMIT | SEC_LARGE_PAGES , and | FILE_MAP_LARGE_PAGES

on the CreateFileMapping() and MapViewOfFile() - which the discussion thread does not use, as well i am not using the Get/Set WorkingSetSize()

the persistent problem is any inclusion/use of above manifest constants, (which i may simply avoid to use)

when i exclude:
| SEC_COMMIT | SEC_LARGE_PAGES , and | FILE_MAP_LARGE_PAGES -and- my mapping size is > 4MB then handles are returned;
this is true for using paging file and my own file mapping object;

when i include:
| SEC_COMMIT | SEC_LARGE_PAGES , and | FILE_MAP_LARGE_PAGES -and- my mapping size is any size (eg: 0x200000) then null handles are returned;

0 Votes 0 ·

After researching, I found that this seems to be related to the Windows update, maybe you need to try to update the system to solve the problem.

0 Votes 0 ·
Show more comments
jayrhoades-8813 avatar image
0 Votes"
jayrhoades-8813 answered jayrhoades-8813 commented

here is attached txt (cpp) mainline that when put into a visual studio c++ console app will represent this ongoing discussion;
this in lieu of posting a zip of the console app that with intellisense browse db is just way too big....
if anyone has any insights into why using map_use_large_pages on specify size of 6 * GETLARGEPAGEMINIMUM results in 1450
on windows 10 version 1909 box with 48GB of memory, baffles me;
kind regards; @DavidLowndes-6766
jsr;

87453-filemaplargepages.txt



· 6
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

With MAP_TO_LOCAL_FILE 0 and MAP_USE_LARGE_PAGES 1 in order to exercise the problematic scenario, for me the code in your Privilege function reports a failure because AdjustTokenPrivileges results in GetLastError returning ERROR_NOT_ALL_ASSIGNED.

What does it do for you?

0 Votes 0 ·

@DavidLowndes-6766
thanks you for your reply; see here the privilege is "Local Security Policy" and for me return no error as shown:

87359-capture.png


and this is found on "Local Security Policy" as shown:
87522-capture.png


0 Votes 0 ·
capture.png (28.6 KiB)
capture.png (31.7 KiB)

Despite adding my account to that Policy, I still get ERROR_NOT_ALL_ASSIGNED returned.

0 Votes 0 ·
Show more comments
GaryNebbett avatar image
0 Votes"
GaryNebbett answered jayrhoades-8813 commented

Hello @jayrhoades-8813,

Can you try compiling and running this program and then reporting what it shows? Success would look something like this:

0x200000
0xD8, 0
0x1B000000, 0

I checked that large page(s) were actually used and it seems that they are:

0: kd> !vad 1B000000
VAD Level Start End Commit
ffffce8206e94b60 6 1b000 1b1ff 0 Mapped LargePag READWRITE Pagefile section, shared commit 0x200

The program can be compiled by just invoking the command "%SystemRoot%\Microsoft.NET\Framework64\v4.0.30319\csc" with the name of the file containing the saved program text. You can add the "-platform:x64" option to the compilation if you want but the default (anycpu) should be OK too if there is nothing unusual about your environment.

Gary

 using System;
 using System.Reflection;
 using System.Runtime.InteropServices;
    
 class A
 {
     static int Main(string[] args)
     {
  try
  {
      Privilege lockmem = new Privilege("SeLockMemoryPrivilege");
    
      lockmem.Enable();
    
      X();
    
      lockmem.Revert();
  }
  catch (Exception ex)
  {
      Console.Error.WriteLine(ex);
  }
    
  return 0;
     }
    
     static void X()
     {
  UIntPtr n = GetLargePageMinimum();
    
  Console.WriteLine("0x{0:X}", n.ToUInt64());
    
  IntPtr h = CreateFileMapping(INVALID_HANDLE_VALUE, null, PAGE_READWRITE | SEC_COMMIT | SEC_LARGE_PAGES, 0, n.ToUInt32(), "Gary");
    
  Console.WriteLine("0x{0:X}, {1}", h.ToInt64(), Marshal.GetLastWin32Error());
    
  IntPtr p = MapViewOfFile(h, FILE_MAP_ALL_ACCESS | FILE_MAP_LARGE_PAGES, 0, 0, n);
    
  Console.WriteLine("0x{0:X}, {1}", p.ToInt64(), Marshal.GetLastWin32Error());
    
  UnmapViewOfFile(p);
  CloseHandle(h);
     }
    
     static IntPtr INVALID_HANDLE_VALUE = new IntPtr(-1);
    
     const uint SEC_COMMIT = 0x08000000;
     const uint SEC_LARGE_PAGES = 0x80000000;
     const uint FILE_MAP_ALL_ACCESS = 0x000F001F;
     const uint FILE_MAP_LARGE_PAGES = 0x20000000;
     const uint PAGE_READWRITE = 0x04;
    
     [DllImport("kernel32.dll", SetLastError = true)]
     static extern bool CloseHandle(IntPtr handle);
    
     [DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
     static extern IntPtr CreateFileMapping(IntPtr file, object sa, uint protect, uint n2, uint n, string name);
    
     [DllImport("kernel32.dll", SetLastError = true)]
     static extern IntPtr MapViewOfFile(IntPtr mapping, uint access, uint m2, uint m, UIntPtr n);
    
     [DllImport("kernel32.dll", SetLastError = true)]
     static extern bool UnmapViewOfFile(IntPtr p);
    
     [DllImport("kernel32.dll")]
     static extern UIntPtr GetLargePageMinimum();
 }
    
 class Privilege
 {
     Type t = Type.GetType("System.Security.AccessControl.Privilege");
    
     object privilege;
    
     public Privilege(string name)
     {
  privilege = t.GetConstructor(new Type[] {typeof(string)}).Invoke(new object[] {name});
     }
    
     public void Enable()
     {
  t.GetMethod("Enable", BindingFlags.Public | BindingFlags.Instance).Invoke(privilege, null);
     }
    
     public void Revert()
     {
  t.GetMethod("Revert", BindingFlags.Public | BindingFlags.Instance).Invoke(privilege, null);
     }
 }



· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.


@GaryNebbett;
thanks you so much for your comment -and- yes will do;
this is largely what i have already existing on the C# managed side;
could not make work with inclusion of | SEC_COMMIT | SEC_LARGE_PAGES, and | FILE_MAP_LARGE_PAGES,
as always returned 1450 for any size specified;

so i switched gear and attempt to do largely same as above on the c++ side, always results in 1450 as just noted.

i will try above and comment back

0 Votes 0 ·

@GaryNebbett
thanks you so much for your comment. please find shown results:

item A:
88240-gary.png
as can see this fails

Item B: this with identical code with the exclusion of | SEC_COMMIT | SEC_LARGE_PAGES , and | FILE_MAP_LARGE_PAGES
88321-gary2.png
as can see this is success;

baffles me, any ideas? and thanks you;
jsr;

0 Votes 0 ·
gary.png (56.6 KiB)
gary2.png (44.2 KiB)
GaryNebbett avatar image
0 Votes"
GaryNebbett answered jayrhoades-8813 commented

Hello @jayrhoades-8813,

As a first step, let's just try a VirtualAlloc with MEM_LARGE_PAGES and check whether that works (code below).

Gary

 using System;
 using System.Reflection;
 using System.Runtime.InteropServices;
        
 class A
 {
     static int Main(string[] args)
     {
     try
     {
         Privilege lockmem = new Privilege("SeLockMemoryPrivilege");
        
         lockmem.Enable();
        
         X();
        
         lockmem.Revert();
     }
     catch (Exception ex)
     {
         Console.Error.WriteLine(ex);
     }
        
     return 0;
     }
        
     static void X()
     {
     UIntPtr n = GetLargePageMinimum();
        
     Console.WriteLine("0x{0:X}", n.ToUInt64());
    
     IntPtr p0 = VirtualAlloc(IntPtr.Zero, n, MEM_RESERVE | MEM_COMMIT | MEM_LARGE_PAGES, PAGE_READWRITE);
        
     Console.WriteLine("0x{0:X}, {1}", p0.ToInt64(), Marshal.GetLastWin32Error());
    
     IntPtr h = CreateFileMapping(INVALID_HANDLE_VALUE, null, PAGE_READWRITE | SEC_COMMIT | SEC_LARGE_PAGES, 0, n.ToUInt32(), "Gary");
        
     Console.WriteLine("0x{0:X}, {1}", h.ToInt64(), Marshal.GetLastWin32Error());
        
     IntPtr p = MapViewOfFile(h, FILE_MAP_ALL_ACCESS | FILE_MAP_LARGE_PAGES, 0, 0, n);
        
     Console.WriteLine("0x{0:X}, {1}", p.ToInt64(), Marshal.GetLastWin32Error());
        
     UnmapViewOfFile(p);
     CloseHandle(h);
     }
        
     static IntPtr INVALID_HANDLE_VALUE = new IntPtr(-1);
        
     const uint MEM_COMMIT = 0x00001000;
     const uint MEM_RESERVE = 0x00002000;
     const uint MEM_LARGE_PAGES = 0x20000000;
     const uint SEC_COMMIT = 0x08000000;
     const uint SEC_LARGE_PAGES = 0x80000000;
     const uint FILE_MAP_ALL_ACCESS = 0x000F001F;
     const uint FILE_MAP_LARGE_PAGES = 0x20000000;
     const uint PAGE_READWRITE = 0x04;
        
     [DllImport("kernel32.dll", SetLastError = true)]
     static extern bool CloseHandle(IntPtr handle);
        
     [DllImport("kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
     static extern IntPtr CreateFileMapping(IntPtr file, object sa, uint protect, uint n2, uint n, string name);
        
     [DllImport("kernel32.dll", SetLastError = true)]
     static extern IntPtr MapViewOfFile(IntPtr mapping, uint access, uint m2, uint m, UIntPtr n);
        
     [DllImport("kernel32.dll", SetLastError = true)]
     static extern bool UnmapViewOfFile(IntPtr p);
        
     [DllImport("kernel32.dll")]
     static extern UIntPtr GetLargePageMinimum();
    
     [DllImport("kernel32.dll", SetLastError = true)]
     static extern IntPtr VirtualAlloc(IntPtr p, UIntPtr n, uint type, uint protect);
 }
        
 class Privilege
 {
     Type t = Type.GetType("System.Security.AccessControl.Privilege");
        
     object privilege;
        
     public Privilege(string name)
     {
     privilege = t.GetConstructor(new Type[] {typeof(string)}).Invoke(new object[] {name});
     }
        
     public void Enable()
     {
     t.GetMethod("Enable", BindingFlags.Public | BindingFlags.Instance).Invoke(privilege, null);
     }
        
     public void Revert()
     {
     t.GetMethod("Revert", BindingFlags.Public | BindingFlags.Instance).Invoke(privilege, null);
     }
 }
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@GaryNebbett

nope; the VirtualAlloc() returns 0x0 and 1450 from get last err;

88331-gary3.png
hum.... something seem not so good....



jsr;

0 Votes 0 ·
gary3.png (26.0 KiB)
GaryNebbett avatar image
0 Votes"
GaryNebbett answered GaryNebbett edited

Hello @jayrhoades-8813,

I guess that you have access to more than one 64-bit Windows system; is this behaviour widespread (e.g. what percentage of your 64-bit Windows systems exhibit this behaviour)?

Because memory management is such a fundamental OS (Operating System) service, there is not much more that we can do to troubleshoot this problem remotely. You may have to try local kernel debugging if you really want to pursue the cause of this problem...

One my system, I can allocate 6000 × GetLargePageMinimum() without error but 7000 × GetLargePageMinimum() fails with error 1450, so ERROR_NO_SYSTEM_RESOURCES/STATUS_INSUFFICIENT_RESOURCES seems sometimes to be an accurate description of the problem...

Gary

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

GaryNebbett avatar image
0 Votes"
GaryNebbett answered GaryNebbett commented

Hello @jayrhoades-8813,

Here are some more suggestions.

If you have an existing crash dump file for the system, analyze it with a debugger. Find the base of the kernel image (e.g. with "lm m nt") and then check the Page Table Entry (PTE) for that page (using the "!pte" command). The kernel should be mapped with large pages and the !pte output should include the text "LARGE PAGE".

0: kd> lm m nt
start end module name
fffff803`3ec00000 fffff803`3fc46000 nt (pdb symbols) c:\windows\symbols\ntkrnlmp.pdb\769C521E4833ECF72E21F02BF33691A51\ntkrnlmp.pdb
0: kd> !pte fffff803`3ec00000
VA fffff8033ec00000
PXE at FFFF804020100F80 PPE at FFFF8040201F0060 PDE at FFFF80403E00CFB0 PTE at FFFF807C019F6000
contains 000000007B009063 contains 000000007B00A063 contains 8A000000030000A1 contains 0000000000000000
pfn 7b009 ---DA--KWEV pfn 7b00a ---DA--KWEV pfn 3000 --L-A--KR-V LARGE PAGE pfn 3000

Another command that can be issued when analyzing the crash dump is "rM aa". This should produce output like the following:

0: kd> rM aa
rax=0000000000000000 rbx=ffffce81f6010000 rcx=ffffbe00605bf440
rdx=0000000000000001 rsi=0000000000000000 rdi=ffffce8202f7b010
rip=fffff8033f5a82c4 rsp=ffff9782f9edf3e0 rbp=0000000000000000
r8=8000000000000000 r9=0000000000000000 r10=0000000000000000
r11=0000000000000000 r12=0000000000000002 r13=0000000000000000
r14=fffff8033c713180 r15=ffffce820ec7c080
iopl=0 nv up di pl zr na po nc
cs=0010 ss=0018 ds=002b es=002b fs=0053 gs=002b efl=00040046
cr0=0000000080050033 cr2=00007ffd9c0ab1e0 cr3=00000002c2ce4002
cr8=0000000000000002
dr0=0000000000000000 dr1=0000000000000000 dr2=0000000000000000
dr3=0000000000000000 dr6=00000000ffff0ff0 dr7=0000000000000400 cr4=0000000000370678
kdr0=0000000000000000 kdr1=0000000000000000 kdr2=0000000000000000
kdr3=0000000000000000 kdr6=00000000ffff0ff0 kdr7=0000000000000400
nt!IopLiveDumpEndMirroringCallback+0xb4:
fffff803`3f5a82c4 498d8e00010000 lea rcx,[r14+100h]

If you post this output, then we will be able to see what has been configured in the processor's control registers.

Try rebooting the system and then running the test programs when the system comes back up - this will maximize the chances of success if there really is a resource shortage.

Gary






· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @GarryNebbett

thanks you so much for your commentaries;
ok; here is my next question: why would simple test program behave [badly] as run from inside visual studio, and then behave in different way run via a double click/ on command line....
i intend to try test on other win 10 box, and reboot my win10 development box and carry on;
depending upon above, i may resort to kd, but would rather a remedy discovered prior to such....
thanks you;
jsr;

0 Votes 0 ·

Hello @jayrhoades-8813,

I don't think that you have previously mentioned that the test programs work when run from the command line but not when started by Visual Studio - if that is the case, it would have been useful to mention it earlier.

I don't use (and have not installed) Visual Studio, so I don't know how it starts programs. It could, for example, attach processes that it creates to a Job object (using AssignProcessToJobObject) and set Virtual Memory limits for the job.

Gary

0 Votes 0 ·