How to prebuild C# generics before runtime

vb 276

Hi,

Currently, C# generics are built at runtime. First time, generics execution is very slow because runtime compilation.

How to avoid runtime compilation...how to get target x64 or x86 binaries where generics are already compiled to native binary.

Is there LINK option for that I am missing?

Thanks in advance!
Vladimir

Accepted answer

Alberto Poblacion 1,556 Reputation points

2021-01-14T14:50:48.283+00:00

If I understand correctly, you want to bypass the JIT-compiler (Just In Time). For classical .Net Framework, you have a tool called NGEN.EXE (Native Image GENerator), which can take your IL files and turn them into executable binaries. Note that this is unrelated to whether you use Generics or not; the JIT and NGEN apply to any code with or without Generics.

Link to documentation.
Please sign in to rate this answer.

1 person found this answer helpful.

0 comments No comments
Sign in to comment

1 additional answer

vb 276 Reputation points

2021-01-14T15:33:41.067+00:00

So, you want to say there is no LINK/BUILD option in Visual Studio that can build native image?

Only NGEN.exe that you must call from script or manually from cmd?
Please sign in to rate this answer.
Alberto Poblacion 1,556 Reputation points

2021-01-14T15:51:30.573+00:00

Indeed, the build/link option generates what appears to be an .exe file, but it only contains a tiny bit of executable code that loads the .Net runtime, and the rest is just Intermediate Language (IL) which needs to be JIT-compiled before it can be executed. To convert the IL into a native binary, you need to post-process the file with NGEN. Note that this refers to the "classical" .Net Framework; for .Net Core there is a separate set of tools.

vb 276 Reputation points

2021-01-15T09:37:29.23+00:00

First, thank you very much for answer!

Hm, NGEN is almost no use...did you see MSDN topic about it?

It is strange, and probably there is some good reason why C# does not have a prori generation of best optimal code and does not generate best (fastest) code.
This is big drawback.

For example, experimentally, one line of generic code first time took 20 milisec to execute...and when you compare with 1 ms to CopyMemory for 8 Mb memory...it is unforgivable.
For time demanding applications, like "realtime" applications, DirectShow filters, DirectX ... when 60 FPS = 16 ms is time you want to generate drawing commands and present new screen to user...and while you are doing it...it must first compile it?

vb 276 Reputation points

2021-01-15T09:41:45.733+00:00

It would be nice, logical, when you first start application, that system first compile it to best binary, and then start execution. Second start, get this binary from cache without compilation. Why to have separate tool like NGEN?

Also, I have not find any attribute or #pragma like declaration to explicitly say ... "these lines of code/function must fly...compile it best you can for speed".

C# is here for very long time...it must be a good reason why this is not implemented jet.

Viorel 112.5K Reputation points

2021-01-15T09:53:01.207+00:00

If you force the construction of generic types on initialization stage (executing something like new List<MyStructure> or new MyGenericClass<MyStructure>), when the performance is not so important, then it should use the already created code later at run-time.

Probably C# is not always fully suitable for real-time applications. You can make some parts in other languages, such as C++ or Assembler.

vb 276 Reputation points

2021-01-15T10:07:36.913+00:00

Yes, I agree...use C++ whenever you need speed. (assembler sometimes)

But, I don't see logical reason for C# not to be fast as C++. Ok, you must have CPU optimized for boundary condition checks.... more watts per instruction... but should be as fast as C++.

For example, you could even train code to be faster...to automatically find heat points...and next "binary" product iteration to be faster. Each time you test exe, ... JIT could find points where further optimization is needed...save that in cache...and next time app is faster. I hope currently system is doing something like this.

Because, if user start application, and each start JIT compiles from beginning...olala....

vb 276 Reputation points

2021-01-15T10:24:44.417+00:00

Still, I don't understand generics cannot be set to natural form when you build application...

if you, for example, have class SomeGenClass<T>

and you specify explicitly in the code.... SomeGenClass<byte> ... it should be set, like in C++ template after build, in complied form where all elements "T" are replaced for "byte".

Alberto Poblacion 1,556 Reputation points

2021-01-15T13:31:33.667+00:00

Well, it does... more or less. When you start the IL exe, for each method that is executed the method is JIT-Compiled and cached in memory. This is fast because only the methods that you actually invoke need to be compiled. If a method is invoked again, it does not need to be recompiled, since the executable is already cached in memory.
However, there is currently no option to save to disk these pieces that have already been compiled. Would it be worth doing? I'm not sure. The JIT compiler is so fast that there is a possibility that loading the compiled code from disk might be slower that JIT-compiling the IL. Someone must have analyzed it before deciding that the "good" options were the ones that we have, i.e., either precompile everything with NGEN or JIT-compile in memory the pieces that are being used.

vb 276 Reputation points

2021-01-15T13:56:16.147+00:00

I was expecting bytecode compiled once, at the first start, into native binaries. Each other start, native images are loaded and executed. And, if new bytecode compiler is installed on a system, only then it re-compiles bytecode to natives. I don't think loading from hard disk was ever problem...and especially now when SSD is common. But, I was wrong.
(If JIT is faster, then NGEN images are slower? I'm not sure. Maybe loading is sometimes faster, but execution is always slower.)

So, only solution for me is to start script NGEN during install per each assembly.

Also, there is solution like Multicore JIT...but I don't like idea saving its profile at location other than program files. I haven't tried, but security will not permit saving Multicore JIT profile at program files location.

Alberto Poblacion 1,556 Reputation points

2021-01-15T13:57:39.857+00:00

No, C# works differently. It does not replace the type in the template like C++. Instead, Generics have direct support in the libraries of the Framework. So the T is passed as a parameter and its type is used at runtime. This means that your code contains a single compiled instance of SomeGenClass<T>, regardless of how many different T's as used within your program.

vb 276 Reputation points

2021-01-15T14:17:09.917+00:00

But, this is bytecode level you are talking. After JIT...compiled into native form like C++ compiler does.
Of course, NGEN will do the same...generic class will become native code.
So what is the difference in not to evaluate each bytecode SomeGenClass<byte> into bytecode "SomeGenClassByte" during application bytecode generation (aka build).
No one will use this class from outside or inside as Generics<T>, because, it is explicitly declared in the code as Generics<SomeType>. After the C# msbuild, I don't see need for implemented bytecode generics to stay non-implemented bytecode generics. You can leave signature in bytecode Generics<T>...for eventual late reuse from other applications...but...where it is fixed as Generics<SomeType> ... it is sealed, declared, "programmer want to have GenericsSomeType in this place."!
Of course, you will have somewhat longer bytecode after build, but, JIT will compile it faster,

Alberto Poblacion 1,556 Reputation points

2021-01-15T14:33:53.81+00:00

Errr... no, it does not exactly get "compiled into native form", or at least not completely. When you write a line of source code like this:
var list = new List<int>();
it gets compiled into IL code like this:
IL_0001: newobj instance void class [mscorlib]System.Collections.Generic.List`1<int32>::.ctor()
As you can see, it is a call into the mscorlib library, and the library is the one that knows how to process the Generic. In this case, the argument points to System.Collections.Generic, but it works the same way if you call a class of your own.
When you compile this into native code, the thing that gets compiled is the newobj IL code, which is converted into a CALL into the appropriate endpoint of the library. But the library still treats the Generic as Generic, it does not generate a native version that replaces the T with int.
And no, this will not be as fast as C++. If you need to do video processing or similar, C# will not give you top performance.

vb 276 Reputation points

2021-01-15T14:52:16.177+00:00

So, conclusion is not to use generics in critical places. NGEN will not produce optimized native code from explicit SomeGeneric<int> ?

If this is true, lol, generics are trouble.

For var...OK ... it is var... but for explicit SomeGeneric<int> not to become optimized native code after NGEN...it is bad issue!
For SomeGeneric<int> not to become optimized native code after NGEN ... I am hoping this is not true. Not making any sense, totally qualifies generics as practically unusable. I am in disbelief here... are you sure? :D Maybe we have misunderstood each other. Don't scare me!

( Don't worry, I would never use C# for video encoding lol ... C++ and assembler... C# is for businesses part of app, for orchestration. To use C# for heavy loads...there must be CPU optimized for C#, or use unsafe C# pointers...pointers could be much faster)

Viorel 112.5K Reputation points

2021-01-15T15:02:41.123+00:00

According to documentation, if T is a value type (e.g. byte or a structure), then the system will generate a new specialized version of the generic type, where T is substituted. (Different code for each T).

If T is a reference type (class), then the system generates a single specialized version, which is shared by all reference types. (A single code for all T).

vb 276 Reputation points

2021-01-15T15:13:16.797+00:00

Of course, it is logical that after JIT or NGEN compilation into native code...generics are lost...code is optimized for some explicit value type variant.

For reference type is also expected to have one generated native code same as all classes are...only pointer to class memory block is changing...native class code is the same.

vb 276 Reputation points

2021-01-17T19:59:23.623+00:00

Hi!

I have been looking all over...exempt disassembly....

There is:

Build...when you start C# code build...it produces bytecode...( aka CIL, aka MSIL, aka IL). It is C# source compiled to bytecode and has metadata about code types like particular generic you wrote.

Each start...JIT...compiles code to native on demand...(standard)... bytecode is compiled to native just before execution reaches it.

NGEN ... bytecode is pre-compiled to native code, but you have also C# metadata for reflection to work in runtime. Probably, most reflection metadata is stored in original bytecode image, and NGEN image has metadata references to original bytecode.

.NET Native ... this is fully compiled to native... but has no metadata about types and you cannot use reflection. (this is only UWP feature, and that makes me angry, but I am happy because it is flawed, mind-limited as same as UWP is)

Multi-core JIT ... JIT compiles using multi core approach and has some extra metadata-profile for speed.

vb 276 Reputation points

2021-01-17T20:02:26.657+00:00

Hi!

I have been looking all over...exempt disassembly....

There is:

Build...when you start C# code build...it produces bytecode...( aka CIL, aka MSIL, aka IL). It is C# source compiled to bytecode and has metadata about code types like particular generic you wrote.

Each start...JIT...compiles code to native on demand...(standard)... bytecode is compiled to native just before execution reaches it.

NGEN ... bytecode is pre-compiled to native code, but you have also C# metadata for reflection to work in runtime. Probably, most reflection metadata is stored in original bytecode image, and NGEN image has metadata references to original bytecode.

.NET Native ... this is fully compiled to native... but has no metadata about types and you cannot use reflection. (this is only UWP feature, and that makes me angry, but I am happy because it is flawed, mind-limited as same as UWP is)

Multi-core JIT ... JIT compiles using multi core approach and has some extra metadata.

vb 276 Reputation points

2021-01-17T20:18:45.447+00:00

In all cases... SomeGeneric<int> is compiled to native "SomeGenericInt", (or, if it is not, it is big flaw.)

For "var"...is different...you need to have runtime choice..."switch" to appropriate value. It is like C++ VARIANT...you must look structure VARIANT.vt field and them make decision how to interpret it into final cast value. If it is VARIANT string...you must interpret it to pointer where actual string is in memory. If it is integer...then you cast part of the variant structure into int. Equivalent is "var", you have extra code to interpret it correctly in the runtime.

For SomeGeneric<T>...that is set in code like:

SomeGeneric<int> a = new SomeGeneric<int>(); => ...it is obvious ...it is compiled in native image.

But, for runtime reflection to work, you also must have SomeGeneric<T> metadata passively stored and standing by your native code.

vb 276 Reputation points

2021-01-17T20:42:10.777+00:00

For NGEN it is worth to mention:

Yes, is can trigger JIT. When some referenced system assembly is changed (like during windows update)...JIT will try to reinterpret these parts of your native NGEN code that reference changed assemblies. This will have negative prefomance effect in comparative to firstly NGEN-ed image.

Appropriate conclusion is for NGEN...whenever there is windows update...create new build, and deliver new version of your application so user will again ... NGEN code during install. This will limit JIT triggering.

It is strange that there is no simple flag for setup....to NGEN automatically when install completes. Now, one have to maintain NGEN-script. Every time one add/change assemblies, one must update NGEN script by hand.

vb 276 Reputation points

2021-01-17T21:17:37.77+00:00

For all I know...if:

For SomeGeneric<T>...that is set in code like: SomeGeneric<int> a = new SomeGeneric<int>(); => ...it is obvious ...it is compiled in native image.

If this is not true, then, Microsoft must start reality check. Because, reinterpreting is slow as hell, and it would be bad design....and certainly it is not innovation.

vb 276 Reputation points

2021-01-30T12:19:04.857+00:00

Hi!

I have found that .NET Core has publish option ReadyToRun ...

It compiles code to native. Final result is mix of IL and native code. IL metadata is needed for runtime reflection and native code is optimized per system (win, linux, macOS) and platform (x86, x64 or ARM).

It looks like ReadyToRun native code is NOT generated at the client machine. If this is true, than it means that native code is not optimized for particular client CPU, rather, it produces some not well optimized general native result.

vb 276 Reputation points

2021-01-30T12:44:08.093+00:00

So..., simplified pattern to optimize native code at client place would be like this...

Generate, setup & ship IL-code assemblies having flag set "NGEN-Me".

On a client, when such assemble is executed...do JIT-ing at first. If assembly has flag "NGEN-Me", start slowly background build, build generating NGEN-ed image, image optimized for platform and current client CPU.

When an assembly native version build is completed, run it at next application startup instead of IL code.

If user change CPU, fallback to JIT-ing until new, recreated NGEN-ed, per CPU optimized image build is completed.

If there is reflection...of course...this parts of code must be JIT-ed.
(Ideally ... publish IL+NGEN ... client reNGEN)

This pattern I would expect to be available by 2015. year..or much before. This AOT pattern would be logical at the first for anybody who intended to create bytecode ready to run on any machine.

vb 276 Reputation points

2021-01-30T13:02:53.553+00:00

All current C# AOT solutions are incomplete...
I don't believe anybody in industry is doing AOT (Ahead Of Time compilation) correctly...and that is big shame after so many years bytecode is known as common jargon.

vb 276 Reputation points

2021-02-01T11:55:14.61+00:00

Hi!

I have live example for ReadyToRun where you can see that generics are really compiled into native....case:

I have successfully converted Framework 4.8 solution into .NET 5.0last. Solution uses Microsoft.EntityFrameworkCore.Sqlite where Generics are all over the place.

Now compare .NET 5.0 with and without of ReadyToRun...for only connect to database performances...

Release build, IL is JIT-ed into native at first pass...timing: 1-2 second! Like eternity! Just to create first new instance of DbContext class...200 ms...and rest to reflect db model and connect.

Published Release ReadyToRun timing: Less then 300 ms! For database context it is like immediately!

At the first testing, after conversion, I was like to write new topic how connecting to database is so ZX Spectrum Basic slow. Now, I am satisfied.

Best regards!

vb 276 Reputation points

2021-02-01T18:38:34.41+00:00

PS... LOL ... Uh...now I see I was looking at old Framework 4.8 setup binaries like they are ReadyToRun. Tough, result is much better at ReadyToRun 5.0 version but not as good as old provider I have used at Framework 4.8.

vb 276 Reputation points

2021-02-03T17:12:32.593+00:00

LOL!!!

I have tested what is actually true... @ ReadyToRun AOT variant.

Answer is NO!!! Generics are not compiled into native when using ReadyToRun ... JIT will run generics!!!

I have this timing results when using Microsoft.EntityFrameworkCore.DbContext as example.
(Also, for to be sure, I have tried with another generics.)

Code is like this...

vb 276 Reputation points

2021-02-03T17:16:24.537+00:00

Run...

CContext c1; CContext c2; CContext c3; CContext c4; CContext c5; using (MeasureTime.Start("1")) { c1 = new CContext(); } using (MeasureTime.Start("2")) { c2 = new CContext(); } using (MeasureTime.Start("3")) { c3 = new CContext(); } using (MeasureTime.Start("4")) { c4 = new CContext(); } using (MeasureTime.Start("5")) { c5 = new CContext(); }

vb 276 Reputation points

2021-02-03T17:16:53.753+00:00

where MeasureTime class has Stopwatch and on Dispose it starts MessageBox message showing Stopwatch.Start,.Stop,.Elipsed => elipsed time
and...

public class ClassA { public Int64 ID { get; set; } public String Value { get; set; } } public class ClassB { public Int64 ID { get; set; } public String Value { get; set; } } public class CContext : Microsoft.EntityFrameworkCore.DbContext { private DbSet<ClassA> classA; public DbSet<ClassA> ClassA { get { return classA; } set { classA = value; } } private DbSet<ClassB> classB; public DbSet<ClassB> ClassB { get { return classB; } set { classB = value; } } }

vb 276 Reputation points

2021-02-03T17:30:28.057+00:00

Timing results:

NO ReadyToRun (Built)...

1- 150 ms
2- 0,20 ms
3- 0,20 ms
4- 0,05 ms
5- 0,05 ms

ReadyToRun (Published)...

1- 63 ms
2- 0,80 ms
3- 0,06 ms
4- 0,04 ms
5- 0,04 ms

Conclusion...
It is obvious that JIT is working here... in both versions!
Thou, ReadyToRun timing is third of Build time for the first run,
it is astronomical difference how fast actually code can go, and SHOULD GO.

I'm very disappointed! ReadyToRun is only Half-ReadyToRun!
Only way to optimize particular code so JIT is not fired...is to do similar code in a loop before actual critical code is executed!?
I'm not sure that I can NGEN NET 5.0 assemblies...and not sure if NGEN will create better generics start times.
I will open another topic about this ReadyToRun issue.

vb 276 Reputation points

2021-02-03T17:40:16.69+00:00

I'm disappointed that, after so many years, we cannot tell compiler...by some pragma like...

#pragma DoNativeOptimization ...some code... #end

Or/and attribute for class/method/property.

Before runtime, compiler will never be so smart to deduce, which part of code is critical.
It is obvious this is the missing link.
Currently, we are in the Stone age of AOT. LOL
Sign in to comment