Const Generics for .NET, allowing compile-time constants as generic arguments

Const Generics in .NET
Enze He
Tokyo .NET Developers Meetup #62
2024/05/29

About me
Enze He
あだ名：さわや（Sawaya）
The University of Tokyo
Research fields:
Compute Vision and Graphics for main, compilers and runtime for hobby
GitHub: @hez2010
Microsoft MVP for .NET

Generics in .NET of Today for HPC
No statical arithmetic validation
Matrix<int> m1 = new() { Row = 3, Col = 4 };
Matrix<int> m2 = new() { Row = 5, Col = 6 };
var result = m1 * m2; // Runtime Exception: Matrix3x4 cannot be multiplied by Matrix5x6
Vector2, Vector3, Vector4 and variadic-sized Vector only
Vector2 vec2 = new(); // SIMD accelerated
// SIMD accelerated, with downgraded performance due to variadic size
Vector<float> vec = new();

User-defined Vector types won't be hardware-accelerated
struct Vector5 { ... }
Vector5 vec5 = new(); // No SIMD acceleration
Relying on Unsafe APIs and manual SIMD for acceleration
Vector256.LoadUnsafe(ref MemoryMarshal.GetArrayDataReference(array));
if (Avx2.IsSupported)
{
...
}
else if (...)
{
...
}

No generic on-stack buffer types
You have to create InlineArray one-by-one
[InlineArray(2)] struct MyBuffer2<T> { T _elem; }
...

Let's resolve all of them!
Introducing Const Generics

What is Const Generics
Constant Generics
Passing constant values to type parameters
Allowing code specialization with given constant values

High-level Design
C# Syntax
Reflection APIs
Generic InlineArray : ValueArray
Other Useful APIs

C# Syntax
class Foo<int A, float B>
{
public static void M<long X>()
{
Console.WriteLine((A + X) * B); // Use constant type arguments as values directly
}
public void N<long C>()
{
Foo<42, 42.42f>.M<C>(); // Passing constant generic arguments
Console.WriteLine(typeof(C)); // System.Int32 (the value of C)
Console.WriteLine(typeof(42)); // System.Int32 (42)
}
public void GenericOnGeneric<T, T Value>() // Generic constant generic parameters
{
GenericOnGeneric<bool, false>();
GenericOnGeneric<int, 42>();
GenericOnGeneric<float, 42f>();
}
}

Reflection APIs
namespace System;
public abstract class Type
{
public virtual bool IsConstValue { get; }
public virtual object ConstValue { get; }
public static Type MakeConstValueType(object value);
}

Reflection APIs
class Foo<T, int N> { }
var foo = new Foo<string, 42>();
foo.GetType(); // Foo<string, int (42)>
foo.GetType().GetGenericArguments()[0]; // System.String
foo.GetType().GetGenericArguments()[1].IsConstValue; // true
foo.GetType().GetGenericArguments()[1].HasElementType; // true
foo.GetType().GetGenericArguments()[1].ConstValue; // 42
foo.GetType().GetGenericArguments()[1].GetElementType(); // System.Int32
var t = typeof(42); // Type.MakeConstValue(42)
var d = typeof(Foo<,>);
d.GetGenericArguments()[1].IsConstValue; // false
d.GetGenericArguments()[1].HasElementType; // true
d.GetGenericArguments()[1].ConstValue; // InvalidOperationException
d.GetGenericArguments()[1].GetElementType(); // System.Int32
d.MakeGenericType(typeof(string), t); // Foo<string, int (42)>

Generic InlineArray : ValueArray
namespace System;
public struct ValueArray<T, int N>
{
public int Length => N;
public T this[int index] { get; set; }
private T _firstElem;
}
Reuse the same facilities of InlineArray , so that the layout of a ValueArray<T, N> is a
field _firstElem of T getting repeated for N times.

Something More
A niche syntax for declaring ValueArray :
int[42] s; // ValueArray<int, 42>
int[42, 42] w; // ValueArray<ValueArray<int, 42>, 42>
params with fixed number of parameters:
Foo(1, 2, 3, 4, 5);
void Foo(params int[5] args) { } // Only receives 5 arguments of int

Other Useful APIs
Matrix<T, int Row, int Col> : fixed-sized matrix to supersede Matrix3x3 ,
Matrix4x4 and etc.
Vector<T, int Size> : fixed-sized vector to supersede Vector2 , Vector3 and etc.
Tensor<T, int Rank> : tensor types for AI/ML purpose
Span<T, int Dimension> : ND-span that can support multiple dimension arrays
List<T, int Length> and its friends: arbitrary list types can have a fixed size now
... And more!

Low-level Design
IL Syntax
Metadata
Signature Encoding
Runtime Type Descriptor
JIT (Just-in-time) compiler
VM changes

IL Syntax
Use literal to express a type parameter receiving a constant value.
Use type (value) to express a type argument containing a constant value.
Use ldtoken to get the constant value from a type parameter.

IL Syntax
Example
.class public auto ansi beforefieldinit Foo`2<T, literal int32 N>
extends [System.Runtime]System.Object
{
.method public hidebysig specialname rtspecialname
instance void .ctor () cil managed { /* constructor */ }
.method public hidebysig newslot virtual
instance void M<literal int32 V, literal int32 W> () cil managed
{
newobj instance void class Foo`2<string, int32 (42)>::.ctor()
call instance void class Foo`2<string, int32 (42)>::M<int32 (128), !!V>()
ldtoken !!W
call void [System.Console]System.Console::WriteLine(int32)
ret
}
}

Metadata
In a typical .NET assembly, we save type information (including definitions, references
and specializations) into the metadata.
Metadata is stored as tables in the assembly
The table GenericParameterRec saves the information about generic parameters
To support defining a type parameter taking a constant value:
Save the type of the type parameter in the metadata
i.e. Adding a column Type to the generic parameter table GenericParameterRec
A record containing a valid Type field is a constant type parameter

Signature Encoding
In .NET, each type and method has its signature encoded as per ECMA-335, which
defines the standard of CIL (Common Intermediate Language).
When we call a method in C#:
The C# compiler emits a call instruction with the signature of the target method
The JIT compiler resolves the target using signature
To support passing constant values to a type parameter:
Adding ELEMENT_TYPE_CTARG as the prefix of a constant type argument
Encoding constant type arguments as ELEMENT_TYPE_CTARG <TypeSpec> <Value>
eg. ELEMENT_TYPE_CTARG ELEMENT_TYPE_I4 42

Runtime Type Descriptor
TypeDesc is a data structure representing a type in .NET runtime.
Use a ConstValueTypeDesc to hold the constant value and its type.
class ConstValueTypeDesc {
private:
uint64_t m_value;
TypeHandle m_type;
};

The JIT compiler
The importer is responsible for importing IL code into IR so that the JIT compiler can
work with.

The JIT compiler
Types and methods are encoded as tokens in the metadata table
Importer need to do a lookup when hit a token to get the target.
Then the importer can create corresponding tree nodes for doing a method call or
something else

The JIT compiler
When the JIT see a ldtoken instruction on a type parameter:
The type parameter is a constant value:
Treat it as a constant value
Create a constant node holding the value
Create tree nodes for pushing it to the stack
Otherwise:
Create tree nodes for converting the token to a TypeHandle
Create tree nodes for calling a runtime helper to instantiate a Type object

Hardware Acceleration (Future Improvements)
Make Vector<T, int Size> a well-known intrinsic type
Importer/JIT reads the const generic Size and sizeof(T) to decide SIMD
width
Map when Size * sizeof(T) == {64, 128, 256, 512} bits to
Vector64/128/256/512 respectively
Treat as JIT SIMD vector type (TYP_SIMD{64,128,256,512}) with element type
T and count Size
Allowing generic programming including its size!
Follow exactly what we are doing for Vector64/128/256/512

Specialized helper methods based on Size
With extension methods
Non-standard widths?
Non power-of-two or widths other than 64/128/256/512: tile across native
lanes with a masked tail first; scalar is a last resort
Heuristics pick strategy based on ISA, element size, and Size
Built-in helpers for lane operations, like GetLower{Upper}() , WithLower{Upper}
(x) and Slice<int Start, int Count>()

Very wide vectors (but power-of-two)
Exact tiling, no tail
Choose the widest supported native lane and process lanes with full-
width SIMD; unroll or vector-loop
Operations
Element-wise ops: run independently per tile; combine results by
concatenation
Horizontal ops (sum/min/max): perform per-tile reduction → cross-tile
combine
Shuffles/permutations: decompose into intra-tile + inter-tile exchanges

VM changes
Handle cases where a type parameter contains a Type field which indicates it's a
type parameter receiving constant values
Generic type validation for preventing type mismatch
Handle cases where a constant type argument occurs in the signature so that it can
be represented as a TypeHandle holding a constant value
Type loading
Type specialization based on constant type arguments
JIT-EE interface: interchanging data between the JIT and the Execution Engine (EE)
Adding methods for querying the value and type from a type handle
Adding methods for querying the value type from a type parameter

Use cases
Fixed buffers types with type safety: ValueArray<T, int Length> and etc.
Multi-dimension Span : Span<T, int Dim>
Constraint shape
Matrix<T, Row, NewCol> Multiply<NewCol>(Matrix<T, Col, NewCol> rMatrix)
Constant coefficients embedded into a type
struct EpsilonFloating<T, T Epsilon> where T : INumber<T>
{
public static bool operator ==(EpsilonFloating<T, Epsilon> a, EpsilonFloating<T, Epsilon> b)
=> T.Abs(a.value - b.value) <= Epsilon;
}
Expression type abstraction: IBin<Add, ICon<int, 42>, ICon<int, V>> : 42 + V
SIMD: fix-sized types like Vector<T, int Size> , Matrix<T, int Row, int Col>
... And more!

Proposals on GitHub
Runtime: (#89730) https://github.com/dotnet/runtime/issues/89730
C#: (#7508) https://github.com/dotnet/csharplang/discussions/7508

Fully-working .NET SDK with Constant Generics
Prototype
SDK download: https://1drv.ms/u/s!ApWNk8G_rszRgrxP32IMKhW-V8iWug?e=JBn8wU
Note:
No Vector<T, int Size> or Matrix<T, int Row, int Col> support
No generic constraints support
No generic arithmetic operations support

Further Considerations
Constant arithmetic?
class Foo<int T> { private Foo<T + 1> foo; }
Arithmetic generic constraints?
class Foo<int T> where T : > 42 { }
Support for strings and arbitrary value types?
class Foo<string S> { private void M() { new Foo<"hello">(); } }
Hardware accelerations?

Const Generics for .NET, allowing compile-time constants as generic arguments

More Related Content

Similar to Const Generics for .NET, allowing compile-time constants as generic arguments

Recently uploaded

Const Generics for .NET, allowing compile-time constants as generic arguments