First-class span support causes translation failure with Contains

[jamesgurung]

Since installing .NET 9, I have come across a bug in production where setting <LangVersion>preview</LangVersion> results in certain LINQ queries causing an InvalidOperationException. This did not happen with .NET 9 RC2, however now that I've installed the .NET 9.0.100 SDK it always occurs even if I roll back EF Core.

Minimal repro:

Project.csproj

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>net9.0</TargetFramework>
    <LangVersion>preview</LangVersion> <!-- NO EXCEPTION IF THIS LINE IS REMOVED -->
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.EntityFrameworkCore" Version="9.0.0" />
    <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="9.0.0" />
  </ItemGroup>
</Project>

Program.cs

using Microsoft.EntityFrameworkCore;

var ctx = new BloggingContext();
ctx.Database.EnsureCreated();
var list = Array.Empty<int>();
var query = ctx.Blogs.Where(b => list.Contains(b.Id)).ToQueryString(); // <-- THROWS InvalidOperationException

public class BloggingContext : DbContext
{
  public DbSet<Blog> Blogs { get; set; }
  protected override void OnConfiguring(DbContextOptionsBuilder options)
  {
    options.UseSqlite("DataSource=file::memory:?cache=shared");
    options.LogTo(Console.WriteLine);
    options.EnableSensitiveDataLogging();
  }
}

public class Blog
{
  public int Id { get; set; }
}

Exception

System.InvalidOperationException: 'An exception was thrown while attempting to evaluate the LINQ query parameter expression 'op_Implicit(value(Program+<>c__DisplayClass0_0).list)'. See the inner exception for more information.'

   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.<Evaluate>g__EvaluateCore|70_0(Expression expression, String& parameterName, Boolean& isContextAccessor)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Evaluate(Expression expression, String& parameterName, Boolean& isContextAccessor)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.ProcessEvaluatableRoot(Expression evaluatableRoot, State& state, Boolean forceEvaluation)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.EvaluateList(IReadOnlyList`1 expressions, State[] expressionStates, List`1& children, Func`2 pathFromParentGenerator)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.VisitMethodCall(MethodCallExpression methodCall)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit(Expression expression, State& state)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.VisitLambda[T](Expression`1 lambda)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit(Expression expression, State& state)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.VisitUnary(UnaryExpression unary)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit(Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit[T](ReadOnlyCollection`1 expressions, Func`2 elementVisitor, StateType& aggregateStateType, State[]& expressionStates, Boolean poolExpressionStates)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit(ReadOnlyCollection`1 expressions, StateType& aggregateStateType, State[]& expressionStates, Boolean poolExpressionStates)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.VisitMethodCall(MethodCallExpression methodCall)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.Visit(Expression expression, State& state)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.ExtractParameters(Expression expression, IParameterValues parameterValues, Boolean parameterize, Boolean clearParameterizedValues, Boolean precompiledQuery, IReadOnlySet`1& nonNullableReferenceTypeParameters)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.ExtractParameters(Expression expression, IParameterValues parameterValues, Boolean parameterize, Boolean clearParameterizedValues)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExtractParameters(Expression query, IParameterValues parameterValues, IDiagnosticsLogger`1 logger, Boolean compiledQuery, Boolean generateContextAccessors)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteCore[TResult](Expression query, Boolean async, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToQueryString(IQueryable source)
   at Program.<Main>$(String[] args) in D:\Projects\EFCoreDemo\Program.cs:line 6

Inner Exception

GenericArguments[1], 'System.Span`1[System.Int32]', on 'System.Linq.Expressions.Interpreter.FuncCallInstruction`2[T0,TRet]' violates the constraint of type 'TRet'.

   at System.RuntimeType.ValidateGenericArguments(MemberInfo definition, RuntimeType[] genericArguments, Exception e)
   at System.RuntimeType.MakeGenericType(Type[] instantiation)
   at System.Linq.Expressions.Interpreter.CallInstruction.GetHelperType(MethodInfo info, Type[] arrTypes)
   at System.Linq.Expressions.Interpreter.CallInstruction.SlowCreate(MethodInfo info, ParameterInfo[] pis)
   at System.Linq.Expressions.Interpreter.CallInstruction.Create(MethodInfo info, ParameterInfo[] parameters)
   at System.Linq.Expressions.Interpreter.LightCompiler.CompileMethodCallExpression(Expression object, MethodInfo method, IArgumentProvider arguments)
   at System.Linq.Expressions.Interpreter.LightCompiler.Compile(Expression expr)
   at System.Linq.Expressions.Interpreter.LightCompiler.CompileConvertUnaryExpression(Expression expr)
   at System.Linq.Expressions.Interpreter.LightCompiler.Compile(Expression expr)
   at System.Linq.Expressions.Interpreter.LightCompiler.CompileTop(LambdaExpression node)
   at System.Linq.Expressions.Expression`1.Compile(Boolean preferInterpretation)
   at Microsoft.EntityFrameworkCore.Query.Internal.ExpressionTreeFuncletizer.<Evaluate>g__EvaluateCore|70_0(Expression expression, String& parameterName, Boolean& isContextAccessor)

Version

EF Core version: 9.0.0
Database provider: Microsoft.EntityFrameworkCore.Sqlite
Target framework: .NET 9.0
Operating system: Windows 11
IDE: Visual Studio 2022 17.13 Preview 1

[roji]

@jamesgurung I pasted both your csproj and Program.cs exactly as is with .NET SDK 9.0.100, and the program ran fine... Printing the result of ToQueryString() gives the following:

SELECT "b"."Id"
FROM "Blogs" AS "b"
WHERE "b"."Id" IN (
    SELECT "l"."value"
    FROM json_each(@__list_0) AS "l"
)

Can you please double-check and confirm?

[jamesgurung]

Hi @roji, thanks for getting back so quickly! I'm consistently getting the error in Visual Studio 2022 17.13.0 Preview 1.0, but I've just checked and it runs fine on the command line with dotnet run. Does this mean it's a VS issue?

[roji]

Hmm, am not sure about that - it wouldn't make sense for VS to be the cause of a bug like that... @ajcvickers feel like trying to repro this?

[pinkfloydx33]

Is this related to this: dotnet/runtime#109757?

[roji]

@pinkfloydx33 definitely seems related - let's see what happens there. If the change is intended on the .NET side, we could look at reacting to it here in EF.

[ChrisJollyAU]

I was trying to build main and later release/9.0 using VS 2022 17.13 preview 1

8>D:\toolkits\efcore\test\EFCore.Specification.Tests\CustomConvertersTestBase.cs(1187,49,1187,50): error CS0023: Operator '.' cannot be applied to operand of type 'void'
8>D:\toolkits\efcore\test\EFCore.Specification.Tests\CustomConvertersTestBase.cs(1188,49,1188,50): error CS0023: Operator '.' cannot be applied to operand of type 'void'
8>D:\toolkits\efcore\test\EFCore.Specification.Tests\Query\PrimitiveCollectionsQueryTestBase.cs(430,67,430,74): error CS8620: Argument of type 'string[]' cannot be used for parameter 'span' of type 'Span<string?>' in 'bool MemoryExtensions.Contains<string?>(Span<string?> span, string? value)' due to differences in the nullability of reference types.
8>D:\toolkits\efcore\test\EFCore.Specification.Tests\Query\PrimitiveCollectionsQueryTestBase.cs(433,68,433,75): error CS8620: Argument of type 'string[]' cannot be used for parameter 'span' of type 'Span<string?>' in 'bool MemoryExtensions.Contains<string?>(Span<string?> span, string? value)' due to differences in the nullability of reference types.
8>D:\toolkits\efcore\test\EFCore.Specification.Tests\Query\PrimitiveCollectionsQueryTestBase.cs(577,67,577,76): error CS8620: Argument of type 'string[]' cannot be used for parameter 'span' of type 'Span<string?>' in 'bool MemoryExtensions.Contains<string?>(Span<string?> span, string? value)' due to differences in the nullability of reference types.

On the CustomConvertersTestBase it is an error in this portion. Specifically, with the v.Reverse(). Instead of picking up the IEnumerable version it is picking up with MemoryExtensions.Reverse(Span). Defined as public static void Reverse<T> (this Span<T> span);

I will say that if I hover of the Reverse, the tooltip shows me the Span version, however if I click on it and navigate to its definition I come to the IEnumerable

b.Property(e => e.ByteArray5)
    .HasConversion(
        new ValueConverter<byte[], byte[]>(
            v => v.Reverse().Concat(new byte[] { 4, 20 }).ToArray(),
            v => v.Reverse().Skip(2).ToArray()),
        bytesComparer)
    .HasMaxLength(7);

Note that the Directory.Build.props for the test folder does define <LangVersion>preview</LangVersion>

Also, whether or not it has any part of this, the VS Installer for 17.13 preview 1 still installs 9.0.0-rc.2.24474.4 although it doesnt appear to be being used.

Somehow or other, at least in some instances when the LangVersion is on preview, it almost like it the preferences have changed so that it is preferring/finding the MemoryExtensions functions first before the Linq version

Edit to add: Tried in 17.12. and it looks fine there even when LangVersion is on preview. Appears to be something only popping up when you use 17.13 preview 1 AND LangVersion is on preview

[roji]

Thanks for diving into it @ChrisJollyAU... That's all sounds quite odd indeed. It feels like we should wait and see if the change (and all its consequences) are indeed intentional and desired (in dotnet/runtime#109757); we can react in EF e.g. by normalizing MemoryExtensions.Reverse() to Enumerable.Reverse() at some point in the query pipeline (though there seem to be deeper consequences around generics, as that issue shows). But it seems prudent to first see what's said on the runtime side I think...

[ChrisJollyAU]

@jamesgurung Can you try with VS 17.12? Even with LangVersion on preview? Would be nice for extra confirmation it only affects the preview branch of Visual Studio.

@roji I primarily use the preview branch of VS and haven't encountered this anywhere during the various 17.12 previews. Might just need to make a note of this as a known issue for 17.13 for now.

[ilGianfri]

I've attached a simple console app to repro this issue ConsoleApp2.zip

[ChrisJollyAU]

@ilGianfri Tried in both VS 17.12 and VS 17.13p1 and had no errors

[ilGianfri]

@ilGianfri Tried in both VS 17.12 and VS 17.13p1 and had no errors

That's strange, I get this

[ChrisJollyAU]

@ilGianfri Yeah, I get that if I update the project to target .Net 9 (your attached is targeting net 8).

I only get that when using 17.13p1. If you also get it on 17.12 please try a full clean and rebuild so that there are no artifacts from the roslyn version used by 17.13p1

This is something you would only pick up from 17.13p1 (further details coming up)

[ilGianfri]

@ilGianfri Yeah, I get that if I update the project to target .Net 9 (your attached is targeting net 8).

I only get that when using 17.13p1. If you also get it on 17.12 please try a full clean and rebuild so that there are no artifacts from the roslyn version used by 17.13p1

This is something you would only pick up from 17.13p1 (further details coming up)

I tried to rebuild it using 17.12 and I don't get the crash, I get the crash if I rebuild it using using 17.13p1 on both .NET 8 and .NET 9