Skip to content

Unresolved symbols: value p1(line 15) when pickling #6988

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
eed3si9n opened this issue Aug 4, 2019 · 5 comments · Fixed by #7009
Closed

Unresolved symbols: value p1(line 15) when pickling #6988

eed3si9n opened this issue Aug 4, 2019 · 5 comments · Fixed by #7009

Comments

@eed3si9n
Copy link
Member

eed3si9n commented Aug 4, 2019

minimized code

Here's a macro that finds the enclosing class's constructor parameter, and picks up the first argument.

FirstArg.scala

package foo

case class FirstArg(value: Any, source: String)
object FirstArg {
  inline given FirstArg = ${Macros.argsImpl}
}

object Macros {
  import language.experimental.macros
  import scala.quoted._

  def argsImpl(given qctx: QuoteContext): Expr[FirstArg] = {
    import qctx.tasty.{_, given}

    def enclosingParamList(owner: Symbol): Seq[Seq[Symbol]] =
      if owner.isClassDef then
         owner.tree match
           case tdef: ClassDef =>
             tdef.constructor.paramss map { _ map {_.symbol }}
      else enclosingParamList(owner.owner)

    def literal(value: String): Expr[String] =
      Literal(Constant(value)).seal.asInstanceOf[Expr[String]]
    val paramss = enclosingParamList(rootContext.owner)
    val firstArg = paramss.flatten.head
    '{ FirstArg(${Ref(firstArg).seal}, ${literal(firstArg.name)}) }
    // '{ FirstArg("?", ${literal(firstArg.name)}) }
  }
}

Test1.scala

class Test1 {
  var firstArgName = ""
  var firstArgValue: Any = ""
  def t1(): Unit = {
    val x = new Foo("something", 2L, false)
    assert("p1" == firstArgName)
    assert("something" == firstArgValue)
  }
  def debug(given foo.FirstArg): Unit = {
    firstArgName = summon[foo.FirstArg].source
    firstArgValue = summon[foo.FirstArg].value
  }
  class Foo(p1: String, p2: Long, p3: Boolean) {
    debug
  }
}
Stack trace
[error] java.lang.AssertionError: assertion failed: unresolved symbols: value p1(line 15) when pickling /private/tmp/ref/src/test/scala/Test1.scala
[error] 	at dotty.DottyPredef$.assertFail(DottyPredef.scala:16)
[error] 	at dotty.tools.dotc.core.tasty.TreePickler.pickle(TreePickler.scala:696)
[error] 	at dotty.tools.dotc.transform.Pickler.run$$anonfun$10$$anonfun$8(Pickler.scala:60)
[error] 	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[error] 	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[error] 	at scala.collection.immutable.List.foreach(List.scala:392)
[error] 	at dotty.tools.dotc.transform.Pickler.run$$anonfun$2(Pickler.scala:83)
[error] 	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[error] 	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[error] 	at scala.collection.immutable.List.foreach(List.scala:392)
[error] 	at dotty.tools.dotc.transform.Pickler.run(Pickler.scala:83)
[error] 	at dotty.tools.dotc.core.Phases$Phase.runOn$$anonfun$1(Phases.scala:316)
[error] 	at scala.collection.immutable.List.map(List.scala:286)
[error] 	at dotty.tools.dotc.core.Phases$Phase.runOn(Phases.scala:318)
[error] 	at dotty.tools.dotc.transform.Pickler.runOn(Pickler.scala:87)
[error] 	at dotty.tools.dotc.Run.runPhases$4$$anonfun$4(Run.scala:158)
[error] 	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[error] 	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[error] 	at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
[error] 	at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
[error] 	at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
[error] 	at dotty.tools.dotc.Run.runPhases$5(Run.scala:170)
[error] 	at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:178)
[error] 	at dotty.runtime.function.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
[error] 	at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:102)
[error] 	at dotty.tools.dotc.Run.compileUnits(Run.scala:185)
[error] 	at dotty.tools.dotc.Run.compileSources(Run.scala:120)
[error] 	at dotty.tools.dotc.Run.compile(Run.scala:104)
[error] 	at dotty.tools.dotc.Driver.doCompile(Driver.scala:34)
[error] 	at dotty.tools.dotc.Driver.process(Driver.scala:172)
[error] 	at dotty.tools.dotc.Main.process(Main.scala)
[error] 	at xsbt.CachedCompilerImpl.run(CachedCompilerImpl.java:69)
[error] 	at xsbt.CompilerInterface.run(CompilerInterface.java:41)
[error] 	at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
[error] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] 	at java.lang.reflect.Method.invoke(Method.java:498)
[error] 	at sbt.internal.inc.AnalyzingCompiler.call(AnalyzingCompiler.scala:237)
[error] 	at sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:111)
[error] 	at sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:90)
[error] 	at sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$3(MixedAnalyzingCompiler.scala:82)
[error] 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
[error] 	at sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:133)
[error] 	at sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:73)
[error] 	at sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:116)
[error] 	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:307)
[error] 	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:307)
[error] 	at sbt.internal.inc.Incremental$.doCompile(Incremental.scala:106)
[error] 	at sbt.internal.inc.Incremental$.$anonfun$compile$4(Incremental.scala:87)
[error] 	at sbt.internal.inc.IncrementalCommon.recompileClasses(IncrementalCommon.scala:116)
[error] 	at sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:63)
[error] 	at sbt.internal.inc.Incremental$.$anonfun$compile$3(Incremental.scala:89)
[error] 	at sbt.internal.inc.Incremental$.manageClassfiles(Incremental.scala:134)
[error] 	at sbt.internal.inc.Incremental$.compile(Incremental.scala:80)
[error] 	at sbt.internal.inc.IncrementalCompile$.apply(Compile.scala:67)
[error] 	at sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:311)
[error] 	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:269)
[error] 	at sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:159)
[error] 	at sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:238)
[error] 	at sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:69)
[error] 	at sbt.Defaults$.compileIncrementalTaskImpl(Defaults.scala:1549)
[error] 	at sbt.Defaults$.$anonfun$compileIncrementalTask$1(Defaults.scala:1523)
[error] 	at scala.Function1.$anonfun$compose$1(Function1.scala:44)
[error] 	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:40)
[error] 	at sbt.std.Transform$$anon$4.work(System.scala:67)
[error] 	at sbt.Execute.$anonfun$submit$2(Execute.scala:269)
[error] 	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error] 	at sbt.Execute.work(Execute.scala:278)
[error] 	at sbt.Execute.$anonfun$submit$1(Execute.scala:269)
[error] 	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] 	at java.lang.Thread.run(Thread.java:748)
[error] (Test / compileIncremental) java.lang.AssertionError: assertion failed: unresolved symbols: value p1(line 15) when pickling /private/tmp/ref/src/test/scala/Test1.scala

notes

Ref #5789

@liufengyun
Copy link
Contributor

liufengyun commented Aug 7, 2019

@eed3si9n : I played with the example in #7009, it seems to work after some tweaks. The original code does not work because Ref(param) refers a local variable, which is not available in the class body when pickling happens.

/cc: @nicolasstucki

I just checked, ScalaTest does not use Ref --- the constructor was initially introduced in 5419a4c#diff-e053c9a8021e18ea26ab8509e53cd72dR237.

@eed3si9n
Copy link
Member Author

eed3si9n commented Aug 7, 2019

In the code where I came across this issue, my Ref(...) also references method parameters. https://github.com/eed3si9n/sourcecode/blob/4000024c96d611f46d481bf239387ad232263ae2/sourcecode/src-3/sourcecode/Macros.scala#L159-L170

    def enclosingParamList: Seq[Seq[Symbol]] = {
      nearestEnclosingMethod(rootContext.owner) match {
        case IsDefDefSymbol(x) =>
          x.tree.paramss map { _ map {
            _.symbol
          }}
        case IsClassDefSymbol(x) =>
          x.tree.constructor.paramss map { _ map {
            _.symbol
          }}
      }
    }

@liufengyun
Copy link
Contributor

@eed3si9n Thanks for the pointer, it seems to be pretty useful.

/cc: @nicolasstucki what about just keep Ref?

@anatoliykmetyuk
Copy link
Contributor

May be related: #7008

@liufengyun
Copy link
Contributor

liufengyun commented Aug 9, 2019

I played more with Ref, its usage in macros can easily lead to cyclic references or other crashes.

inline def debug: Unit = ${Macros.debugImpl}

object Macros {
  import scala.quoted._
  import scala.tasty._

  def debugImpl given (qctx: QuoteContext): Expr[Unit] = {
    import qctx.tasty._

    def nearestEnclosingDef(owner: Symbol): Symbol =
      owner match {
        case IsDefDefSymbol(x)   => x
        case IsClassDefSymbol(x) => x
        case _                   => nearestEnclosingDef(owner.owner)
      }

    def trace(sym: Symbol): Term = {
      val ref = Ref(sym)
      '{ println(${ref.seal}) }.unseal
    }

    val stats = nearestEnclosingDef(rootContext.owner) match {
        case IsDefDefSymbol(x) =>
          x.tree.paramss.flatten map { param =>
            trace(param.symbol)
          }
        case IsClassDefSymbol(x) =>
          x.tree.constructor.paramss.flatten map { param =>
            trace(x.field(param.symbol.name).get)
          }
      }

    Block(stats, '{()}.unseal).seal.cast[Unit]
  }
}
object Test {
  def main(args: Array[String]): Unit = {
    new Foo("hello", 35, false)
    bar("world", 100, true)
  }

  def bar(a1: String, a2: Long, a3: Boolean): Unit = { // error: cyclic reference without explicit return type
    debug       // error: crash
  }

  class Foo(p1: String, p2: Long, p3: Boolean) {
    debug
  }
}
stack trace
java.lang.AssertionError: assertion failed: undefined symbol val a1#6174, ower = method bar, defined = Set(val a2#4091, val a3#4092, val a1#4090), at line 18 while compiling tests/run-macros/i6988b/Test_2.scala
Fatal compiler crash when compiling: tests/run-macros/i6988b:
assertion failed: undefined symbol val a1#6174, ower = method bar, defined = Set(val a2#4091, val a3#4092, val a1#4090), at line 18
dotty.DottyPredef$.assertFail(DottyPredef.scala:16)
dotty.tools.dotc.transform.TreeChecker$Checker.assertDefined(TreeChecker.scala:194)
dotty.tools.dotc.transform.TreeChecker$Checker.typedIdent(TreeChecker.scala:320)
dotty.tools.dotc.typer.Typer.typedNamed$1(Typer.scala:1985)
dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:2060)
dotty.tools.dotc.typer.ReTyper.typedUnadapted(ReTyper.scala:124)
dotty.tools.dotc.transform.TreeChecker$Checker.typedUnadapted(TreeChecker.scala:282)
dotty.tools.dotc.typer.Typer.typed(Typer.scala:2099)
dotty.tools.dotc.typer.Typer.typed(Typer.scala:2111)
dotty.tools.dotc.transform.TreeChecker$Checker.typed(TreeChecker.scala:267)
dotty.tools.dotc.typer.Typer.typedInlined(Typer.scala:1234)
dotty.tools.dotc.transform.TreeChecker$Checker.typedInlined$$anonfun$1(TreeChecker.scala:443)
dotty.tools.dotc.transform.TreeChecker$Checker.withDefinedSyms(TreeChecker.scala:169)
dotty.tools.dotc.transform.TreeChecker$Checker.typedInlined(TreeChecker.scala:443)
dotty.tools.dotc.typer.Typer.typedUnnamed$1(Typer.scala:2026)
dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:2061)
dotty.tools.dotc.typer.ReTyper.typedUnadapted(ReTyper.scala:124)
dotty.tools.dotc.transform.TreeChecker$Checker.typedUnadapted(TreeChecker.scala:282)
dotty.tools.dotc.typer.ProtoTypes$FunProto.$anonfun$3(ProtoTypes.scala:328)

Meanwhile, it seems that exposing symbol.tree during macro expansion is dangerous, which could easily lead to cycles. It becomes safer if blackbox macros expand after typer.

liufengyun added a commit to dotty-staging/dotty that referenced this issue Dec 16, 2019
liufengyun added a commit to dotty-staging/dotty that referenced this issue Jan 14, 2020
nicolasstucki added a commit that referenced this issue Jan 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants