Skip to content

Rewrite right-associative rewrite #7741

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 5 commits into
base: 2.13.x
Choose a base branch
from

Conversation

som-snytt
Copy link
Contributor

@som-snytt som-snytt commented Feb 12, 2019

Defer any rewriting from parser to typedApply.

The application is typechecked in the normal way
and evaluation order rejiggered after. In particular,
there is no need to re-typecheck after re-inlining,
for more precise types.

Original intention was to piggyback on the fix for 1980 by inlining a bit more, but typing is largely done by the time the rewrite is rewritten. This is retronym's suggested follow-up, perhaps.

Fixes scala/bug#4518
Fixes scala/bug#5073
Fixes scala/bug#10693
Fixes scala/bug#11117

Aggravates bugs 5073 (pos/tcpoly_ticket2096.scala) and maybe 11397 (run/t4225e.scala, which demonstrates 10693).

@som-snytt
Copy link
Contributor Author

@lrytz thx for bumping this on the one-year anniversary.

@lrytz
Copy link
Member

lrytz commented Feb 13, 2020

🎈

@lrytz
Copy link
Member

lrytz commented Feb 13, 2020

Here's an example that shows the issue with this approach:

class A {
  def /: (z: A)(op: Int): A = ???
  def self = this
  def bar(x: A, y: A) = (x.self /: y.self)(1)
}

in typedApply of y.self./:(x.self), we build a block

{
  final <synthetic> <artifact> val rassoc$1: A = x.self;
  y.self./:(rassoc$1)
}

Calling typed(blk) fails because the typer through typedBlock, and typing blocks is not FUN in terms of mode. So we get

error: missing argument list for method /: in class A
Unapplied methods are only converted to functions when a function type is expected.

Just putting the typer in FUNmode wouldn't be enough, as the second param list needs to go into the block.

Names and defaults has the same problem, and does it with a lot of complexity (it builds a block for the inner Apply, then takes it apart for outer Applys and inserts them into the block). Here, it should be possible to avoid this complexity and just build the block at the very end, after type checking the full application (y.self./:(x.self)(1))?

@lrytz
Copy link
Member

lrytz commented Feb 13, 2020

A different issue caused by this PR is that we get a lint warning on (1, 2) :: Nil

Test.scala:2: warning: adapted the argument list to the expected 2-tuple: add additional parens instead
        signature: List.::[B >: A](elem: B): List[B]
  given arguments: 1, 2
 after adaptation: List.::((1, 2): (Int, Int))
  val x = (1,2) :: Nil
                ^

before the parser would spit out

    val x = {
      final <synthetic> <artifact> val rassoc$1 = scala.Tuple2(1, 2);
      Nil.$colon$colon(rassoc$1)
    }

but now it's val x = Nil.$colon$colon(1, 2).

@som-snytt
Copy link
Contributor Author

Thanks @lrytz for the insight. If I wake up later today, I will try to grok it.

I wasn't sure if returning to this PR was worth the communal trouble; should I change the title to "Complexify right-associative rewrite"?

If we had mini-phases, we could have a mini-phase called p5073 after the ticket. Every ticket could have a test and a custom transform.

There's a similar challenge in the stabilization PR about whether to muck with trees in-flight, though there the effect is artificial rather than actual order of evaluation.

@lrytz
Copy link
Member

lrytz commented Feb 13, 2020

worth the communal trouble

I like the the simplification, it seems more tractable

similar challenge in the stabilization PR

Yeah.. I'm looking at scala/bug#11756 tomorrow, I think it would be good to have in 2.13.2, maybe just a minimal fix without changing the strategy.

@som-snytt som-snytt marked this pull request as draft April 17, 2020 00:53
@som-snytt som-snytt closed this Apr 20, 2020
@som-snytt som-snytt reopened this May 9, 2020
@SethTisue SethTisue modified the milestones: 2.13.3, 2.13.4 May 12, 2020
@som-snytt som-snytt closed this Jul 27, 2020
@SethTisue SethTisue removed this from the 2.13.4 milestone Jul 28, 2020
@som-snytt som-snytt reopened this May 19, 2021
@scala-jenkins scala-jenkins added this to the 2.13.7 milestone May 19, 2021
@som-snytt som-snytt changed the title Simplify right-associative rewrite Simplify right-associative rewrite [ci: last-only] May 20, 2021
@som-snytt som-snytt force-pushed the issue/5073-rassoc-method-value branch from 6e754c2 to 7d8ebee Compare May 20, 2021 04:14
@som-snytt
Copy link
Contributor Author

som-snytt commented May 20, 2021

@lrytz A year later, I piggybacked on your fix for stabilization.

Edit: I see the (42, 27) :: Nil idiom is idiomatic, so I will try to mitigate that. I mentioned it on discord.

Appselmode sounds like a Swiss tart made with little apples.

@som-snytt
Copy link
Contributor Author

som-snytt commented May 20, 2021

The previous easy tweak worked, but now:

[error] /home/travis/build/scala/scala/src/reflect/scala/reflect/internal/util/package.scala:17:23: Unused import
[error] import scala.language.existentials // scala/bug#6541
[error]                       ^
[error] one error found

I'll try a local bootstrap after all.

...oh I guess it's fatal on CI now. More late pandemic brain fog.

Except for the part where

[error] /home/jenkins/workspace/scala-2.13.x-validate-main/src/reflect/scala/reflect/internal/util/package.scala:46:13: inferred existential type Class[?0] forSome { type ?0 >: _$1; type _$1 }, which cannot be expressed by wildcards, should be enabled
[error] by making the implicit value scala.language.existentials visible.
[error] ----
[error] This can be achieved by adding the import clause 'import scala.language.existentials'
[error] or by setting the compiler option -language:existentials.
[error] See the Scaladoc for value scala.language.existentials for a discussion
[error] why the feature should be explicitly enabled.
[error]       clazz.getSuperclass :: clazz.getInterfaces.toList map (c => shortClass(c)) mkString " with "
[error]             ^
[error] one error found
[error] (reflect / Compile / compileIncremental) Compilation failed

Rebroke this:

    def shortClass(clazz: Class[_]): String = {
  final <synthetic> <artifact> val rassoc$1: Class[?0] forSome { type ?0 >: _$1; type _$1 } = clazz.getSuperclass();
  scala.Predef.wrapRefArray[Class[_]](clazz.getInterfaces()).toList.::[Class[_]](rassoc$1)
}.map[String](((c: Class[_]) => C.this.shortClass(c))).mkString(" with ")
  }

// vs
    def shortClass(clazz: Class[_]): String = {
      final <synthetic> <artifact> val rassoc$1: Class[?0] = clazz.getSuperclass();
      scala.Predef.wrapRefArray[Class[_]](clazz.getInterfaces()).toList.::[Class[_]](rassoc$1).map[String](((c: Class[_]) => C.this.shortClass(c))).mkString(" with ")
    }

@som-snytt som-snytt force-pushed the issue/5073-rassoc-method-value branch from effa62a to 8dd8424 Compare May 20, 2021 07:36
@som-snytt som-snytt closed this Jun 5, 2023
@SethTisue SethTisue removed this from the 2.13.12 milestone Jun 5, 2023
@som-snytt som-snytt reopened this Nov 9, 2024
@scala-jenkins scala-jenkins added this to the 2.13.16 milestone Nov 9, 2024
@som-snytt som-snytt force-pushed the issue/5073-rassoc-method-value branch 3 times, most recently from b6afeeb to c646f90 Compare November 16, 2024 20:13
@som-snytt
Copy link
Contributor Author

som-snytt commented Nov 16, 2024

Reviewing previous comments, I see the initial effort in 2019 needed APPSELmode; then the follow-up in 2021 prompted lrytz to ask did you even read my PR description, the way a professor might say did you even read the paper that describes this fundamental concept; not sure what I was talking about in 2022-23.

Currently, there are a few old TODO comments I wouldn't mind addressing that are out of scope;

  • not sure the "arg" annotation is needed to detect that a view was applied (replacing the ApplyView with an attachment is one of the TODOs);
  • test that deprecated infix type arg is not broken.
  • The commit could be split into "trivial refactor or picking whitespace nits" and actual changes.
  • 30 LOC transformRassoc could be moved to an outer scope where it is less distracting to the reader.
  • run/t5073 is run/1980b.

Tip: You cannot create task list items within closed issues or issues with linked pull requests.

@som-snytt
Copy link
Contributor Author

Does anyone get the Merriam-Webster email? Today's word is "steadfast". Their quiz:

Rearrange the letters to form a verb meaning "to make steadfast": ZBIILEAST

@som-snytt som-snytt force-pushed the issue/5073-rassoc-method-value branch from c646f90 to 1261ad5 Compare November 17, 2024 20:05
Parser marks application as right-associative.
Typer identifies subexpressions to pre-compute
in order to preserve left-to-right evaluation.
@som-snytt som-snytt force-pushed the issue/5073-rassoc-method-value branch 2 times, most recently from 2883426 to 92bd67a Compare November 17, 2024 23:55
@som-snytt som-snytt force-pushed the issue/5073-rassoc-method-value branch from cb5f090 to 15b6afb Compare November 18, 2024 08:02
@som-snytt som-snytt marked this pull request as ready for review November 18, 2024 09:44
@som-snytt
Copy link
Contributor Author

Rebased and followed up on the items. Did not split the first commit. Did remove the two special Apply classes, as handling them specially is error-prone; using attachments is comfortable by now.

def isStab(x: Tree) = x match { case ValDef(_, nm, _, _) => nm.startsWith(nme.STABILIZER_PREFIX) case _ => false }
val (stabs, rassocs) = all.partition(isStab)
context.pendingStabilizers = Nil
Block(rassocs ++ stabs.reverse, expr).setPos(expr.pos).setType(expr.tpe)
Copy link
Member

@lrytz lrytz Nov 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can there be multiple in rassocs? reverse?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, and this is correctly ordered, as I'm sure a test demonstrates.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

def f = hashCode :: toString :: Nil

    def f: List[Any] = {
      final <synthetic> <artifact> val rassoc$2: Int = C.this.hashCode();
      final <synthetic> <artifact> val rassoc$1: String = C.this.toString();
      scala.`package`.Nil.::[String](rassoc$1).::[Any](rassoc$2)
    };

Comment on lines 6360 to 6361
case view @ Apply(coercion, coerced :: Nil) if view.hasAttachment[AppliedImplicitView.type] && shouldInsertRassocs =>
treeCopy.Apply(view, coercion, addStabilizers(coerced) :: Nil)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this piece made me wonder.

so the example is this

class L
object E extends L
class C(val h: Int, val t: L) extends L

class D(val l: () => L) {
  def #:: (o: Int): L = new C(o, l())
}

object T {
  implicit def toD(l: => L): D = new D(() => l)
}

a #:: b #:: E

When the outer implicit conversion is typed (adaptToMember, T.toD(T.toD(E).#::(rassoc$1)))

  • we end up in typedApply as expected. The argument T.toD(E).#::(rassoc$1) is typed without APPSELmode, that flag is removed in typedArg.
  • but the argument tree is already typed, and importantly, context.pendingStabilizers is already populated with rassoc$1
  • at the beginning of def typed
    • shouldInsertStabilizers is true, we save the pending one in a local and set context.pendingStabilizers = Nil
    • this means the stabilizer is not added by mistake. the logic around that in def typed doesn't work with trees that are already typed

we can probably fix that instead of the special case here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no emoji reaction with blurry eyes, but I'll think about it later.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try turning context.pendingStabilizers into a tree attachment.

Copy link
Member

@lrytz lrytz Nov 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a commit, but it's not replacing context.pendingStabilizers unfortunately. Maybe you have a better idea, I added some comments to try to explain what this is about.

@lrytz lrytz force-pushed the issue/5073-rassoc-method-value branch from 5839607 to 9240ec7 Compare November 19, 2024 15:20
@lrytz lrytz force-pushed the issue/5073-rassoc-method-value branch from 9240ec7 to 4bb452f Compare November 19, 2024 15:34
@lrytz
Copy link
Member

lrytz commented Nov 19, 2024

Bug found while trying stuff

object T extends App {
  def x = println("x")
  def y = println("y")
  object Nol {
    def ::(x: Any)(implicit o: Ordering[Int]): Nol.type = Nol
  }

  x :: y :: Nol
}

The rassoc transformation is not done (not in your version, and not with my commit on top).

@som-snytt
Copy link
Contributor Author

OK, I'm awake. Thanks for the code comments, which I read, it's useful to have a published paper that explains the theoretical basis.

I see why you didn't like the "special case" (I don't think you liked it two years ago). I'll have a second coffee before applying myself to review.

My first thought was that the new attachment is a different way to pronounce "tomato", but it's probably also a cleaner basis for fixing bugs in the general approach.

@lrytz
Copy link
Member

lrytz commented Nov 20, 2024

What would really help: a single test file for everything around stabilizers and rassocs, their combinations, interaction with implicit params and conversions (and other features?).

Scala 3 seems to get it all right, at least the few examples I tried. Can we align with the way it's implemented there? I didn't take a look.

@SethTisue SethTisue modified the milestones: 2.13.16, 2.13.17 Dec 12, 2024
@SethTisue SethTisue marked this pull request as draft February 28, 2025 02:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants