-
Notifications
You must be signed in to change notification settings - Fork 3.1k
Special treatment of emptyValDef in reify #3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
emptyValDef has special meaning in the compiler, so reify needs to preserve it by identity and not just by structure.
gkossakowski
referenced
this pull request
in gkossakowski/scala
Jan 11, 2012
Merge-in forward jumps fixes, turned on specialization and few other things.
axel22
referenced
this pull request
in axel22/scala-github
Feb 3, 2012
This change resolves some issues with ParCtrie splitters and their `remaining` method, which currently evaluates the size of the Ctrie. Since this is still not done lazily, nor in parallel, it has a certain cost, which is unacceptable. Change #1: The `shouldSplitFurther` method is by default implemented by calling the `remaining` method. This method now forwards the call to the same method in the splitter which is by default implemented in the same way as before, but can be overridden by custom collections such as the ParCtrie. Change #2: ParCtrie splitter now has a `level` member which just counts how many times the method has been split. This information is used to override the default `shouldSplitFurther` implementation. Change #3: The tasks and splitters rely heavily on the `remaining` method in the splitter for most operations. There is an additional method called `isRemainingCheap` which returns true by default, but can be overridden by custom collections such as the `Ctrie`.
paulp
added a commit
that referenced
this pull request
Feb 14, 2012
Now it copies in the current versions of BoxesRunTime and ScalaRunTime and applies patches to them, and the whole build is automated. # This is the only thing I actually typed, the rest is fancy echo. $ test/instrumented/mkinstrumented.sh build % rm -rf /scratch/trunk1/test/instrumented/classes % cp /scratch/trunk1/test/instrumented/../../src/library/scala/runtime/BoxesRunTime.java /scratch/trunk1/test/instrumented/../../src/library/scala/runtime/ScalaRunTime.scala /scratch/trunk1/test/instrumented/library/scala/runtime % patch BoxesRunTime.java /scratch/trunk1/test/instrumented/boxes.patch patching file BoxesRunTime.java % patch ScalaRunTime.scala /scratch/trunk1/test/instrumented/srt.patch patching file ScalaRunTime.scala Hunk #3 succeeded at 63 (offset 23 lines). Hunk #4 succeeded at 78 (offset 23 lines). Hunk #5 succeeded at 81 (offset 23 lines). Hunk #6 succeeded at 96 (offset 23 lines). % /scratch/trunk1/test/instrumented/../../build/pack/bin/scalac -d /scratch/trunk1/test/instrumented/classes /scratch/trunk1/test/instrumented/library/scala/runtime/BoxesRunTime.java /scratch/trunk1/test/instrumented/library/scala/runtime/ScalaRunTime.scala % javac -cp /scratch/trunk1/test/instrumented/../../build/pack/lib/scala-library.jar -d /scratch/trunk1/test/instrumented/classes /scratch/trunk1/test/instrumented/library/scala/runtime/BoxesRunTime.java % cd /scratch/trunk1/test/instrumented/classes % jar cf instrumented.jar . % mv -f instrumented.jar /scratch/trunk1/test/instrumented/../../test/files/speclib /scratch/trunk1/test/files/speclib/instrumented.jar has been created.
odersky
added a commit
to odersky/scala
that referenced
this pull request
Jul 14, 2012
Fixed fingerPrinting scheme to work with rehashes, also added finger prints to typedIdent searches.
adriaanm
referenced
this pull request
in adriaanm/scala
Jul 14, 2012
Fixed fingerPrinting scheme to work with rehashes, also added finger prints to typedIdent searches.
xeno-by
added a commit
that referenced
this pull request
Feb 27, 2013
First of all, GIL should only apply to runtime reflection, because noone is going to run toolboxes in multiple threads: a) that's impossible, b/c the compiler isn't thread safe, b) ToolBox api prevents that. Secondly, the only things in symbols which require synchronization are: 1) info/validTo (completers aren't thread-safe), 2) rawInfo and its dependencies (it shares a mutable field with info) 3) non-trivial caches like in typeAsMemberOfLock If you think about it, other things like sourceModule or associatedFile don't need synchronization, because they are either set up when a symbol is created or cloned or when it's completed. The former is obviously safe, while the latter is safe as well, because before acquiring init-dependent state of symbols, the compiler calls `initialize`, which is synchronized. We can say that symbols can be in four possible states: 1) being created, 2) created, but not yet initialized, 3) initializing, 4) initialized. in runtime reflection can undergo is init. #3 is dangerous and needs protection
xeno-by
added a commit
to xeno-by/scala
that referenced
this pull request
Jul 25, 2013
First of all, GIL should only apply to runtime reflection, because noone is going to run toolboxes in multiple threads: a) that's impossible, b/c the compiler isn't thread safe, b) ToolBox api prevents that. Secondly, the only things in symbols which require synchronization are: 1) info/validTo (completers aren't thread-safe), 2) rawInfo and its dependencies (it shares a mutable field with info) 3) non-trivial caches like in typeAsMemberOfLock If you think about it, other things like sourceModule or associatedFile don't need synchronization, because they are either set up when a symbol is created or cloned or when it's completed. The former is obviously safe, while the latter is safe as well, because before acquiring init-dependent state of symbols, the compiler calls `initialize`, which is synchronized. We can say that symbols can be in four possible states: 1) being created, 2) created, but not yet initialized, 3) initializing, 4) initialized. in runtime reflection can undergo is init. scala#3 is dangerous and needs protection
xeno-by
added a commit
to xeno-by/scala
that referenced
this pull request
Aug 6, 2013
First of all, GIL should only apply to runtime reflection, because noone is going to run toolboxes in multiple threads: a) that's impossible, b/c the compiler isn't thread safe, b) ToolBox api prevents that. Secondly, the only things in symbols which require synchronization are: 1) info/validTo (completers aren't thread-safe), 2) rawInfo and its dependencies (it shares a mutable field with info) 3) non-trivial caches like in typeAsMemberOfLock If you think about it, other things like sourceModule or associatedFile don't need synchronization, because they are either set up when a symbol is created or cloned or when it's completed. The former is obviously safe, while the latter is safe as well, because before acquiring init-dependent state of symbols, the compiler calls `initialize`, which is synchronized. We can say that symbols can be in four possible states: 1) being created, 2) created, but not yet initialized, 3) initializing, 4) initialized. in runtime reflection can undergo is init. scala#3 is dangerous and needs protection
xeno-by
added a commit
to xeno-by/scala
that referenced
this pull request
Aug 9, 2013
First of all, GIL should only apply to runtime reflection, because noone is going to run toolboxes in multiple threads: a) that's impossible, b/c the compiler isn't thread safe, b) ToolBox api prevents that. Secondly, the only things in symbols which require synchronization are: 1) info/validTo (completers aren't thread-safe), 2) rawInfo and its dependencies (it shares a mutable field with info) 3) non-trivial caches like in typeAsMemberOfLock If you think about it, other things like sourceModule or associatedFile don't need synchronization, because they are either set up when a symbol is created or cloned or when it's completed. The former is obviously safe, while the latter is safe as well, because before acquiring init-dependent state of symbols, the compiler calls `initialize`, which is synchronized. We can say that symbols can be in four possible states: 1) being created, 2) created, but not yet initialized, 3) initializing, 4) initialized. Of those only scala#3 is dangerous and needs protection, which is what this commit does.
xeno-by
added a commit
to xeno-by/scala
that referenced
this pull request
Oct 18, 2013
First of all, GIL should only apply to runtime reflection, because noone is going to run toolboxes in multiple threads: a) that's impossible, b/c the compiler isn't thread safe, b) ToolBox api prevents that. Secondly, the only things in symbols which require synchronization are: 1) info/validTo (completers aren't thread-safe), 2) rawInfo and its dependencies (it shares a mutable field with info) 3) non-trivial caches like in typeAsMemberOfLock If you think about it, other things like sourceModule or associatedFile don't need synchronization, because they are either set up when a symbol is created or cloned or when it's completed. The former is obviously safe, while the latter is safe as well, because before acquiring init-dependent state of symbols, the compiler calls `initialize`, which is synchronized. We can say that symbols can be in four possible states: 1) being created, 2) created, but not yet initialized, 3) initializing, 4) initialized. Of those only scala#3 is dangerous and needs protection, which is what this commit does.
xeno-by
added a commit
to xeno-by/scala
that referenced
this pull request
Nov 12, 2013
When an application of a blackbox macro is used as an implicit candidate, no expansion is performed until the macro is selected as the result of the implicit search. This makes it impossible to dynamically calculate availability of implicit macros.
xeno-by
added a commit
that referenced
this pull request
Nov 13, 2013
When an application of a blackbox macro is used as an implicit candidate, no expansion is performed until the macro is selected as the result of the implicit search. This makes it impossible to dynamically calculate availability of implicit macros.
retronym
added a commit
that referenced
this pull request
Dec 12, 2013
Introduce Unliftable for Quasiquotes (take #3)
retronym
added a commit
that referenced
this pull request
Feb 12, 2014
Swathes of important logic are duplicated between `findMember` and `findMembers` after they separated on grounds of irreconcilable differences about how fast they should run: d905558 Variation #10 to optimze findMember fcb0c01 Attempt #9 to opimize findMember. 71d2ceb Attempt #8 to opimize findMember. 77e5692 Attempty #7 to optimize findMember 275115e Fixing problem that caused fingerprints to fail in e94252e Attemmpt #6 to optimize findMember 73e61b8 Attempt #5 to optimize findMember. 04f0b65 Attempt #4 to optimize findMember 0e3c70f Attempt #3 to optimize findMember 41f4497 Attempt #2 to optimize findMember 1a73aa0 Attempt #1 to optimize findMember This didn't actually bear fruit, and the intervening years have seen the implementations drift. Now is the time to reunite them under the banner of `FindMemberBase`. Each has a separate subclass to customise the behaviour. This is primarily used by `findMember` to cache member types and to assemble the resulting list of symbols in an low-allocation manner. While there I have introduced some polymorphic calls, the call sites are only bi-morphic, and our typical pattern of compilation involves far more `findMember` calls, so I expect that JIT will keep the virtual call cost to an absolute minimum. Test results have been updated now that `findMembers` correctly excludes constructors and doesn't inherit privates. Coming up next: we can actually fix SI-7475!
This was referenced Mar 1, 2015
retronym
added a commit
that referenced
this pull request
Apr 10, 2015
Under `-Ydelambdafy:method`, a public, static accessor method is created to expose the private method containing the body of the lambda. Currently this accessor method has its parameters in the same order structure as those of the lambda body method. What is this order? There are three categories of parameters: 1. lambda parameters 2. captured parameters (added by lambdalift) 3. self parameters (added to lambda bodies that end up in trait impl classes by mixin, and added unconditionally to the static accessor method.) These are currently emitted in order #3, #1, #2. Here are examples of the current behaviour: BEFORE (trait): ``` % cat sandbox/test.scala && scalac-hash v2.11.5 -Ydelambdafy:method sandbox/test.scala && javap -private -classpath . 'Test$class' trait Member; class Capture; trait LambdaParam trait Test { def member: Member def foo { val local = new Capture (arg: LambdaParam) => "" + arg + member + local } } Compiled from "test.scala" public abstract class Test$class { public static void foo(Test); private static final java.lang.String $anonfun$1(Test, LambdaParam, Capture); public static void $init$(Test); public static final java.lang.String accessor$1(Test, LambdaParam, Capture); } ``` BEFORE (class): ``` % cat sandbox/test.scala && scalac-hash v2.11.5 -Ydelambdafy:method sandbox/test.scala && javap -private -classpath . Test trait Member; class Capture; trait LambdaParam abstract class Test { def member: Member def foo { val local = new Capture (arg: LambdaParam) => "" + arg + member + local } } Compiled from "test.scala" public abstract class Test { public abstract Member member(); public void foo(); private final java.lang.String $anonfun$1(LambdaParam, Capture); public Test(); public static final java.lang.String accessor$1(Test, LambdaParam, Capture); } ``` Contrasting the class case with Java: ``` % cat sandbox/Test.java && javac -d . sandbox/Test.java && javap -private -classpath . Test public abstract class Test { public static class Member {}; public static class Capture {}; public static class LambaParam {}; public static interface I { public abstract Object c(LambaParam arg); } public abstract Member member(); public void test() { Capture local = new Capture(); I i1 = (LambaParam arg) -> "" + member() + local; } } Compiled from "Test.java" public abstract class Test { public Test(); public abstract Test$Member member(); public void test(); private java.lang.Object lambda$test$0(Test$Capture, Test$LambaParam); } ``` We can see that in Java 8 lambda parameters come after captures. If we want to use Java's LambdaMetafactory to spin up our anoymous FunctionN subclasses on the fly, our ordering must change. I can see three options for change: 1. Adjust `LambdaLift` to always prepend captured parameters, rather than appending them. I think we could leave `Mixin` as it is, it already prepends the self parameter. This would result a parameter ordering, in terms of the list above: #3, #2, #1. 2. More conservatively, do this just for methods known to hold lambda bodies. This might avoid needlessly breaking code that has come to depend on our binary encoding. 3. Adjust the parameters of the accessor method only. The body of this method can permute params before calling the lambda body method. This commit implements option #2. In also prototyped #1, and found it worked so long as I limited it to non-constructors, to sidestep the need to make corresponding changes elsewhere in the compiler to avoid the crasher shown in the enclosed test case, which was minimized from a bootstrap failure from an earlier a version of this patch. We would need to defer option #1 to 2.12 in any case, as some of these lifted methods are publicied by the optimizer, and we must leave the signatures alone to comply with MiMa. I've included a test that shows this in all in action. However, that is currently disabled, as we don't have a partest category for tests that require Java 8.
retronym
added a commit
that referenced
this pull request
Aug 10, 2015
The log messages intented to chronicle implicit search were always being filtered out by virtue of the fact that the the tree passed to `printTyping` was already typed, (e.g. with an implicit MethodType.) This commit enabled printing in this case, although it still filters out trees that are deemed unfit for typer tracing, such as `()`. In the context of implicit search, this happens to filter out the noise of: ``` | | | [search #2] start `()`, searching for adaptation to pt=Unit => Foo[Int,Int] (silent: value <local Test> in Test) implicits disabled | | | [search #3] start `()`, searching for adaptation to pt=(=> Unit) => Foo[Int,Int] (silent: value <local Test> in Test) implicits disabled | | | \-> <error> ``` ... which I think is desirable. The motivation for this fix was to better display the interaction between implicit search and type inference. For instance: ``` class Foo[A, B] class Test { implicit val f: Foo[Int, String] = ??? def t[A, B](a: A)(implicit f: Foo[A, B]) = ??? t(1) } ``` ```` % scalac -Ytyper-debug sandbox/instantiate.scala ... | |-- t(1) BYVALmode-EXPRmode (site: value <local Test> in Test) | | |-- t BYVALmode-EXPRmode-FUNmode-POLYmode (silent: value <local Test> in Test) | | | [adapt] [A, B](a: A)(implicit f: Foo[A,B])Nothing adapted to [A, B](a: A)(implicit f: Foo[A,B])Nothing | | | \-> (a: A)(implicit f: Foo[A,B])Nothing | | |-- 1 BYVALmode-EXPRmode-POLYmode (site: value <local Test> in Test) | | | \-> Int(1) | | solving for (A: ?A, B: ?B) | | solving for (B: ?B) | | [search #1] start `[A, B](a: A)(implicit f: Foo[A,B])Nothing` inferring type B, searching for adaptation to pt=Foo[Int,B] (silent: value <local Test> in Test) implicits disabled | | [search #1] considering f | | [adapt] f adapted to => Foo[Int,String] based on pt Foo[Int,B] | | [search #1] solve tvars=?B, tvars.constr= >: String <: String | | solving for (B: ?B) | | [search #1] success inferred value of type Foo[Int,=?String] is SearchResult(Test.this.f, TreeTypeSubstituter(List(type B),List(String))) | | |-- [A, B](a: A)(implicit f: Foo[A,B])Nothing BYVALmode-EXPRmode (site: value <local Test> in Test) | | | \-> Nothing | | [adapt] [A, B](a: A)(implicit f: Foo[A,B])Nothing adapted to [A, B](a: A)(implicit f: Foo[A,B])Nothing | | \-> Nothing ```
losvald
pushed a commit
to losvald/scala
that referenced
this pull request
Nov 15, 2015
Fix spurious "member Nothing parent" errors via a script
retronym
added a commit
that referenced
this pull request
Oct 16, 2016
Manually tested with: ``` % cat sandbox/test.scala package p { object X { def f(i: Int) = ??? ; def f(s: String) = ??? } object Main { val res = X.f(3.14) } } % qscalac -Ytyper-debug sandbox/test.scala |-- p EXPRmode-POLYmode-QUALmode (site: package <root>) | \-> p.type |-- object X BYVALmode-EXPRmode (site: package p) | |-- super EXPRmode-POLYmode-QUALmode (silent: <init> in X) | | |-- this EXPRmode (silent: <init> in X) | | | \-> p.X.type | | \-> p.X.type | |-- def f BYVALmode-EXPRmode (site: object X) | | |-- $qmark$qmark$qmark EXPRmode (site: method f in X) | | | \-> Nothing | | |-- Int TYPEmode (site: value i in X) | | | \-> Int | | |-- Int TYPEmode (site: value i in X) | | | \-> Int | | \-> [def f] (i: Int)Nothing | |-- def f BYVALmode-EXPRmode (site: object X) | | |-- $qmark$qmark$qmark EXPRmode (site: method f in X) | | | \-> Nothing | | |-- String TYPEmode (site: value s in X) | | | [adapt] String is now a TypeTree(String) | | | \-> String | | |-- String TYPEmode (site: value s in X) | | | [adapt] String is now a TypeTree(String) | | | \-> String | | \-> [def f] (s: String)Nothing | \-> [object X] p.X.type |-- object Main BYVALmode-EXPRmode (site: package p) | |-- X.f(3.14) EXPRmode (site: value res in Main) | | |-- X.f BYVALmode-EXPRmode-FUNmode-POLYmode (silent: value res in Main) | | | |-- X EXPRmode-POLYmode-QUALmode (silent: value res in Main) | | | | \-> p.X.type | | | \-> (s: String)Nothing <and> (i: Int)Nothing | | |-- 3.14 BYVALmode-EXPRmode (silent: value res in Main) | | | \-> Double(3.14) | | [search #1] start `<?>`, searching for adaptation to pt=Double => String (silent: value res in Main) implicits disabled | | [search #2] start `<?>`, searching for adaptation to pt=(=> Double) => String (silent: value res in Main) implicits disabled | | [search #3] start `<?>`, searching for adaptation to pt=Double => Int (silent: value res in Main) implicits disabled | | 1 implicits in companion scope | | [search #4] start `<?>`, searching for adaptation to pt=(=> Double) => Int (silent: value res in Main) implicits disabled | | 1 implicits in companion scope | | second try: <error> and 3.14 | | [search #5] start `p.X.type`, searching for adaptation to pt=p.X.type => ?{def f(x$1: ? >: Double(3.14)): ?} (silent: value res in Main) implicits disabled | | [search #6] start `p.X.type`, searching for adaptation to pt=(=> p.X.type) => ?{def f(x$1: ? >: Double(3.14)): ?} (silent: value res in Main) implicits disabled sandbox/test.scala:4: error: overloaded method value f with alternatives: (s: String)Nothing <and> (i: Int)Nothing cannot be applied to (Double) val res = X.f(3.14) ^ ```
jvican
added a commit
to jvican/scala
that referenced
this pull request
Mar 24, 2017
The following commit message is a squash of several commit messages. - This is the 1st commit message: Add position to stub error messages Stub errors happen when we've started the initialization of a symbol but key information of this symbol is missing (the information cannot be found in any entry of the classpath not sources). When this error happens, we better have a good error message with a position to the place where the stub error came from. This commit goes into this direction by adding a `pos` value to `StubSymbol` and filling it in in all the use sites (especifically `UnPickler`). This commit also changes some tests that test stub errors-related issues. Concretely, `t6440` is using special Partest infrastructure and doens't pretty print the position, while `t5148` which uses the conventional infrastructure does. Hence the difference in the changes for both tests. - This is the commit message scala#2: Add partest infrastructure to test stub errors `StubErrorMessageTest` is the friend I introduce in this commit to help state stub errors. The strategy to test them is easy and builds upon previous concepts: we reuse `StoreReporterDirectTest` and add some methods that will compile the code and simulate a missing classpath entry by removing the class files from the class directory (the folder where Scalac compiles to). This first iteration allow us to programmatically check that stub errors are emitted under certain conditions. - This is the commit message scala#3: Improve contents of stub error message This commit does three things: * Keep track of completing symbol while unpickling First, it removes the previous `symbolOnCompletion` definition to be more restrictive/clear and use only positions, since only positions are used to report the error (the rest of the information comes from the context of the `UnPickler`). Second, it adds a new variable called `lazyCompletingSymbol` that is responsible for keeping a reference to the symbol that produces the stub error. This symbol will usually (always?) come from the classpath entries and therefore we don't have its position (that's why we keep track of `symbolOnCompletion` as well). This is the one that we have to explicitly use in the stub error message, the culprit so to speak. Aside from these two changes, this commit modifies the existing tests that are affected by the change in the error message, which is more precise now, and adds new tests for stub errors that happen in complex inner cases and in return type of `MethodType`. * Check that order of initialization is correct With the changes introduced previously to keep track of position of symbols coming from source files, we may ask ourselves: is this going to work always? What happens if two symbols the initialization of two symbols is intermingled and the stub error message gets the wrong position? This commit adds a test case and modifications to the test infrastructure to double check empirically that this does not happen. Usually, this interaction in symbol initialization won't happen because the `UnPickler` will lazily load all the buckets necessary for a symbol to be truly initialized, with the pertinent addresses from which this information has to be deserialized. This ensures that this operation is atomic and no other symbol initialization can happen in the meantime. Even though the previous paragraph is the feeling I got from reading the sources, this commit creates a test to double-check it. My attempt to be better safe than sorry. * Improve contents of the stub error message This commit modifies the format of the previous stub error message by being more precise in its formulation. It follows the structured format: ``` s"""|Symbol '${name.nameKind} ${owner.fullName}.$name' is missing from the classpath. |This symbol is required by '${lazyCompletingSymbol.kindString} ${lazyCompletingSymbol.fullName}'. ``` This format has the advantage that is more readable and explicit on what's happening. First, we report what is missing. Then, why it was required. Hopefully, people working on direct dependencies will find the new message friendlier. Having a good test suite to check the previously added code is important. This commit checks that stub errors happen in presence of well-known and widely used Scala features. These include: * Higher kinded types. * Type definitions. * Inheritance and subclasses. * Typeclasses and implicits. - This is the commit message scala#4: Use `lastTreeToTyper` to get better positions The previous strategy to get the last user-defined position for knowing what was the root cause (the trigger) of stub errors relied on instrumenting `def info`. This instrumentation, while easy to implement, is inefficient since we register the positions for symbols that are already completed. However, we cannot do it only for uncompleted symbols (!hasCompleteInfo) because the positions won't be correct anymore -- definitions using stub symbols (val b = new B) are for the compiler completed, but their use throws stub errors. This means that if we initialize symbols between a definition and its use, we'll use their positions instead of the position of `b`. To work around this we use `lastTreeToTyper`. We assume that stub errors will be thrown by Typer at soonest. The benefit of this approach is better error messages. The positions used in them are now as concrete as possible since they point to the exact tree that **uses** a symbol, instead of the one that **defines** it. Have a look at `StubErrorComplexInnerClass` for an example. This commit removes the previous infrastructure and replaces it by the new one. It also removes the fields positions from the subclasses of `StubSymbol`s. - This is the commit message scala#5: Keep track of completing symbols Make sure that cycles don't happen by keeping track of all the symbols that are being completed by `completeInternal`. Stub errors only need the last completing symbols, but the whole stack of symbols may be useful to reporting other error like cyclic initialization issues. I've added this per Jason's suggestion. I've implemented with a list because `remove` in an array buffer is linear. Array was not an option because I would need to resize it myself. I think that even though list is not as efficient memory-wise, it probably doesn't matter since the stack will usually be small. - This is the commit message scala#6: Remove `isPackage` from `newStubSymbol` Remove `isPackage` since in 2.12.x its value is not used.
jvican
added a commit
to jvican/scala
that referenced
this pull request
Mar 24, 2017
The following commit message is a squash of several commit messages. - This is the 1st commit message: Add position to stub error messages Stub errors happen when we've started the initialization of a symbol but key information of this symbol is missing (the information cannot be found in any entry of the classpath not sources). When this error happens, we better have a good error message with a position to the place where the stub error came from. This commit goes into this direction by adding a `pos` value to `StubSymbol` and filling it in in all the use sites (especifically `UnPickler`). This commit also changes some tests that test stub errors-related issues. Concretely, `t6440` is using special Partest infrastructure and doens't pretty print the position, while `t5148` which uses the conventional infrastructure does. Hence the difference in the changes for both tests. - This is the commit message scala#2: Add partest infrastructure to test stub errors `StubErrorMessageTest` is the friend I introduce in this commit to help state stub errors. The strategy to test them is easy and builds upon previous concepts: we reuse `StoreReporterDirectTest` and add some methods that will compile the code and simulate a missing classpath entry by removing the class files from the class directory (the folder where Scalac compiles to). This first iteration allow us to programmatically check that stub errors are emitted under certain conditions. - This is the commit message scala#3: Improve contents of stub error message This commit does three things: * Keep track of completing symbol while unpickling First, it removes the previous `symbolOnCompletion` definition to be more restrictive/clear and use only positions, since only positions are used to report the error (the rest of the information comes from the context of the `UnPickler`). Second, it adds a new variable called `lazyCompletingSymbol` that is responsible for keeping a reference to the symbol that produces the stub error. This symbol will usually (always?) come from the classpath entries and therefore we don't have its position (that's why we keep track of `symbolOnCompletion` as well). This is the one that we have to explicitly use in the stub error message, the culprit so to speak. Aside from these two changes, this commit modifies the existing tests that are affected by the change in the error message, which is more precise now, and adds new tests for stub errors that happen in complex inner cases and in return type of `MethodType`. * Check that order of initialization is correct With the changes introduced previously to keep track of position of symbols coming from source files, we may ask ourselves: is this going to work always? What happens if two symbols the initialization of two symbols is intermingled and the stub error message gets the wrong position? This commit adds a test case and modifications to the test infrastructure to double check empirically that this does not happen. Usually, this interaction in symbol initialization won't happen because the `UnPickler` will lazily load all the buckets necessary for a symbol to be truly initialized, with the pertinent addresses from which this information has to be deserialized. This ensures that this operation is atomic and no other symbol initialization can happen in the meantime. Even though the previous paragraph is the feeling I got from reading the sources, this commit creates a test to double-check it. My attempt to be better safe than sorry. * Improve contents of the stub error message This commit modifies the format of the previous stub error message by being more precise in its formulation. It follows the structured format: ``` s"""|Symbol '${name.nameKind} ${owner.fullName}.$name' is missing from the classpath. |This symbol is required by '${lazyCompletingSymbol.kindString} ${lazyCompletingSymbol.fullName}'. ``` This format has the advantage that is more readable and explicit on what's happening. First, we report what is missing. Then, why it was required. Hopefully, people working on direct dependencies will find the new message friendlier. Having a good test suite to check the previously added code is important. This commit checks that stub errors happen in presence of well-known and widely used Scala features. These include: * Higher kinded types. * Type definitions. * Inheritance and subclasses. * Typeclasses and implicits. - This is the commit message scala#4: Use `lastTreeToTyper` to get better positions The previous strategy to get the last user-defined position for knowing what was the root cause (the trigger) of stub errors relied on instrumenting `def info`. This instrumentation, while easy to implement, is inefficient since we register the positions for symbols that are already completed. However, we cannot do it only for uncompleted symbols (!hasCompleteInfo) because the positions won't be correct anymore -- definitions using stub symbols (val b = new B) are for the compiler completed, but their use throws stub errors. This means that if we initialize symbols between a definition and its use, we'll use their positions instead of the position of `b`. To work around this we use `lastTreeToTyper`. We assume that stub errors will be thrown by Typer at soonest. The benefit of this approach is better error messages. The positions used in them are now as concrete as possible since they point to the exact tree that **uses** a symbol, instead of the one that **defines** it. Have a look at `StubErrorComplexInnerClass` for an example. This commit removes the previous infrastructure and replaces it by the new one. It also removes the fields positions from the subclasses of `StubSymbol`s. - This is the commit message scala#5: Keep track of completing symbols Make sure that cycles don't happen by keeping track of all the symbols that are being completed by `completeInternal`. Stub errors only need the last completing symbols, but the whole stack of symbols may be useful to reporting other error like cyclic initialization issues. I've added this per Jason's suggestion. I've implemented with a list because `remove` in an array buffer is linear. Array was not an option because I would need to resize it myself. I think that even though list is not as efficient memory-wise, it probably doesn't matter since the stack will usually be small. - This is the commit message scala#6: Remove `isPackage` from `newStubSymbol` Remove `isPackage` since in 2.12.x its value is not used.
jvican
added a commit
to jvican/scala
that referenced
this pull request
Mar 24, 2017
The following commit message is a squash of several commit messages. - This is the 1st commit message: Add position to stub error messages Stub errors happen when we've started the initialization of a symbol but key information of this symbol is missing (the information cannot be found in any entry of the classpath not sources). When this error happens, we better have a good error message with a position to the place where the stub error came from. This commit goes into this direction by adding a `pos` value to `StubSymbol` and filling it in in all the use sites (especifically `UnPickler`). This commit also changes some tests that test stub errors-related issues. Concretely, `t6440` is using special Partest infrastructure and doens't pretty print the position, while `t5148` which uses the conventional infrastructure does. Hence the difference in the changes for both tests. - This is the commit message scala#2: Add partest infrastructure to test stub errors `StubErrorMessageTest` is the friend I introduce in this commit to help state stub errors. The strategy to test them is easy and builds upon previous concepts: we reuse `StoreReporterDirectTest` and add some methods that will compile the code and simulate a missing classpath entry by removing the class files from the class directory (the folder where Scalac compiles to). This first iteration allow us to programmatically check that stub errors are emitted under certain conditions. - This is the commit message scala#3: Improve contents of stub error message This commit does three things: * Keep track of completing symbol while unpickling First, it removes the previous `symbolOnCompletion` definition to be more restrictive/clear and use only positions, since only positions are used to report the error (the rest of the information comes from the context of the `UnPickler`). Second, it adds a new variable called `lazyCompletingSymbol` that is responsible for keeping a reference to the symbol that produces the stub error. This symbol will usually (always?) come from the classpath entries and therefore we don't have its position (that's why we keep track of `symbolOnCompletion` as well). This is the one that we have to explicitly use in the stub error message, the culprit so to speak. Aside from these two changes, this commit modifies the existing tests that are affected by the change in the error message, which is more precise now, and adds new tests for stub errors that happen in complex inner cases and in return type of `MethodType`. * Check that order of initialization is correct With the changes introduced previously to keep track of position of symbols coming from source files, we may ask ourselves: is this going to work always? What happens if two symbols the initialization of two symbols is intermingled and the stub error message gets the wrong position? This commit adds a test case and modifications to the test infrastructure to double check empirically that this does not happen. Usually, this interaction in symbol initialization won't happen because the `UnPickler` will lazily load all the buckets necessary for a symbol to be truly initialized, with the pertinent addresses from which this information has to be deserialized. This ensures that this operation is atomic and no other symbol initialization can happen in the meantime. Even though the previous paragraph is the feeling I got from reading the sources, this commit creates a test to double-check it. My attempt to be better safe than sorry. * Improve contents of the stub error message This commit modifies the format of the previous stub error message by being more precise in its formulation. It follows the structured format: ``` s"""|Symbol '${name.nameKind} ${owner.fullName}.$name' is missing from the classpath. |This symbol is required by '${lazyCompletingSymbol.kindString} ${lazyCompletingSymbol.fullName}'. ``` This format has the advantage that is more readable and explicit on what's happening. First, we report what is missing. Then, why it was required. Hopefully, people working on direct dependencies will find the new message friendlier. Having a good test suite to check the previously added code is important. This commit checks that stub errors happen in presence of well-known and widely used Scala features. These include: * Higher kinded types. * Type definitions. * Inheritance and subclasses. * Typeclasses and implicits. - This is the commit message scala#4: Use `lastTreeToTyper` to get better positions The previous strategy to get the last user-defined position for knowing what was the root cause (the trigger) of stub errors relied on instrumenting `def info`. This instrumentation, while easy to implement, is inefficient since we register the positions for symbols that are already completed. However, we cannot do it only for uncompleted symbols (!hasCompleteInfo) because the positions won't be correct anymore -- definitions using stub symbols (val b = new B) are for the compiler completed, but their use throws stub errors. This means that if we initialize symbols between a definition and its use, we'll use their positions instead of the position of `b`. To work around this we use `lastTreeToTyper`. We assume that stub errors will be thrown by Typer at soonest. The benefit of this approach is better error messages. The positions used in them are now as concrete as possible since they point to the exact tree that **uses** a symbol, instead of the one that **defines** it. Have a look at `StubErrorComplexInnerClass` for an example. This commit removes the previous infrastructure and replaces it by the new one. It also removes the fields positions from the subclasses of `StubSymbol`s. - This is the commit message scala#5: Keep track of completing symbols Make sure that cycles don't happen by keeping track of all the symbols that are being completed by `completeInternal`. Stub errors only need the last completing symbols, but the whole stack of symbols may be useful to reporting other error like cyclic initialization issues. I've added this per Jason's suggestion. I've implemented with a list because `remove` in an array buffer is linear. Array was not an option because I would need to resize it myself. I think that even though list is not as efficient memory-wise, it probably doesn't matter since the stack will usually be small. - This is the commit message scala#6: Remove `isPackage` from `newStubSymbol` Remove `isPackage` since in 2.12.x its value is not used.
jvican
added a commit
to jvican/scala
that referenced
this pull request
Mar 24, 2017
The following commit message is a squash of several commit messages. - This is the 1st commit message: Add position to stub error messages Stub errors happen when we've started the initialization of a symbol but key information of this symbol is missing (the information cannot be found in any entry of the classpath not sources). When this error happens, we better have a good error message with a position to the place where the stub error came from. This commit goes into this direction by adding a `pos` value to `StubSymbol` and filling it in in all the use sites (especifically `UnPickler`). This commit also changes some tests that test stub errors-related issues. Concretely, `t6440` is using special Partest infrastructure and doens't pretty print the position, while `t5148` which uses the conventional infrastructure does. Hence the difference in the changes for both tests. - This is the commit message scala#2: Add partest infrastructure to test stub errors `StubErrorMessageTest` is the friend I introduce in this commit to help state stub errors. The strategy to test them is easy and builds upon previous concepts: we reuse `StoreReporterDirectTest` and add some methods that will compile the code and simulate a missing classpath entry by removing the class files from the class directory (the folder where Scalac compiles to). This first iteration allow us to programmatically check that stub errors are emitted under certain conditions. - This is the commit message scala#3: Improve contents of stub error message This commit does three things: * Keep track of completing symbol while unpickling First, it removes the previous `symbolOnCompletion` definition to be more restrictive/clear and use only positions, since only positions are used to report the error (the rest of the information comes from the context of the `UnPickler`). Second, it adds a new variable called `lazyCompletingSymbol` that is responsible for keeping a reference to the symbol that produces the stub error. This symbol will usually (always?) come from the classpath entries and therefore we don't have its position (that's why we keep track of `symbolOnCompletion` as well). This is the one that we have to explicitly use in the stub error message, the culprit so to speak. Aside from these two changes, this commit modifies the existing tests that are affected by the change in the error message, which is more precise now, and adds new tests for stub errors that happen in complex inner cases and in return type of `MethodType`. * Check that order of initialization is correct With the changes introduced previously to keep track of position of symbols coming from source files, we may ask ourselves: is this going to work always? What happens if two symbols the initialization of two symbols is intermingled and the stub error message gets the wrong position? This commit adds a test case and modifications to the test infrastructure to double check empirically that this does not happen. Usually, this interaction in symbol initialization won't happen because the `UnPickler` will lazily load all the buckets necessary for a symbol to be truly initialized, with the pertinent addresses from which this information has to be deserialized. This ensures that this operation is atomic and no other symbol initialization can happen in the meantime. Even though the previous paragraph is the feeling I got from reading the sources, this commit creates a test to double-check it. My attempt to be better safe than sorry. * Improve contents of the stub error message This commit modifies the format of the previous stub error message by being more precise in its formulation. It follows the structured format: ``` s"""|Symbol '${name.nameKind} ${owner.fullName}.$name' is missing from the classpath. |This symbol is required by '${lazyCompletingSymbol.kindString} ${lazyCompletingSymbol.fullName}'. ``` This format has the advantage that is more readable and explicit on what's happening. First, we report what is missing. Then, why it was required. Hopefully, people working on direct dependencies will find the new message friendlier. Having a good test suite to check the previously added code is important. This commit checks that stub errors happen in presence of well-known and widely used Scala features. These include: * Higher kinded types. * Type definitions. * Inheritance and subclasses. * Typeclasses and implicits. - This is the commit message scala#4: Use `lastTreeToTyper` to get better positions The previous strategy to get the last user-defined position for knowing what was the root cause (the trigger) of stub errors relied on instrumenting `def info`. This instrumentation, while easy to implement, is inefficient since we register the positions for symbols that are already completed. However, we cannot do it only for uncompleted symbols (!hasCompleteInfo) because the positions won't be correct anymore -- definitions using stub symbols (val b = new B) are for the compiler completed, but their use throws stub errors. This means that if we initialize symbols between a definition and its use, we'll use their positions instead of the position of `b`. To work around this we use `lastTreeToTyper`. We assume that stub errors will be thrown by Typer at soonest. The benefit of this approach is better error messages. The positions used in them are now as concrete as possible since they point to the exact tree that **uses** a symbol, instead of the one that **defines** it. Have a look at `StubErrorComplexInnerClass` for an example. This commit removes the previous infrastructure and replaces it by the new one. It also removes the fields positions from the subclasses of `StubSymbol`s. - This is the commit message scala#5: Keep track of completing symbols Make sure that cycles don't happen by keeping track of all the symbols that are being completed by `completeInternal`. Stub errors only need the last completing symbols, but the whole stack of symbols may be useful to reporting other error like cyclic initialization issues. I've added this per Jason's suggestion. I've implemented with a list because `remove` in an array buffer is linear. Array was not an option because I would need to resize it myself. I think that even though list is not as efficient memory-wise, it probably doesn't matter since the stack will usually be small. - This is the commit message scala#6: Remove `isPackage` from `newStubSymbol` Remove `isPackage` since in 2.12.x its value is not used.
jvican
added a commit
to jvican/scala
that referenced
this pull request
Mar 24, 2017
The following commit message is a squash of several commit messages. - This is the 1st commit message: Add position to stub error messages Stub errors happen when we've started the initialization of a symbol but key information of this symbol is missing (the information cannot be found in any entry of the classpath not sources). When this error happens, we better have a good error message with a position to the place where the stub error came from. This commit goes into this direction by adding a `pos` value to `StubSymbol` and filling it in in all the use sites (especifically `UnPickler`). This commit also changes some tests that test stub errors-related issues. Concretely, `t6440` is using special Partest infrastructure and doens't pretty print the position, while `t5148` which uses the conventional infrastructure does. Hence the difference in the changes for both tests. - This is the commit message scala#2: Add partest infrastructure to test stub errors `StubErrorMessageTest` is the friend I introduce in this commit to help state stub errors. The strategy to test them is easy and builds upon previous concepts: we reuse `StoreReporterDirectTest` and add some methods that will compile the code and simulate a missing classpath entry by removing the class files from the class directory (the folder where Scalac compiles to). This first iteration allow us to programmatically check that stub errors are emitted under certain conditions. - This is the commit message scala#3: Improve contents of stub error message This commit does three things: * Keep track of completing symbol while unpickling First, it removes the previous `symbolOnCompletion` definition to be more restrictive/clear and use only positions, since only positions are used to report the error (the rest of the information comes from the context of the `UnPickler`). Second, it adds a new variable called `lazyCompletingSymbol` that is responsible for keeping a reference to the symbol that produces the stub error. This symbol will usually (always?) come from the classpath entries and therefore we don't have its position (that's why we keep track of `symbolOnCompletion` as well). This is the one that we have to explicitly use in the stub error message, the culprit so to speak. Aside from these two changes, this commit modifies the existing tests that are affected by the change in the error message, which is more precise now, and adds new tests for stub errors that happen in complex inner cases and in return type of `MethodType`. * Check that order of initialization is correct With the changes introduced previously to keep track of position of symbols coming from source files, we may ask ourselves: is this going to work always? What happens if two symbols the initialization of two symbols is intermingled and the stub error message gets the wrong position? This commit adds a test case and modifications to the test infrastructure to double check empirically that this does not happen. Usually, this interaction in symbol initialization won't happen because the `UnPickler` will lazily load all the buckets necessary for a symbol to be truly initialized, with the pertinent addresses from which this information has to be deserialized. This ensures that this operation is atomic and no other symbol initialization can happen in the meantime. Even though the previous paragraph is the feeling I got from reading the sources, this commit creates a test to double-check it. My attempt to be better safe than sorry. * Improve contents of the stub error message This commit modifies the format of the previous stub error message by being more precise in its formulation. It follows the structured format: ``` s"""|Symbol '${name.nameKind} ${owner.fullName}.$name' is missing from the classpath. |This symbol is required by '${lazyCompletingSymbol.kindString} ${lazyCompletingSymbol.fullName}'. ``` This format has the advantage that is more readable and explicit on what's happening. First, we report what is missing. Then, why it was required. Hopefully, people working on direct dependencies will find the new message friendlier. Having a good test suite to check the previously added code is important. This commit checks that stub errors happen in presence of well-known and widely used Scala features. These include: * Higher kinded types. * Type definitions. * Inheritance and subclasses. * Typeclasses and implicits. - This is the commit message scala#4: Use `lastTreeToTyper` to get better positions The previous strategy to get the last user-defined position for knowing what was the root cause (the trigger) of stub errors relied on instrumenting `def info`. This instrumentation, while easy to implement, is inefficient since we register the positions for symbols that are already completed. However, we cannot do it only for uncompleted symbols (!hasCompleteInfo) because the positions won't be correct anymore -- definitions using stub symbols (val b = new B) are for the compiler completed, but their use throws stub errors. This means that if we initialize symbols between a definition and its use, we'll use their positions instead of the position of `b`. To work around this we use `lastTreeToTyper`. We assume that stub errors will be thrown by Typer at soonest. The benefit of this approach is better error messages. The positions used in them are now as concrete as possible since they point to the exact tree that **uses** a symbol, instead of the one that **defines** it. Have a look at `StubErrorComplexInnerClass` for an example. This commit removes the previous infrastructure and replaces it by the new one. It also removes the fields positions from the subclasses of `StubSymbol`s. - This is the commit message scala#5: Keep track of completing symbols Make sure that cycles don't happen by keeping track of all the symbols that are being completed by `completeInternal`. Stub errors only need the last completing symbols, but the whole stack of symbols may be useful to reporting other error like cyclic initialization issues. I've added this per Jason's suggestion. I've implemented with a list because `remove` in an array buffer is linear. Array was not an option because I would need to resize it myself. I think that even though list is not as efficient memory-wise, it probably doesn't matter since the stack will usually be small. - This is the commit message scala#6: Remove `isPackage` from `newStubSymbol` Remove `isPackage` since in 2.12.x its value is not used.
da-liii
pushed a commit
to da-liii/scala
that referenced
this pull request
Nov 11, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
emptyValDef has special meaning in the compiler, so reify needs to preserve it by identity and not just by structure. Review by @xeno-by