Skip to content

Proposal: Move towards a more flexible, low level compiler testing system like that of Go. #284

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dmitshur opened this issue Aug 29, 2015 · 4 comments
Labels

Comments

@dmitshur
Copy link
Member

Motivation

When implementing tests for #279, I ran into a difficulty in that I needed my test program to import "time" package but not have any references to time.Time so that it would get DCEed. Normally, it's easy to do:

package main

import "time"

func main() {
    time.Sleep(time.Second)
}

However, when I tried to make it a test, that requires importing "testing" package. But the problem is that package already imports "time" and references time.Time internally, causing it not get DCEed.

Proposal

Let's move towards using a more flexible, low level testing system like is used in internal Go compiler tests (https://github.com/golang/go/tree/master/test folder) for compiler tests.

Using "testing" package is a very high level, large tool. It's great for package tests. But not if you want to test various compiler-level aspects. For example, you might want to test that something fails to compile, or causes a panic, etc.

Plan

If accepted, my implementation plan is to convert some of the existing Go compiler tests to run under GopherJS compiler. I was successful in getting 230~ passes and 40 fails. I can triage and skip the failing tests, and add the passing ones to GopherJS test suite.

That naturally paves the way to reuse that existing system for additional GopherJS specific compiler tests. That system has various actions like "rundircmpout", "compile", "compiledir", "build", "runoutput", "rundir", "errorcheck", "errorcheckdir", "errorcheckoutput", "run", "cmpout", "skip" which make it flexible and easy to add specific tests.

Closing Remarks

This is not a high priority and doesn't need to be done urgently, rather I propose we naturally move in this direction.

@dmitshur
Copy link
Member Author

dmitshur commented Sep 7, 2015

PR #296 is the first step in this direction.

@neelance
Copy link
Member

#296 was a really good step in this direction. Thanks for that. Do you think we should keep this issue open and add a checklist for further steps or shall we close it and first triage the failures you spotted and then go from there?

@dmitshur
Copy link
Member Author

I'm okay with marking this proposal as "accepted" and closing it perhaps (or how else to deal with accepted proposals?). It could also be kept open until it's done, but that may mean it'll be open for a long time, since this will be a gradual process.

My plan is to port more of the Go test-suite when the list of known unresolved failures starts getting closer to 0. It is also my plan to use this new way to run low level compiler tests if I need to add future compiler tests.

@dmitshur
Copy link
Member Author

This should also be included for #331.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants