testWidgets causes incorrect file/line/column for tests (returns location of the test() call inside flutter_test) #18195

DanTup · 2018-06-05T09:44:35Z

The test framework returns the location for a test so that an editor can jump to it when the user clicks on it. For tests that use testWidgets it returns the location of the test call inside testWidgets so it won't take the user to where they expect.

Example:

{
	"test": {
		"id": 3,
		"name": "Changing locale",
		"suiteID": 0,
		"groupIDs": [
			2
		],
		"line": 60,
		"column": 18,
		"url": "package:flutter_test/src/widget_tester.dart"
	},
	"type": "testStart",
	"time": 21952
}

I need to use this information to navigate to the test when a user clicks in the tree on the left. Currently I ignore any package: URIs so it just does nothing, but it makes the test runner much less useful.

The text was updated successfully, but these errors were encountered:

DanTup · 2018-06-06T16:47:32Z

@jakemac53 @natebosch Is this something that'd be better in the test repo? I'm not sure where it can be fixed (but I do think we need to do something).

jakemac53 · 2018-06-06T16:52:45Z

I wonder if we could look at the stack and report the last line that came from the original test file?

DanTup · 2018-06-06T16:58:56Z

Sounds reasonable to me (though probably it's the first frame that matches that you want, if you're starting at the top). It'd still fall down if the user had their own wrapper over test that happened to be in the same file, but I think it'd catch the huge majority of cases.

I just tested in Android Studio and seems that suffers because of this too - you can only jump to a file from the suite node, you can't navigate from a test. Surprised it hasn't come up before!

jakemac53 · 2018-06-06T21:35:39Z

Can you test out with this branch https://github.com/dart-lang/test/tree/report-original-test-location?

This adds additional fields, root_line, root_column, and root_url to the json reporter.

If this covers your use case it shouldn't be to much work to add some tests and land it, although I haven't thought through the implications of adding fields to this protocol to much.

@natebosch wdyt?

natebosch · 2018-06-07T00:17:50Z

Seems sensible to me.

https://github.com/dart-lang/test/blob/report-original-test-location/doc/json_reporter.md is the docs for the format, the assumption is that it is safe to add new fields.

DanTup · 2018-06-07T08:39:46Z

I'm struggling to get Flutter to use this version... I've set path: in my pubspec for both my checked out flutter_test and in my stocks example app, and also made it use the local path for flutter_test. The .packages files all updated and show paths to the checked out version; yet I'm not seeing this (I even changed the protocolVersion in the local copy to check, and it's not getting the updated version).

I'll test without Flutter (just putting a helper in another file), but it won't be quite as accurate a test.

DanTup · 2018-06-07T08:44:56Z

In a non-Flutter test, this looks perfect :-)

{
"line": 4,
"column": 3,
"url": "file:///Users/dantup/Dev/Dart-Code/test/test_projects/hello_world/test/helpers.dart",
"root_line": 6,
"root_column": 3,
"root_package_url": "file:///Users/dantup/Dev/Dart-Code/test/test_projects/hello_world/test/main.dart"
}

jakemac53 · 2018-06-07T13:41:17Z

In a non-Flutter test, this looks perfect :-)

Flutter does have their own test platform, and they might have forked the json reporter as well - so that probably has to be updated in a similar fashion, but it should be trivial.

DanTup · 2018-06-07T13:48:46Z

flutter_test has a dependency on test and a comment about pinning to a specific version, so I assumed it used it; but there is the possibility it's doing some stuff (like the reporter) itself. That might also explain why we can debug tests using the VM using the json reporter in Flutter but not the VM. In any case, once it's done here we can patch the Flutter one to be the same, I can look into that.

DanTup · 2018-06-07T15:52:47Z

From discussions with Devon, I don't think it has its own reporter. So if this lands here I'll try revving the version in Flutter locally and re-test (and if no good, I'll track down what needs fixing in Flutter).

jakemac53 · 2018-06-07T21:03:21Z

landed in 0.12.40

DanTup · 2018-06-18T16:04:05Z

Flutter has been updated and this all works great now 👍

7e205b3 Roll src/fuchsia/sdk/mac from 1MVsE... to 4MCVP... (flutter#18217) 9193d8f Roll src/third_party/skia 0dc207f836da..a14084ba1b41 (3 commits) (flutter#18219) 403931f Add FlValue (flutter#18185) 46a6d96 [SkParagraph] Copy text height behavior to the Skia paragraph style (flutter#18178) e7ee47d [web] Implement matrix parameter for linear gradient (flutter#18208) 28d9985 Reland again "Remove layer integral offset snapping flutter#17112" (flutter#18160) 5e361f5 Roll src/third_party/skia c66cd987f7c0..0dc207f836da (5 commits) (flutter#18212) 716dbf0 Refactor GLFW embedding to support headless mode (flutter#18205) 23cca32 Manual Roll of Dart 39e0e75fcf...ce62ad2e8b (flutter#18211) 15f72b8 Support EventChannel C++ plugin API for Linux/Windows (flutter#17015) e5a7ca5 Handle leak of message handle when no engine present (flutter#18157) d3bde19 add docs to platformviewios (and some drive-by changes) (flutter#17593) b582d77 Roll src/third_party/skia 0066adefa97d..c66cd987f7c0 (4 commits) (flutter#18206) 5e7d6d0 Roll src/third_party/skia edea19858ccc..0066adefa97d (3 commits) (flutter#18203) 88b9d42 Remove the global engine entry timestamp (flutter#18182) 0dffdd5 Roll src/third_party/dart e86e4d61834a..ce62ad2e8b40 (13 commits) (flutter#18201) 56fe6d2 Roll src/third_party/skia 2871ab0727bf..edea19858ccc (3 commits) (flutter#18198) bcdf991 Fixed ChildSceneLayer elevation issue on Fuchsia. (flutter#18144) ede658e [profiling] CPU Profiling support for iOS (flutter#18087) d043923 Roll src/third_party/skia e3d1de7c5281..2871ab0727bf (1 commits) (flutter#18197) 807f191 Roll src/third_party/dart 733153eb517c..e86e4d61834a (6 commits) (flutter#18195) fe3f8df Roll src/third_party/skia 3b2db26c59d6..e3d1de7c5281 (4 commits) (flutter#18192) 34b0629 Roll fuchsia/sdk/core/mac-amd64 from jMJqf... to 1MVsE... (flutter#18194) e9b2afe Roll fuchsia/sdk/core/linux-amd64 from RpHTv... to MhpFP... (flutter#18191) 35fa006 Roll src/third_party/skia 88d04cb51acf..3b2db26c59d6 (1 commits) (flutter#18190) e54bb9e Roll src/third_party/dart 4da5b40fb6dc..733153eb517c (23 commits) (flutter#18188)

7e205b3 Roll src/fuchsia/sdk/mac from 1MVsE... to 4MCVP... (#18217) 9193d8f Roll src/third_party/skia 0dc207f836da..a14084ba1b41 (3 commits) (#18219) 403931f Add FlValue (#18185) 46a6d96 [SkParagraph] Copy text height behavior to the Skia paragraph style (#18178) e7ee47d [web] Implement matrix parameter for linear gradient (#18208) 28d9985 Reland again "Remove layer integral offset snapping #17112" (#18160) 5e361f5 Roll src/third_party/skia c66cd987f7c0..0dc207f836da (5 commits) (#18212) 716dbf0 Refactor GLFW embedding to support headless mode (#18205) 23cca32 Manual Roll of Dart 39e0e75fcf...ce62ad2e8b (#18211) 15f72b8 Support EventChannel C++ plugin API for Linux/Windows (#17015) e5a7ca5 Handle leak of message handle when no engine present (#18157) d3bde19 add docs to platformviewios (and some drive-by changes) (#17593) b582d77 Roll src/third_party/skia 0066adefa97d..c66cd987f7c0 (4 commits) (#18206) 5e7d6d0 Roll src/third_party/skia edea19858ccc..0066adefa97d (3 commits) (#18203) 88b9d42 Remove the global engine entry timestamp (#18182) 0dffdd5 Roll src/third_party/dart e86e4d61834a..ce62ad2e8b40 (13 commits) (#18201) 56fe6d2 Roll src/third_party/skia 2871ab0727bf..edea19858ccc (3 commits) (#18198) bcdf991 Fixed ChildSceneLayer elevation issue on Fuchsia. (#18144) ede658e [profiling] CPU Profiling support for iOS (#18087) d043923 Roll src/third_party/skia e3d1de7c5281..2871ab0727bf (1 commits) (#18197) 807f191 Roll src/third_party/dart 733153eb517c..e86e4d61834a (6 commits) (#18195) fe3f8df Roll src/third_party/skia 3b2db26c59d6..e3d1de7c5281 (4 commits) (#18192) 34b0629 Roll fuchsia/sdk/core/mac-amd64 from jMJqf... to 1MVsE... (#18194) e9b2afe Roll fuchsia/sdk/core/linux-amd64 from RpHTv... to MhpFP... (#18191) 35fa006 Roll src/third_party/skia 88d04cb51acf..3b2db26c59d6 (1 commits) (#18190) e54bb9e Roll src/third_party/dart 4da5b40fb6dc..733153eb517c (23 commits) (#18188)

github-actions · 2021-09-03T00:01:34Z

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

DanTup mentioned this issue Jun 5, 2018

Display Flutter test results in a tree Dart-Code/Dart-Code#980

Closed

5 tasks

jakemac53 mentioned this issue Jun 7, 2018

Report original test location in the json_reporter if different dart-lang/test#853

Merged

DanTup mentioned this issue Jun 11, 2018

Upgrade all packages with "flutter update-packages --force-upgrade" #18353

Closed

DanTup closed this as completed Jun 18, 2018

a-siva mentioned this issue May 8, 2020

Manual roll of engine 7e205b37e5...3953c3ccd1 #56631

Closed

a-siva mentioned this issue May 8, 2020

Manual roll of engine 7e205b37e5...3953c3ccd1 #56677

Merged

github-actions bot locked as resolved and limited conversation to collaborators Sep 3, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

testWidgets causes incorrect file/line/column for tests (returns location of the test() call inside flutter_test) #18195

testWidgets causes incorrect file/line/column for tests (returns location of the test() call inside flutter_test) #18195

DanTup commented Jun 5, 2018

DanTup commented Jun 6, 2018

Uh oh!

jakemac53 commented Jun 6, 2018

Uh oh!

DanTup commented Jun 6, 2018

Uh oh!

jakemac53 commented Jun 6, 2018

Uh oh!

natebosch commented Jun 7, 2018

Uh oh!

DanTup commented Jun 7, 2018

Uh oh!

DanTup commented Jun 7, 2018

Uh oh!

jakemac53 commented Jun 7, 2018

Uh oh!

DanTup commented Jun 7, 2018

Uh oh!

DanTup commented Jun 7, 2018

Uh oh!

jakemac53 commented Jun 7, 2018

Uh oh!

DanTup commented Jun 18, 2018

Uh oh!

github-actions bot commented Sep 3, 2021

Uh oh!

testWidgets causes incorrect file/line/column for tests (returns location of the test() call inside flutter_test) #18195

testWidgets causes incorrect file/line/column for tests (returns location of the test() call inside flutter_test) #18195

Comments

DanTup commented Jun 5, 2018

DanTup commented Jun 6, 2018

Uh oh!

jakemac53 commented Jun 6, 2018

Uh oh!

DanTup commented Jun 6, 2018

Uh oh!

jakemac53 commented Jun 6, 2018

Uh oh!

natebosch commented Jun 7, 2018

Uh oh!

DanTup commented Jun 7, 2018

Uh oh!

DanTup commented Jun 7, 2018

Uh oh!

jakemac53 commented Jun 7, 2018

Uh oh!

DanTup commented Jun 7, 2018

Uh oh!

DanTup commented Jun 7, 2018

Uh oh!

jakemac53 commented Jun 7, 2018

Uh oh!

DanTup commented Jun 18, 2018

Uh oh!

github-actions bot commented Sep 3, 2021

Uh oh!