|
1 | 1 | # Flutter devicelab |
2 | 2 |
|
3 | | -This package contains the test framework and tests that run on physical devices. |
4 | | -More generally the tests are referred to as "tasks" in the API, but since we |
5 | | -primarily use it for testing, this document refers to them as "tests". |
| 3 | +"Devicelab" (a.k.a. "cocoon") is a physical lab that tests Flutter on real |
| 4 | +Android and iOS devices. |
| 5 | + |
| 6 | +This package contains the code for test framework and the tests. More generally |
| 7 | +the tests are referred to as "tasks" in the API, but since we primarily use it |
| 8 | +for testing, this document refers to them as "tests". |
| 9 | + |
| 10 | +You can see the continuous build results from the master branch at |
| 11 | +http://go/flutter-dashboard/build.html. |
| 12 | + |
| 13 | +# Running tests locally |
| 14 | + |
| 15 | +Do make sure your tests pass locally before deploying to the CI environment. |
| 16 | +Below is a handful of commands that run tests in a similar way to how the |
| 17 | +CI environment runs them. These commands are also useful when you need to |
| 18 | +reproduce a CI test failure locally. |
| 19 | + |
| 20 | +To run a test, use option `-t` (`--task`): |
| 21 | + |
| 22 | +```sh |
| 23 | +dart bin/run.dart -t {NAME_OF_TEST} |
| 24 | +``` |
| 25 | + |
| 26 | +To run multiple tests, repeat option `-t` (`--task`) multiple times: |
| 27 | + |
| 28 | +```sh |
| 29 | +dart bin/run.dart -t test1 -t test2 -t test3 |
| 30 | +``` |
| 31 | + |
| 32 | +To run all tests defined in `manifest.yaml`, use option `-a` (`--all`): |
| 33 | + |
| 34 | +```sh |
| 35 | +dart bin/run.dart -a |
| 36 | +``` |
| 37 | + |
| 38 | +To run tests from a specific stage, use option `-s` (`--stage`): |
| 39 | + |
| 40 | +```sh |
| 41 | +dart bin/run.dart -s {NAME_OF_STAGE} |
| 42 | +``` |
| 43 | + |
| 44 | +# Reproducing broken builds locally |
| 45 | + |
| 46 | +If a commit caused a test to fail, |
| 47 | +[the dashboard](http://go/flutter-dashboard/build.html) might look something |
| 48 | +like this: |
| 49 | + |
| 50 | + |
| 51 | + |
| 52 | +The red circle tells you that a test failed. The number inside tells you how |
| 53 | +many times the devicelab attempted to run the test before giving up on it. |
| 54 | + |
| 55 | +To reproduce the breakage locally `git checkout` the corresponding Flutter |
| 56 | +revision. Note the name of the test that failed. In the example above the |
| 57 | +failing test is `flutter_gallery__transition_perf`. This name can be passed to |
| 58 | +the `run.dart` command. For example: |
| 59 | + |
| 60 | +```sh |
| 61 | +dart bin/run.dart -t flutter_gallery__transition_perf |
| 62 | +``` |
6 | 63 |
|
7 | 64 | # Writing tests |
8 | 65 |
|
@@ -63,34 +120,3 @@ Where: |
63 | 120 | - `{CAPABILITIES}` is an array that lists the capabilities required of |
64 | 121 | the test agent (the computer that runs the test) to run your test. Available |
65 | 122 | capabilities are: `has-android-device`, `has-ios-device`. |
66 | | - |
67 | | -# Running tests locally |
68 | | - |
69 | | -Do make sure your tests pass locally before deploying to the CI environment. |
70 | | -Below is a handful of commands that run tests in a fashion very close to how the |
71 | | -CI environment runs them. These commands are also useful when you need to |
72 | | -reproduce a CI test failure locally. |
73 | | - |
74 | | -To run a test use option `-t` (`--task`): |
75 | | - |
76 | | -```sh |
77 | | -dart bin/run.dart -t {NAME_OF_TEST} |
78 | | -``` |
79 | | - |
80 | | -To run multiple tests repeat option `-t` (`--task`) multiple times: |
81 | | - |
82 | | -```sh |
83 | | -dart bin/run.dart -t test1 -t test2 -t test3 |
84 | | -``` |
85 | | - |
86 | | -To run all tests defined in `manifest.yaml` use option `-a` (`--all`): |
87 | | - |
88 | | -```sh |
89 | | -dart bin/run.dart -a |
90 | | -``` |
91 | | - |
92 | | -To run tests from a specific stage use option `-s` (`--stage`): |
93 | | - |
94 | | -```sh |
95 | | -dart bin/run.dart -s {NAME_OF_STAGE} |
96 | | -``` |
|
0 commit comments