make compiler worker count configurable (flutter#17616)

yjbanov · web-flow · commit 703525502b19 · 2020-05-08T11:18:02.000-07:00
* make compiler worker count configurable
diff --git a/lib/web_ui/dev/README.md b/lib/web_ui/dev/README.md
@@ -1,7 +1,9 @@
 ## What's `felt`?
+
 `felt` stands for "Flutter Engine Local Tester". It's a cli tool that aims to make development in the Flutter web engine more productive and pleasant.
 
 ## What can `felt` do?
+
 `felt` supports multiple commands as follows:
 
 1. **`felt check-licenses`**: Checks that all Dart and JS source code files contain the correct license headers.
@@ -11,26 +13,40 @@
 You could also run `felt help` or `felt help <command>` to get more information about the available commands and arguments.
 
 ## How can I use `felt`?
+
 Once you have your local copy of the engine [setup](https://github.com/flutter/flutter/wiki/Setting-up-the-Engine-development-environment), it's recommended that you add `/path/to/engine/src/flutter/lib/web_ui/dev` to your `PATH`.
 Then you would be able to use the `felt` tool from anywhere:
+
 ```
 felt check-licenses
 ```
+
 or:
+
 ```
 felt build --watch
 ```
 
 If you don't want to add `felt` to your path, you can still invoke it using a relative path like `./web_ui/dev/felt <command>`
 
-## Speeding up your builds
-You can speed up your builds by using more CPU cores. Pass `-j` to specify the desired level of parallelism, like so:
+## Speeding up your builds and tests
+
+You can speed up `ninja` and `dart2js` by adding parallelism and taking advantage of more cores.
+
+To speed up ninja pass `-j` to specify the desired level of parallelism, like so:
+
 ```
 felt build [-w] -j 100
 ```
+
 If you are a Google employee, you can use an internal instance of Goma to parallelize your builds. Because Goma compiles code on remote servers, this option is effective even on low-powered laptops.
 
+By default, when compiling Dart code to JavaScript, we use 4 `dart2js` workers.
+If you need to increase or reduce the number of workers, set the `BUILD_MAX_WORKERS_PER_TASK`
+environment variable to the desired number.
+
 ## Running web engine tests
+
 To run all tests on Chrome. This will run both integration tests and the unit tests:
 
 ```
@@ -86,6 +102,7 @@ felt test test/golden_tests/engine/canvas_golden_test.dart
 ```
 
 To debug a test on Chrome:
+
 ```
 felt test --debug test/golden_tests/engine/canvas_golden_test.dart
 ```
@@ -110,7 +127,9 @@ To make sure you are running the `felt` tool with your changes included, you wou
 ```
 FELT_USE_SNAPSHOT=false felt <command>
 ```
+
 or
+
 ```
 FELT_USE_SNAPSHOT=0 felt <command>
 ```
diff --git a/lib/web_ui/dev/test_runner.dart b/lib/web_ui/dev/test_runner.dart
@@ -392,11 +392,25 @@ class TestCommand extends Command<bool> with ArgUtils {
           '--build-filter=${path.relativeToWebUi}.browser_test.dart.js',
         ],
     ];
+    final Stopwatch stopwatch = Stopwatch()..start();
+
     final int exitCode = await runProcess(
       environment.pubExecutable,
       arguments,
       workingDirectory: environment.webUiRootDir.path,
+      environment: <String, String>{
+        // This determines the number of concurrent dart2js processes.
+        //
+        // By default build_runner uses 4 workers.
+        //
+        // In a testing on a 32-core 132GB workstation increasing this number to
+        // 32 sped up the build from ~4min to ~1.5min.
+        if (io.Platform.environment.containsKey('BUILD_MAX_WORKERS_PER_TASK'))
+          'BUILD_MAX_WORKERS_PER_TASK': io.Platform.environment['BUILD_MAX_WORKERS_PER_TASK'],
+      },
     );
+    stopwatch.stop();
+    print('The build took ${stopwatch.elapsedMilliseconds ~/ 1000} seconds.');
 
     if (exitCode != 0) {
       throw ToolException(
diff --git a/lib/web_ui/dev/utils.dart b/lib/web_ui/dev/utils.dart
@@ -41,6 +41,7 @@ Future<int> runProcess(
   List<String> arguments, {
   String workingDirectory,
   bool mustSucceed: false,
+  Map<String, String> environment = const <String, String>{},
 }) async {
   final io.Process process = await io.Process.start(
     executable,
@@ -50,6 +51,7 @@ Future<int> runProcess(
     // the process is not able to get Dart from path.
     runInShell: io.Platform.isWindows,
     mode: io.ProcessStartMode.inheritStdio,
+    environment: environment,
   );
   final int exitCode = await process.exitCode;
   if (mustSucceed && exitCode != 0) {
