[WIP] Add initial setup for BrowserStack CLI #14456

AdityaHirapara · 2025-05-06T09:48:48Z

Proposed changes

Types of changes

Polish (an improvement for an existing feature)
Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update (improvements to the project's docs)
Specification changes (updates to WebDriver command specifications)
Internal updates (everything related to internal scripts, governance documentation and CI files)

Checklist

I have read the CONTRIBUTING doc
I have added tests that prove my fix is effective or that my feature works
I have added the necessary documentation (if appropriate)
I have added proper type definitions for new commands (if appropriate)

Backport Request

//: # (The current main branch is the development branch for WebdriverIO v9. If your change should be released to the current major version of WebdriverIO (v8), please raise another PR with the same changes against the v8 branch.)

This change is solely for v9 and doesn't need to be back-ported
Back-ported PR at #XXXXX

Further comments

Reviewers: @webdriverio/project-committers

…ort_v9

pkg-pr-new · 2025-05-07T10:58:46Z

Open in StackBlitz

eslint-plugin-wdio

npm i https://pkg.pr.new/webdriverio/webdriverio/eslint-plugin-wdio@14456

@wdio/allure-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/allure-reporter@14456

@wdio/browser-runner

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/browser-runner@14456

@wdio/browserstack-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/browserstack-service@14456

@wdio/appium-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/appium-service@14456

@wdio/cli

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/cli@14456

@wdio/concise-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/concise-reporter@14456

@wdio/config

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/config@14456

@wdio/cucumber-framework

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/cucumber-framework@14456

@wdio/dot-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/dot-reporter@14456

@wdio/firefox-profile-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/firefox-profile-service@14456

@wdio/globals

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/globals@14456

@wdio/jasmine-framework

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/jasmine-framework@14456

@wdio/json-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/json-reporter@14456

@wdio/junit-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/junit-reporter@14456

@wdio/lighthouse-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/lighthouse-service@14456

@wdio/local-runner

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/local-runner@14456

@wdio/logger

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/logger@14456

@wdio/mocha-framework

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/mocha-framework@14456

@wdio/protocols

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/protocols@14456

@wdio/repl

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/repl@14456

@wdio/reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/reporter@14456

@wdio/runner

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/runner@14456

@wdio/sauce-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/sauce-service@14456

@wdio/shared-store-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/shared-store-service@14456

@wdio/smoke-test-cjs-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/smoke-test-cjs-service@14456

@wdio/smoke-test-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/smoke-test-reporter@14456

@wdio/smoke-test-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/smoke-test-service@14456

@wdio/spec-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/spec-reporter@14456

@wdio/static-server-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/static-server-service@14456

@wdio/sumologic-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/sumologic-reporter@14456

@wdio/testingbot-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/testingbot-service@14456

@wdio/types

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/types@14456

@wdio/utils

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/utils@14456

@wdio/webdriver-mock-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/webdriver-mock-service@14456

webdriver

npm i https://pkg.pr.new/webdriverio/webdriverio/webdriver@14456

webdriverio

npm i https://pkg.pr.new/webdriverio/webdriverio@14456

commit: c2cc082

christian-bromann

Can we update the BrowserStack service docs to include this type of feature? I think we got a bit behind of documenting what this service does. See https://webdriver.io/docs/browserstack-service. I recommend to either update the docs, or remove them completely with a link to the BrowserStack docs.

packages/wdio-browserstack-service/src/cli/grpcClient.ts

packages/wdio-browserstack-service/src/cli/cliUtils.ts

packages/wdio-browserstack-service/src/cli/grpcClient.ts

packages/wdio-browserstack-service/src/cli/index.ts

packages/wdio-browserstack-service/src/cli/modules/BaseModule.ts

packages/wdio-browserstack-service/src/cli/modules/TestHubModule.ts

packages/wdio-browserstack-service/package.json

…ort_v9

christian-bromann

One last comment, the rest is ok to me. I would love to see more test coverage for all this added code. Also please address the comment above about documenting this feature. It is unclear at the moment what this is about.

christian-bromann · 2025-05-15T20:40:17Z

packages/wdio-browserstack-service/src/cli/cliUtils.ts

+            return new Promise<string|null>((resolve, reject) => {
+                fetch(binDownloadUrl, {
+                    redirect: 'follow'
+                }).then((response) => {
+                    if (response.ok && response.body) {
+                        const binaryName = null
+                        Readable.fromWeb(response.body).pipe(downloadedFileStream)
+
+                        downloadedFileStream.on('error', function (err: Error) {
+                            logger.error('Got Error while downloading percy binary file' + err)
+                            PerformanceTester.end(PerformanceEvents.SDK_CLI_DOWNLOAD, false, util.format(err))
+                            reject(err)
+                        })
+                        CLIUtils.downloadFileStream(downloadedFileStream, binaryName, zipFilePath, cliDir, resolve, reject)
+                    } else {
+                        const err = 'Got Error in cli binary download response' + response.status
+                        logger.error(err)
+                        PerformanceTester.end(PerformanceEvents.SDK_CLI_DOWNLOAD, false, err)
+                        reject(err)
+                    }
+                }).catch((err) => {
+                    logger.error(`Got Error in cli binary downloading request ${util.format(err)}`)
+                    PerformanceTester.end(PerformanceEvents.SDK_CLI_DOWNLOAD, false, util.format(err))
+                    reject(err)
+                })
+                PerformanceTester.end(PerformanceEvents.SDK_CLI_DOWNLOAD)
+            })


Suggested change

return new Promise<string|null>((resolve, reject) => {

fetch(binDownloadUrl, {

redirect: 'follow'

}).then((response) => {

if (response.ok && response.body) {

const binaryName = null

Readable.fromWeb(response.body).pipe(downloadedFileStream)

downloadedFileStream.on('error', function (err: Error) {

logger.error('Got Error while downloading percy binary file' + err)

PerformanceTester.end(PerformanceEvents.SDK_CLI_DOWNLOAD, false, util.format(err))

reject(err)

})

CLIUtils.downloadFileStream(downloadedFileStream, binaryName, zipFilePath, cliDir, resolve, reject)

} else {

const err = 'Got Error in cli binary download response' + response.status

logger.error(err)

PerformanceTester.end(PerformanceEvents.SDK_CLI_DOWNLOAD, false, err)

reject(err)

}

}).catch((err) => {

logger.error(`Got Error in cli binary downloading request ${util.format(err)}`)

PerformanceTester.end(PerformanceEvents.SDK_CLI_DOWNLOAD, false, util.format(err))

reject(err)

})

PerformanceTester.end(PerformanceEvents.SDK_CLI_DOWNLOAD)

})

const response = await fetch(binDownloadUrl, {

redirect: 'follow'

})

if (response.ok && response.body) {

const binaryName = null

Readable.fromWeb(response.body).pipe(downloadedFileStream)

downloadedFileStream.on('error', function (err: Error) {

logger.error('Got Error while downloading percy binary file' + err)

PerformanceTester.end(PerformanceEvents.SDK_CLI_DOWNLOAD, false, util.format(err))

reject(err)

})

CLIUtils.downloadFileStream(downloadedFileStream, binaryName, zipFilePath, cliDir, resolve, reject)

} else {

const err = 'Got Error in cli binary download response' + response.status

logger.error(err)

PerformanceTester.end(PerformanceEvents.SDK_CLI_DOWNLOAD, false, err)

reject(err)

}

PerformanceTester.end(PerformanceEvents.SDK_CLI_DOWNLOAD)

})

Sure, I am working on adding unit tests here. Will commit with the above requested changes.

AdityaHirapara · 2025-05-19T05:42:57Z

Can we update the BrowserStack service docs to include this type of feature? I think we got a bit behind of documenting what this service does. See https://webdriver.io/docs/browserstack-service. I recommend to either update the docs, or remove them completely with a link to the BrowserStack docs.

@christian-bromann We are building a new CLI binary to handle Browserstack product setup flows centrally, and move
language-agnostic code outside SDKs. So, BrowserStack service will launch this binary at startup and will register hooks to perform common product workflows on the binary. For now, we will also fall back to legacy flow, if binary fails.
This PR only consists of code to download and start/stop binary, it doesn't invoke any product workflows. Since the changes are huge, we are planning to implement this in small PRs like these, so it's easier for reviews. But would like to get your opinion on the same.
The issue with this is that we will have a few PRs at the start, which don't change anything, and just initiate the binary setup. The actual impact will start once we start moving product flows, but PR with all changes till then will be bigger. Should we continue with PRs like these, or are you okay with big PRs, where actual impact is made?

I understand the concern regarding documentation as well, which I have raised with our product team, and we have decided to redirect our documentation to BrowserStack pages. Currently, we don't have any public docs of binary internals, I'll discuss if we can publish that too. Although, for users this implementation doesn't change how they use browserstack service. So, there is no additional feature doc needed for this.

christian-bromann · 2025-05-22T16:14:16Z

and we have decided to redirect our documentation to BrowserStack pages.

This makes sense to me. I am happy if the service docs just reference to BrowserStack docs too.

christian-bromann · 2025-06-02T15:53:13Z

@AdityaHirapara please see #14527, I think the current integration is very unclear. We should come up with a plan to provide users better documentation also on the WebdriverIO side. I am curious about your team suggestions.

As per feedback received from christian-bromann on the PR: webdriverio#14456 (review), we updated the existing BrowserStack service documentation with a link to the BrowserStack documentation.

AdityaHirapara · 2025-06-19T14:12:32Z

@christian-bromann Yes, for documentation, our product team is actively looking into this on revamping the docs. To quickly address query raised

What is the plan to support the BS SDK with an out-of-the-box integration?
1. For WebdriverIO, we don’t have plan to move integration out of WebdriverIO. "WebdriverIO browserstack-service" will continue to provide direct integration support to WDIO users.
If no plan, can we have better detailed documentation about how to pair BS SDK with WDIO without using the "Legacy capabilities" ?
1. Have raised this with our product team to incorporate these in our ongoing documentation updates. We will add a configuration options page specific for WDIO, similar to SDK capabilities page.

AdityaHirapara · 2025-06-19T14:23:39Z

@christian-bromann
The second discussion here was regarding a new implementation we are working on. Reiterating on context, we are implementing architectural changes in browserstack service, to use CLI binary in the background. The main purpose of CLI binary here is to move language-agnostic APIs out of service/SDK, and the service will call CLI binary to perform those actions.

Following this implementation, we need to implement major changes across browserstack service, which will result into Big PR. So,

We can either submit one PR with all changes.
Or we can provide series of PRs, which can be reviewed separately and merged into one release branch.

To give you an idea on upcoming PRs (PRs are currently in internal testing, where mostly unit tests and bug fixes are pending)

We can submit a complete PR like this - https://github.com/AdityaHirapara/webdriverio/pull/14/files
OR we can share all PRs in series, to be merged in 1 base PR before release https://github.com/AdityaHirapara/webdriverio/pull/3/files
https://github.com/AdityaHirapara/webdriverio/pull/5/files
https://github.com/AdityaHirapara/webdriverio/pull/7/files
https://github.com/AdityaHirapara/webdriverio/pull/9/files
https://github.com/rounak610/webdriverio/pull/17/files
https://github.com/AdityaHirapara/webdriverio/pull/10/files
https://github.com/rounak610/webdriverio/pull/18/files
https://github.com/AdityaHirapara/webdriverio/pull/13/files
https://github.com/AdityaHirapara/webdriverio/pull/12/files

Please let us know the preferable route for review and release from your perspective.

In addition, regarding this v9 PR, we have prioritised v8 over v9. This was in consideration of the usage split and customer requests in Q1, we had to prioritise releasing support for additional support for V8. This decision was made to address the more immediate needs of the majority of our users. That said, we will be working on supporting v9 by the next quarter.

So, we are requesting you to release v8 independently for these upcoming changes.

christian-bromann · 2025-06-19T17:24:21Z

@AdityaHirapara this makes me wonder whether it makes sense to have the service live under a different umbrella. I mentioned it before that the current service implementation goes way over what we as WebdriverIO can maintain given we have no expertise with the BrowserStack ecosystem. With these large changes, I wonder if it would be a better decision for the BrowserStack team to maintain a custom WDIO SDK and only have a very small glue layer within the service. At this point there is no value in me reviewing these types of code changes anymore.

Please let me know how you want to proceed.

AdityaHirapara · 2025-06-23T05:56:06Z

@christian-bromann we understand the concern here. Actually, this is the step towards that direction, where we move browserstack product logic out of WDIO service. So, the binary will handle communication with our products.

Based on the user's configuration, the service will be responsible for running sessions on the remote BStack hub, and calling the CLI binary to perform product-specific actions based on a test config. So, going forward, fewer and fewer changes will be required on the service, which is related to Bstack products. Mostly integration changes will be required on the service.

Currently, you will see code only being added, as we will need to keep old modules for fallback, in case the CLI flow fails. We can remove old code, once we verify the stability of the CLI flow.

AdityaHirapara · 2025-07-01T21:12:02Z

@christian-bromann did you get a chance to check above? Does that address your concerns?

christian-bromann · 2025-07-01T21:25:05Z

@AdityaHirapara can you help me understand what the protobuffer files are meant for? Shouldn't this life in a BS maintained package and consumed by the service?

AdityaHirapara · 2025-07-03T14:40:54Z

@christian-bromann protobuffer files are for GRPC contract between CLI binary and browserstack service. Service will now connect with CLI binary over GRPC call, to perform browserstack product-related actions

christian-bromann · 2025-07-08T19:43:40Z

package.json

@@ -25,6 +25,7 @@
    "compile:all:all": "pnpm -r --filter=@wdio/compiler run build",
    "generate": "run-s generate:*",
    "generate:bidi": "tsx ./scripts/bidi/index.ts && pnpm eslint --fix packages/webdriver/src/bidi/",
+    "generate:browserstack-proto": "npx grpc_tools_node_protoc --plugin=protoc-gen-ts=./node_modules/.bin/protoc-gen-ts_proto --ts_proto_out=./packages/wdio-browserstack-service/src/proto --ts_proto_opt=outputServices=grpc-js --proto_path=./packages/wdio-browserstack-service/src/proto/browserstack/sdk/v1/ --ts_proto_opt=importSuffix=.js ./packages/wdio-browserstack-service/src/proto/browserstack/sdk/v1/*.proto",


I am not quite sure about this. Adding complexity to our build is not ideal. Wouldn't it make more sense for BS to create an external client that contains all these proto files and build steps, publish that one to NPM and just import it here?

christian-bromann · 2025-07-08T19:47:11Z

packages/wdio-browserstack-service/src/cli/cliLogger.ts

+    static logToFile(logMessage: string, logLevel: string) {
+        try {
+            if (!this.logFileStream) {
+                this.ensureLogsFolder()
+                this.logFileStream = fs.createWriteStream(this.logFilePath, { flags: 'a' })
+            }
+            if (this.logFileStream && this.logFileStream.writable) {
+                this.logFileStream.write(this.formatLog(logMessage, logLevel))
+            }
+        } catch (error) {
+            log.debug(`Failed to log to file. Error ${error}`)
+        }
+    }


This log message would pollute the wdio files every time you log something to the file which may be less desirable. I suggest:

Suggested change

static logToFile(logMessage: string, logLevel: string) {

try {

if (!this.logFileStream) {

this.ensureLogsFolder()

this.logFileStream = fs.createWriteStream(this.logFilePath, { flags: 'a' })

}

if (this.logFileStream && this.logFileStream.writable) {

this.logFileStream.write(this.formatLog(logMessage, logLevel))

}

} catch (error) {

log.debug(`Failed to log to file. Error ${error}`)

}

}

static logToFile(logMessage: string, logLevel: string) {

if (!this.logFileStream) {

try {

this.ensureLogsFolder()

this.logFileStream = fs.createWriteStream(this.logFilePath, { flags: 'a' })

} catch (error) {

log.debug(`Failed to log to file. Error ${error}`)

this.logFileStream = { writable: false }

}

}

if (this.logFileStream && this.logFileStream.writable) {

this.logFileStream.write(this.formatLog(logMessage, logLevel))

}

}

christian-bromann · 2025-07-08T19:47:58Z

packages/wdio-browserstack-service/src/cli/cliUtils.ts

+import { exec } from 'node:child_process'
+import type { ZipFile, Options as yauzlOptions } from 'yauzl'
+import yauzl from 'yauzl'
+import { fetch } from 'undici'


Any reason we can't just use Node.js native fetch?

christian-bromann · 2025-07-08T19:49:06Z

packages/wdio-browserstack-service/src/cli/cliUtils.ts

+import { Readable } from 'node:stream'
+const logger = BStackLogger
+
+export class CLIUtils {


There is no point IMHO to create a class when all methods are static, just export the functions directly

christian-bromann · 2025-07-08T19:50:35Z

packages/wdio-browserstack-service/src/cli/grpcClient.ts

+/**
+ * GrpcClient - Singleton class for managing gRPC client connections
+ *
+ * This class uses the singleton pattern to ensure only one gRPC client instance exists
+ * throughout the application lifecycle.
+ */


I am sure this is something that should live in its own NPM package as you may want to use it in other environments other than just WDIO

Add initial setup for BrowserStack CLI

ba5693e

AdityaHirapara requested a review from a team as a code owner May 6, 2025 09:48

AdityaHirapara added 5 commits May 6, 2025 15:31

Merge remote-tracking branch 'public/main' into browserstack_cli_supp…

a80dc3b

…ort_v9

Revert pnpm lock conflict

ee120ce

Fix linting changes

4d6b96c

Fix package dependencies

e5971c2

Add setup script for grpc protocols

d1f0c4f

AdityaHirapara added 10 commits May 7, 2025 16:30

Fix import

4d11ff5

Fix package json import

6290db1

Add grpc proto setup for unit test as well

17aecd4

Fix unit test run

d4801d8

Fix unit tests

286b6e0

Fix script

36b4002

Fix script

bcd60a1

Fix script and remove logs

19bd6ec

Fix script

14281db

Fix script

a7fa8f8

AdityaHirapara changed the title ~~Add initial setup for BrowserStack CLI~~ [WIP] Add initial setup for BrowserStack CLI May 8, 2025

christian-bromann requested changes May 8, 2025

View reviewed changes

AdityaHirapara added 5 commits May 10, 2025 13:55

Clean up PR and remove unused code

795b616

Promisify callback functions

2d4dbfa

Remove follow-redirects dependency and fix download flow

f957d48

Fix import

b228ef3

Merge remote-tracking branch 'public/main' into browserstack_cli_supp…

c2cc082

…ort_v9

christian-bromann reviewed May 15, 2025

View reviewed changes

sindhupullapantula mentioned this pull request Jun 17, 2025

Update README.md sindhupullapantula/webdriverio#1

Closed

7 tasks

AdityaHirapara mentioned this pull request Jul 7, 2025

[v8] Implement BrowserStack CLI support in browserstack service #14617

Open

13 tasks

christian-bromann reviewed Jul 8, 2025

View reviewed changes

Uh oh!

[WIP] Add initial setup for BrowserStack CLI #14456

Are you sure you want to change the base?

[WIP] Add initial setup for BrowserStack CLI #14456

Uh oh!

Conversation

AdityaHirapara commented May 6, 2025

Proposed changes

Types of changes

Checklist

Backport Request

Further comments

Reviewers: @webdriverio/project-committers

Uh oh!

pkg-pr-new bot commented May 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

christian-bromann left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

christian-bromann left a comment

Choose a reason for hiding this comment

Uh oh!

christian-bromann May 15, 2025

Choose a reason for hiding this comment

Uh oh!

AdityaHirapara May 19, 2025

Choose a reason for hiding this comment

Uh oh!

AdityaHirapara commented May 19, 2025

Uh oh!

christian-bromann commented May 22, 2025

Uh oh!

christian-bromann commented Jun 2, 2025

Uh oh!

AdityaHirapara commented Jun 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AdityaHirapara commented Jun 19, 2025

Uh oh!

christian-bromann commented Jun 19, 2025

Uh oh!

AdityaHirapara commented Jun 23, 2025

Uh oh!

AdityaHirapara commented Jul 1, 2025

Uh oh!

christian-bromann commented Jul 1, 2025

Uh oh!

AdityaHirapara commented Jul 3, 2025

Uh oh!

christian-bromann Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

christian-bromann Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

christian-bromann Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

christian-bromann Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

christian-bromann Jul 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pkg-pr-new bot commented May 7, 2025 •

edited

Loading

AdityaHirapara commented Jun 19, 2025 •

edited

Loading