Cannot connect to browserless/chrome

[aybartsch]

What happened?

Hi there. I'm trying to connect serenity with "browserless/chrome" which is running in docker container. According to documentation I'm just using the below parameters to connect to the chrome in docker:

-Dwebdriver.driver=chrome -Dwebdriver.remote.url=http://localhost:3000/webdriver

Moreover, I'm setting google chrome options "--headless" and "--no-sandbox" in "serenity.conf". Unfortunately, serenity cannot instantiate the new chromedriver:

net.thucydides.core.webdriver.DriverConfigurationError: Could not instantiate class org.openqa.selenium.chrome.ChromeDriver

	at net.thucydides.core.webdriver.WebDriverFacade.newDriverInstance(WebDriverFacade.java:181)
	at net.thucydides.core.webdriver.WebDriverFacade.newProxyDriver(WebDriverFacade.java:169)
	at net.thucydides.core.webdriver.WebDriverFacade.getProxiedDriver(WebDriverFacade.java:120)
	at net.thucydides.core.webdriver.WebDriverFacade.get(WebDriverFacade.java:209)
	at net.serenitybdd.core.pages.PageObject.openPageAtUrl(PageObject.java:982)
	at net.serenitybdd.core.pages.PageObject.open(PageObject.java:880)
	at net.serenitybdd.core.pages.PageObject.open(PageObject.java:868)
	at com.westernacher.ztr.e2e.steps.basic.LoginSteps.isOnLoginPage(LoginSteps.java:47)
	at com.westernacher.ztr.e2e.steps.basic.LoginSteps.givenTheCentralUserIsOnLoginPage(LoginSteps.java:28)
	at ✽.der zentrale Erfasser befindet sich auf der Loginseite vom ZTR.(classpath:features/basic/LoginCentral.feature:8)
Caused by: net.thucydides.core.webdriver.DriverConfigurationError: WebDriver was unable to create a new instance of type class org.openqa.selenium.chrome.ChromeDriver
WebDriver reported the following message: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure. 
...
Caused by: org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure. 
Host info: host: 'ws-mac-abh', ip: '2a02:8109:9f08:fa00:7c09:4cf4:8b65:3a29%en0'
Build info: version: '4.16.1', revision: '9b4c83354e'
System info: os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '14.4', java.version: '11.0.22'
Driver info: org.openqa.selenium.remote.RemoteWebDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [--no-sandbox, --headless, --no-sandbox, --headless, remote-allow-origins=*], extensions: []}}]}]
Capabilities {browserName: chrome, goog:chromeOptions: {args: [--no-sandbox, --headless, --no-sandbox, --headless, remote-allow-origins=*], extensions: []}}
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:541)
	at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:237)
	at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:164)
	at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:144)
	at net.serenitybdd.core.webdriver.driverproviders.RemoteDriverProvider.newInstance(RemoteDriverProvider.java:86)
	at net.thucydides.core.webdriver.WebDriverFactory.createWebDriver(WebDriverFactory.java:164)
	at net.thucydides.core.webdriver.WebDriverFactory.newWebdriverInstance(WebDriverFactory.java:142)
	... 31 more
Caused by: java.io.UncheckedIOException: java.io.IOException: HTTP/1.1 header parser received no bytes
	at org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute0(JdkHttpClient.java:442)
	at org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
	at org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:55)
	at org.openqa.selenium.remote.http.jdk.JdkHttpClient.execute(JdkHttpClient.java:358)
	at org.openqa.selenium.remote.tracing.TracedHttpClient.execute(TracedHttpClient.java:54)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:115)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:96)
	at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:68)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:163)
	at org.openqa.selenium.remote.TracedCommandExecutor.execute(TracedCommandExecutor.java:51)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:523)

What did you expect to happen?

Since the connection to a selenium grid is working out of the box I would expect that the other URL could also be used to establish a connection to "browserless/chrome". Working setup with selenium grid in docker:

-Dwebdriver.driver=chrome -Dwebdriver.remote.url=http://localhost:4444/wd/hub

Serenity BDD version

4.0.15

JDK version

11.0.21

Execution environment

No response

How to reproduce the bug.

Run a serenity test against docker with "browserless/chrome" started:

-Dwebdriver.driver=chrome -Dwebdriver.remote.url=http://localhost:3000/webdriver

How can we make it happen?

Add it to the Serenity BDD backlog and wait for a volunteer to pick it up

[aybartsch]

The serenity request is already triggering a chrome instance in docker:

2024-04-02 11:53:54 2024-04-02T09:53:54.924Z browserless:job 2CQTK6FVXZET3YP4MCWZINGBJ45SSXL9: /webdriver/session: Inbound WebSocket request.
2024-04-02 11:53:54 2024-04-02T09:53:54.932Z browserless:hardware Checking overload status: CPU 17% Memory 7%
2024-04-02 11:53:54 2024-04-02T09:53:54.933Z browserless:job 2CQTK6FVXZET3YP4MCWZINGBJ45SSXL9: Adding new job to queue.
2024-04-02 11:53:54 2024-04-02T09:53:54.933Z browserless:server Starting new job
2024-04-02 11:53:54 2024-04-02T09:53:54.933Z browserless:system Generating fresh chrome browser
2024-04-02 11:53:54 2024-04-02T09:53:54.933Z browserless:job 2CQTK6FVXZET3YP4MCWZINGBJ45SSXL9: Getting browser.
2024-04-02 11:53:54 2024-04-02T09:53:54.936Z browserless:chrome-helper Launching Chrome with args: {
2024-04-02 11:53:54   "args": [
2024-04-02 11:53:54     "--no-sandbox",
2024-04-02 11:53:54     "--enable-logging",
2024-04-02 11:53:54     "--v1=1",
2024-04-02 11:53:54     "--disable-dev-shm-usage",
2024-04-02 11:53:54     "--no-first-run",
2024-04-02 11:53:54     "--remote-debugging-port=42529",
2024-04-02 11:53:54     "--user-data-dir=/tmp/browserless-data-dir-s5fJ1L"
2024-04-02 11:53:54   ],
2024-04-02 11:53:54   "blockAds": false,
2024-04-02 11:53:54   "dumpio": false,
2024-04-02 11:53:54   "headless": "new",
2024-04-02 11:53:54   "stealth": false,
2024-04-02 11:53:54   "ignoreDefaultArgs": false,
2024-04-02 11:53:54   "ignoreHTTPSErrors": false,
2024-04-02 11:53:54   "pauseOnConnect": false,
2024-04-02 11:53:54   "playwright": false,
2024-04-02 11:53:54   "userDataDir": "/tmp/browserless-data-dir-s5fJ1L",
2024-04-02 11:53:54   "meta": {
2024-04-02 11:53:54     "protocol": null,
2024-04-02 11:53:54     "slashes": null,
2024-04-02 11:53:54     "auth": null,
2024-04-02 11:53:54     "host": null,
2024-04-02 11:53:54     "port": null,
2024-04-02 11:53:54     "hostname": null,
2024-04-02 11:53:54     "hash": null,
2024-04-02 11:53:54     "search": null,
2024-04-02 11:53:54     "query": {},
2024-04-02 11:53:54     "pathname": "/webdriver/session",
2024-04-02 11:53:54     "path": "/webdriver/session",
2024-04-02 11:53:54     "href": "/webdriver/session"
2024-04-02 11:53:54   },
2024-04-02 11:53:54   "executablePath": "/usr/bin/google-chrome",
2024-04-02 11:53:54   "handleSIGINT": false,
2024-04-02 11:53:54   "handleSIGTERM": false,
2024-04-02 11:53:54   "handleSIGHUP": false
2024-04-02 11:53:54 }
2024-04-02 11:53:55 2024-04-02T09:53:55.206Z browserless:chrome-helper Chrome PID: 1534
2024-04-02 11:53:55 2024-04-02T09:53:55.207Z browserless:chrome-helper Finding prior pages
2024-04-02 11:53:55 2024-04-02T09:53:55.217Z browserless:chrome-helper Found 1 pages
2024-04-02 11:53:55 2024-04-02T09:53:55.217Z browserless:chrome-helper Setting up page Unknown
2024-04-02 11:53:55 2024-04-02T09:53:55.217Z browserless:chrome-helper Injecting download dir "/usr/src/app/workspace"
2024-04-02 11:53:55 2024-04-02T09:53:55.218Z browserless:system Chrome launched 285ms
2024-04-02 11:53:55 2024-04-02T09:53:55.218Z browserless:system Got chrome instance
2024-04-02 11:53:55 2024-04-02T09:53:55.218Z browserless:job 2CQTK6FVXZET3YP4MCWZINGBJ45SSXL9: Starting session.
2024-04-02 11:53:55 2024-04-02T09:53:55.218Z browserless:job 2CQTK6FVXZET3YP4MCWZINGBJ45SSXL9: Proxying request to /devtools/browser route: ws://127.0.0.1:42529/devtools/browser/e366bad8-afce-437c-b580-ae5114f2faa0.