Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ChromeDriver] Tests failing with invalid session id #468

Open
PragTob opened this issue Sep 18, 2019 · 14 comments
Open

[ChromeDriver] Tests failing with invalid session id #468

PragTob opened this issue Sep 18, 2019 · 14 comments

Comments

@PragTob
Copy link
Collaborator

PragTob commented Sep 18, 2019

Howdy fellow wallabyers!

We have some tests that are stubbornly flaky on CI and sometimes locally and I can't for the live of me figure out why that might be happening.

I couldn't reproduce locally right now but on CI a colleague had to rerun the tests 5 times for them to pass.

Error

    1) test langing page contains css (PricingEngineWeb.LandingPageTest)
      apps/pricing_engine_web/test/pricing_engine_web/features/landing_page_test.exs:5
      ** (FunctionClauseError) no function clause matching in Wallaby.Experimental.Selenium.WebdriverClient.cast_as_element/2
      The following arguments were given to Wallaby.Experimental.Selenium.WebdriverClient.cast_as_element/2:

          # 1
          %Wallaby.Session{driver: Wallaby.Experimental.Chrome, id: "444f39bc45a713370fb47558d9e539ac", screenshots: [], server: Wallaby.Experimental.Chrome.Chromedriver, session_url: "http://localhost:47863/session/444f39bc45a713370fb47558d9e539ac", url: "http://localhost:47863/session/444f39bc45a713370fb47558d9e539ac"}

          # 2
          {"message", "invalid session id\n  (Driver info: chromedriver=72.0.3626.121,platform=Linux 4.14.133-88.105.amzn1.x86_64 x86_64)"}

      Attempted function clauses (showing 2 out of 2):

          defp cast_as_element(parent, -%{"ELEMENT" => id}-)
          defp cast_as_element(parent, -%{"element-6066-11e4-a52e-4f735466cecf" => id}-)

      code: |> assert_has(css(".login"))
      stacktrace:
        (wallaby) lib/wallaby/experimental/selenium/webdriver_client.ex:566: Wallaby.Experimental.Selenium.WebdriverClient.cast_as_element/2
        (elixir) lib/enum.ex:1340: anonymous fn/3 in Enum.map/2
        (stdlib) maps.erl:232: :maps.fold_1/3
        (elixir) lib/enum.ex:1964: Enum.map/2
        (wallaby) lib/wallaby/experimental/selenium/webdriver_client.ex:43: Wallaby.Experimental.Selenium.WebdriverClient.find_elements/2
        (wallaby) lib/wallaby/driver/log_checker.ex:6: Wallaby.Driver.LogChecker.check_logs!/2
        (wallaby) lib/wallaby/browser.ex:1140: anonymous fn/3 in Wallaby.Browser.execute_query/2
        (wallaby) lib/wallaby/browser.ex:146: Wallaby.Browser.retry/2
        test/pricing_engine_web/features/landing_page_test.exs:8: (test)
 ..........

Test Code

Multiple tests are randomly failing, one of them being our easiest:

  test "langing page contains css", %{session: session} do
    session
    |> visit("/")
    |> assert_has(css(".login"))
    |> assert_has(css(".login__header"))
    |> assert_has(css(".login__main"))
    |> assert_has(Query.text("Welcome to Pricing Hub"))
    |> assert_has(link("test__google-auth-link"))
  end

The others are usually simple CRUD tests (app isn't that complex yet) :) Often times 2 of them are failing. Sometimes it's just one.

There is no database access here. Just good old landing page. The session is also piped through from beginning to end so no issues with stale sessions.

Environment

  • Wallaby 0.23.0
  • Chromedriver configured to use with chromium, Chromedriver 72 on CI, 76 locally (no idea why it's that old on CI...)
  • Elixir 1.9.0/1.9.1
  • Erlang 22
  • On CI tests are running inside a docker container for testing
  • Tests are being executed in parallel/async

The whole thing reminds me a bit of #221

Any ideas/help/input on how to remedy this would be highly appreciated 👌

@michallepicki
Copy link
Contributor

Tests are being executed in parallel/async

How many tests are being run in parallel? (what's the --max-cases/:max_cases ExUnit option?)

@PragTob
Copy link
Collaborator Author

PragTob commented Sep 18, 2019

@michallepicki it's not set explicitly and I don't know how many cores the CI has off the top of my head and I don't have SSH access

edit:
lscpu says 4 CPUs so probably max-cases is set to 8. We thought about reducing it as we think it might be an issue with overloading chromedriver but not quite sure

@PragTob
Copy link
Collaborator Author

PragTob commented Sep 19, 2019

After changing tests to run with --max-cases 2 on CI we haven't seen the failures since. Will continue to observe. So, it might be that it's some sort of problem with overloading or state interfering. Question is just whether it's in wallaby or chromedriver or what not.

For whoever comes afterwards, a team member also had an issue where this occurred all the time but there the issue was non matching chromedriver/chromium versions. That's decidedly not the case on CI as linux package management enforces matching versions there.

WHile our problem seems solved I think I'd like to keep this open because it might still be a problem that's fixable or at least better highlightable in wallaby.

@PragTob
Copy link
Collaborator Author

PragTob commented Oct 30, 2019

For the record, the problem popped up again restricting it to --max-cases 2 fixed it again 🤷‍♂️

I'm unsure what this is. Part of me believes chromedriver just dies.

Which is supported by the error I get sometimes being:

 ** (MatchError) no match of right hand side value: {:error, "unknown error: Chrome failed to start: exited abnormally\n  (chrome not reachable)\n  (The process started from chrome location /usr/lib/chromium/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)\n  (Driver info: chromedriver=72.0.3626.121,platform=Linux 5.0.0-29-generic x86_64)"}
app_1       |      stacktrace:
app_1       |        (pricing_engine_web) test/support/feature_case.ex:35: PricingEngineWeb.FeatureCase.__ex_unit_setup_0/1
app_1       |        (pricing_engine_web) test/support/feature_case.ex:1: PricingEngineWeb.FeatureCase.__ex_unit__/2
app_1       |        test/pricing_engine_web/features/pricing_grid_test.exs:1: PricingEngineWeb.PricingGridTest.__ex_unit__/2

@mhanberg mhanberg changed the title Random test failures with FunctionClauseError) no function clause matching in Wallaby.Experimental.Selenium.WebdriverClient.cast_as_element/2 [ChromeDriver] Tests failing with invalid session id Nov 8, 2019
@PJUllrich
Copy link

Small note for future reference: Make sure that your chromedriver version is the same on your local machine and on your CI. Pin the version on your CI if necessary. Every now and then my CI broke because chromedriver got a new version, which wasn't installed on my local machine, leading to the problem that the tests ran locally, but broke in the CI with the above mentioned error meassage.

@josephan
Copy link

josephan commented Jan 1, 2020

I had the same issue as well, however my tests were passing on CI but not local (macOS).
My CI was running chromedriver 79 and my local was running chromedriver 77.
I fixed it by running brew cask upgrade chromedriver

@sveredyuk
Copy link

sveredyuk commented Apr 12, 2020

+1 to @PragTob

I had the same issue on GitlabCI and local machine.
Gitlab CI + Wallaby.Experimental.Chrome

@PJUllrich
Copy link

I ran into this issue again locally and would like to recommend to verify that your chromedriver has the same version as your Chrome-Browser, if you have one installed. In my case, Chrome updated itself to Version 81, but my chromedriver was still at 79. That lead to the {"error" ,"invalid session id"} error. After updating chromedriver to 81, my tests ran through successfully again.

@bitboxer
Copy link

bitboxer commented May 8, 2020

I also ran into this problem and have no clue on how to make chrome+wallaby work on travis ci. Details here. Would love some help.

@eahanson
Copy link
Contributor

I had the same problem in the past, and my fix was what @PJUllrich recommended: make sure Chrome and Chromedriver are at the same version. Here's a bash script that I run as part of my apps' doctor script:

#!/usr/bin/env bash

chromedriver_path=$(command -v chromedriver)

chrome_path="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
chromedriver_version=$("${chromedriver_path}" --version)
chrome_version=$("${chrome_path}" --version)

chromedriver_major_version=$("${chromedriver_path}" --version | cut -f 2 -d " " | cut -f 1 -d ".")
chrome_major_version=$("${chrome_path}" --version | cut -f 3 -d " " | cut -f 1 -d ".")

if [ "${chromedriver_major_version}" == "${chrome_major_version}" ]; then
  exit 0
else
  echo "Wallaby often fails with 'invalid session id' if Chromedriver and Chrome have different versions."
  echo "Chromedriver version: ${chromedriver_version} (${chromedriver_path})"
  echo "Chrome version      : ${chrome_version} (${chrome_path})"
  exit 1
fi

@lpender
Copy link

lpender commented Mar 30, 2021

@eahanson Thanks for the script it worked great for me!

Little update for Linux compatibility in CI:

#!/usr/bin/env bash

chromedriver_path=$(command -v chromedriver)

unameOut="$(uname -s)"
case "${unameOut}" in
    Linux*)     chrome_path="/usr/bin/google-chrome";;
    Darwin*)    chrome_path="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome";;
esac
chromedriver_version=$("${chromedriver_path}" --version)
chrome_version=$("${chrome_path}" --version)

chromedriver_major_version=$("${chromedriver_path}" --version | cut -f 2 -d " " | cut -f 1 -d ".")
chrome_major_version=$("${chrome_path}" --version | cut -f 3 -d " " | cut -f 1 -d ".")

if [ "${chromedriver_major_version}" == "${chrome_major_version}" ]; then
  echo "Chromedriver matches chrome version ✓"
  exit 0
else
  echo "Wallaby often fails with 'invalid session id' if Chromedriver and Chrome have different versions."
  echo "Chromedriver version: ${chromedriver_version} (${chromedriver_path})"
  echo "Chrome version      : ${chrome_version} (${chrome_path})"
  exit 1
fi

@btihen
Copy link

btihen commented Apr 3, 2021

I have the same problem on MacOS X: 11.2.3 (20D91)
I installed using ASDF: cat .tool-versions
erlang 23.3.1
elixir 1.11.4-otp-23
nodejs 14.15.5
Postgres 13.2

wallaby 0.28

I have the same version of chromedriver as my browser
I couldn't get wallaby to find chromedriver in the path until I installed it locally in my project.

If there is a script to change please tell me which file to change - (I am relatively new to Phoenix/Elixir and completely new to Wallaby)

I followed the setup instructions at: https://www.tddphoenix.com/setting-up/

✔ ~/devel/learning/germanvelasco/tdd_phoenix/chatter [install_wallaby L|✚ 9…9]
19:41 $ mix test
..

  1. test user can visit homepage (ChatterWeb.UserVisitsHomepageTest)
    test/features/user_visits_homepage_test.exs:4
    ** (RuntimeError) invalid session id
    code: |> visit("/")
    stacktrace:
    (wallaby 0.28.0) lib/wallaby/httpclient.ex:136: Wallaby.HTTPClient.check_for_response_errors/1
    (wallaby 0.28.0) lib/wallaby/httpclient.ex:56: Wallaby.HTTPClient.make_request/5
    (wallaby 0.28.0) lib/wallaby/webdriver_client.ex:329: Wallaby.WebdriverClient.visit/2
    (wallaby 0.28.0) lib/wallaby/driver/log_checker.ex:6: Wallaby.Driver.LogChecker.check_logs!/2
    (wallaby 0.28.0) lib/wallaby/browser.ex:1057: Wallaby.Browser.visit/2
    test/features/user_visits_homepage_test.exs:6: (test)

..

Finished in 3.0 seconds
5 tests, 1 failure

Randomized with seed 381981
✘-1 ~/devel/learning/germanvelasco/tdd_phoenix/chatter [install_wallaby L|✚ 9…9]
19:41 $ chromedriver
Starting ChromeDriver 89.0.4389.23 (61b08ee2c50024bab004e48d2b1b083cdbdac579-refs/branch-heads/4389@{#294}) on port 9515
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.

@matreyes
Copy link

matreyes commented Apr 29, 2022

Hi!

I'm using Wallaby for scrapping so it failed randomly (with some pages on specific circumstances)

From developers.google.com:

By default, Docker runs a container with a /dev/shm shared memory space 64MB. This is typically too small for Chrome and will cause Chrome to crash when rendering large pages. To fix, run the container with docker run --shm-size=1gb to increase the size of /dev/shm. Since Chrome 65, this is no longer necessary. Instead, launch the browser with the --disable-dev-shm-usage flag.

So using:

      Wallaby.start_session(
        capabilities: %{
          chromeOptions: %{
            args: [
              "--headless",
              "--no-sandbox",
              "window-size=1280,800",
              "--fullscreen",
              "--disable-gpu",
              "--disable-dev-shm-usage"
            ]
          }
        }
      )

Worked for me (after lot of suffering).

@bottlenecked
Copy link

bottlenecked commented Jul 20, 2023

In case someone has troubles after the 115 release of chromedriver, here's an amended version of the script from @lpender and @eahanson that works for me

#!/usr/bin/env bash

# from https://github.com/elixir-wallaby/wallaby/issues/468#issuecomment-810518368

chromedriver_path=$(command -v chromedriver)
platform="mac-x64"

function get_download_url {
  major_version=$1
  curl -s curl "https://googlechromelabs.github.io/chrome-for-testing/latest-versions-per-milestone-with-downloads.json" \
        | jq -r '.milestones."'$major_version'".downloads.chromedriver | .[] | select(.platform | contains("'$platform'")) | .url'
}

function download_chromedriver {
  download_url=$1  
  curl -s $download_url -o "chromedriver_${platform}.zip"
}

function install_chromedriver_executable {
  unzip -oq "chromedriver_${platform}.zip"
  sudo mv "chromedriver-${platform}/chromedriver" "/usr/local/bin/chromedriver"
  rm "chromedriver_${platform}.zip"
  rm -rf "chromedriver-${platform}"
}

unameOut="$(uname -s)"
case "${unameOut}" in
    Linux*)     chrome_path="/usr/bin/google-chrome";;
    Darwin*)    chrome_path="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome";;
esac
chromedriver_version=$("${chromedriver_path}" --version)
chrome_version=$("${chrome_path}" --version)

chromedriver_major_version=$("${chromedriver_path}" --version | cut -f 2 -d " " | cut -f 1 -d ".")
chrome_major_version=$("${chrome_path}" --version | cut -f 3 -d " " | cut -f 1 -d ".")

if [ "${chromedriver_major_version}" == "${chrome_major_version}" ]; then
  echo "Chromedriver matches chrome version ✓"
  exit 0
else
  echo "Wallaby often fails with 'invalid session id' if Chromedriver and Chrome have different versions."
  echo "Chromedriver version: ${chromedriver_version} (${chromedriver_path})"
  echo "Chrome version      : ${chrome_version} (${chrome_path})"

  echo ""
  echo "Install latest version? [y/n]"
  read install
  echo "install = $install"

  if [[ "$install" == "y" ]]; then
    echo "determining latest chromedriver version under ${chrome_major_version} ..."
    url=$(get_download_url $chrome_major_version)
    echo "downloading chromedriver version ${chrome_major_version}..."
    $(download_chromedriver $url)
    echo "installing chromedriver (requires sudo)..."
    $(install_chromedriver_executable)
    echo "Done :)"
  else
    exit 1
  fi
fi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests