Unable to run selenium in AWS Lambda with Python 3.8

[jiridj]

Hi!

I'm having issues running headless chrome on AWS Lambda with the Python 3.8 runtime. I'm hoping you would have an idea why.

• Chromedriver 87.0.4280.87 for linux64
• Serverless Chrome v1.0.0-57 for amazon-linux-2

This is the exception I get, which according to StackOverflow typically points to incompatibilities between the two libraries above.

{
  "errorMessage": "Message: Service /opt/chromedriver unexpectedly exited. Status code was: 127\n",
  "errorType": "WebDriverException",
  "stackTrace": [
    "  File \"/var/task/serverless_sdk/__init__.py\", line 144, in wrapped_handler\n    return user_handler(event, context)\n",
    "  File \"/var/task/handler.py\", line 13, in scrape\n    driver = webdriver.Chrome('/opt/chromedriver', chrome_options=options)\n",
    "  File \"/opt/python/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py\", line 73, in __init__\n    self.service.start()\n",
    "  File \"/opt/python/lib/python3.8/site-packages/selenium/webdriver/common/service.py\", line 98, in start\n    self.assert_process_still_running()\n",
    "  File \"/opt/python/lib/python3.8/site-packages/selenium/webdriver/common/service.py\", line 109, in assert_process_still_running\n    raise WebDriverException(\n"
  ]
}

What would be the proper version of Chromedriver to run serverless chrome with Python 3.8?

Thanks!

[umihico]

@jiridj
I use serverless-chrome with Python 3.8 in AWS Lambda. In my case, I needed to install dependencies, and installing the latest chrome solves the task. There must be multiple ways to install. My way was below.

• place https://github.com/umihico/docker-selenium-lambda/blob/main/google-chrome.repo under /etc/yum.repos.d/
• yum install -y --enablerepo=google-chrome google-chrome-stable

you can see my Dockerfile here. https://github.com/umihico/docker-selenium-lambda

[jiridj]

Thanks @umihico! This is very helpful!
I interpret your Dockerfile as using chromedriver with chrome, not using headless chrome. Do I read that right?

[umihico]

@jiridj
I installed chrome but not run it. This function runs with adieuadieu's headless-chrome. I just install chrome to solve dependency problem.

[llilicheng4]

so did it work?