[Bug] Crawling job failed: ProtocolError: Protocol error (Page.captureScreenshot)

[optroodt]

Since version 0.13.1, crawling seems to fail at taking screenshots. It does a total of 5 tries, then fails. It doesn't happen always, but most of the time, for different urls, but I haven't found any patterns yet.

Here is some output from the hoarder-workers container:

2024-04-29T05:23:00.415Z info: [Crawler][73] Will crawl "https://ntfy.sh/" for link with id "v1x4322u8dsfcqxu4lphrb11"
2024-04-29T05:23:02.098Z info: [Crawler][73] Successfully navigated to "https://ntfy.sh/". Waiting for the page to load ...
2024-04-29T05:23:06.107Z info: [search][167] Attempting to index bookmark with id tlxoo973txr3tdw5mj5m48n9 ...
2024-04-29T05:23:06.277Z info: [search][167] Completed successfully
2024-04-29T05:23:07.101Z info: [Crawler][73] Finished waiting for the page to load.
2024-04-29T05:23:07.113Z error: [Crawler][73] Crawling job failed: ProtocolError: Protocol error (Page.captureScreenshot): Internal error

Let me know if you want me to gather more details.

[MohamedBassem]

Yeah, I've seen this happen in a deployment for a friend of mine as well. I'll send a patch to make taking the screenshot optional if it fails. Thanks for the report!

[tehsnarf]

Had the same issue with bookmarking some GitHub links.. adding '--enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb' to the command portion of Chrome resolved this for me. Seems to be some cross-site asset sharing shenanigans involved.

  chrome:
    image: gcr.io/zenika-hub/alpine-chrome:123
    restart: unless-stopped
    command:
      - --no-sandbox
      - --disable-gpu
      - --remote-debugging-address=0.0.0.0
      - --remote-debugging-port=9222
      - --hide-scrollbars
      - --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb

Also, I did have to delete the bookmark and remake it, but it seems to have successfully completed afterwords.

[MohamedBassem]

@tehsnarf oh wow. I have no idea what those "features" are :D How did you end up finding about them? Can you also share some of the github links you mentioned were failing before this change?

[tehsnarf]

@MohamedBassem first github link I used: https://github.com/Donkie/Spoolman

I saw them in the chrome log file:

hoarder-workers      | 2024-05-04T00:58:22.823Z info: [Crawler][1] Successfully navigated to "https://github.com/Donkie/Spoolman". Waiting for the page to load ...
hoarder-workers      | 2024-05-04T00:58:27.827Z info: [Crawler][1] Finished waiting for the page to load.
hoarder-workers      | 2024-05-04T00:58:27.857Z error: [Crawler][1] Crawling job failed: ProtocolError: Protocol error (Page.captureScreenshot): Internal error
hoarder-workers      | 2024-05-04T00:58:28.873Z info: [Crawler][1] Will crawl "https://github.com/Donkie/Spoolman" for link with id "j22uq0e1qi5vqtiaim81fsoy"
hoarder-chrome       | [0504/005828.892692:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.
hoarder-workers      | 2024-05-04T00:58:30.187Z info: [Crawler][1] Successfully navigated to "https://github.com/Donkie/Spoolman". Waiting for the page to load ...
hoarder-workers      | 2024-05-04T00:58:35.188Z info: [Crawler][1] Finished waiting for the page to load.
hoarder-workers      | 2024-05-04T00:58:35.201Z error: [Crawler][1] Crawling job failed: ProtocolError: Protocol error (Page.captureScreenshot): Internal error
hoarder-workers      | 2024-05-04T00:58:37.295Z info: [Crawler][1] Will crawl "https://github.com/Donkie/Spoolman" for link with id "j22uq0e1qi5vqtiaim81fsoy"
hoarder-chrome       | [0504/005837.307231:WARNING:runtime_features.cc(728)] AttributionReportingCrossAppWeb cannot be enabled in this configuration. Use --enable-features=ConversionMeasurement,AttributionReportingCrossAppWeb in addition.

added flag, restarted, deleted and remade bookmark, and golden