Skip to content

Commit

Permalink
Always fail crawl if failOnFailedSeed set and seed is 4xx/5xx
Browse files Browse the repository at this point in the history
  • Loading branch information
tw4l committed May 6, 2024
1 parent ddc3e10 commit 81388e6
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion src/crawler.ts
Expand Up @@ -1727,8 +1727,9 @@ self.__bx_behaviors.selectMainBehavior();

let failed = isChromeError;

if (this.params.failOnInvalidStatus && status >= 400) {
if ((this.params.failOnInvalidStatus || failCrawlOnError) && status >= 400) {
// Handle 4xx or 5xx response as a page load error
// Always fail seeds on 4xx or 5xx response if failOnFailedSeed is set
failed = true;
}

Expand Down

0 comments on commit 81388e6

Please sign in to comment.