Adding ignoreddosprotection configuration flag

[ineiti]

This PR adds the following configuration flag:

ignoreddosprotection - ignore a missing 'LinkChecker' response header if the user requests more than 6 requests per second.

Help needed: I don't know how to create the doc/i18n/*.po(|t) files - they seem
to be auto-generated, but I don't see how to do that.

See #778

[cjmayo]

It was partly, something like DOS, if that was within the capability of LinkChecker, not sure about DDOS, but at least to prompt a conversation with the server administrator. This was a new capability, the previous maximum rate had been there for years. How do you decide what it should be in the 2020s...

But also because this was a global setting, so if

• You are using --check-extern and have multiple links to an external site on one of your pages, they could then get requested at quite a rate.
• You normally don't use --check-extern but on one occasion you do and forget you have set a high rate. (Probably other scenarios involving forgetting too)

With that in mind I don't think just a boolean to turn it off is sufficient.

Perhaps adding hosts to Aggregate.maxrated{} is a possibility.

doc/translations.md has some info on translations - but what it doesn't mention is there is a GitHub Action that can do all that automatically in a subsequent PR.

[ineiti]

Thanks for the feedback - would the following be a good change of the PR?

• s/DDOS/DOS/i everywhere
• remove the ignoredosprotection
• add a maxratedhosts which takes an array of hostnames

[cjmayo]

Basically yes, though let's forget DOS completely in the description and just describe the relationship to maxrequestspersecond (maybe it could have that much of an impact but I guess it would only be in certain circumstances and it is an effect, not how LinkChecker is working).

We either have comma-separated lists or MULTILINE. Is this a MULTILINE?

I'm not protective of maxrated either if a better idea comes along, the code can be changed to match. Although it might be less work to discuss any ideas first.

The result of this I imagine is going to be that maxrequestspersecond applies to the maxratedhosts and the rest get the default of 10. Kind of, because it's shared and it depends how many there are. Plus maxrequestspersecond can even be set to less than 10! Not that this is anything new with this PR but a bit complicated... Does that mean that if maxrequestspersecond was < 10 then the maxratedhosts would actually be checked more slowly???

[cjmayo]

Had time to read Aggregate.wait_for_host(). Some of my assumptions / questions above weren't valid.

[cjmayo]

Maybe it would be possible to not add a new option but extend maxrequestspersecond to support MULTILINE as well, accepting per-host values which would also give control over not just applying the default to other hosts.