feat(#3628): add rotating log handler to support log rotation

[yushao2]

Background

Opening this because PR #4844 is stale

Refer to #3628 for more context -- this PR adds the option to rotate logs based on size. Rotation can be configured using 2 settings:

• LOG_MAX_BYTES -- when configured, logs will automatically rotate when it approaches the set size (in bytes)
• LOG_BACKUP_COUNT -- number of backup / rotated logs to keep on disk. If not set and LOG_MAX_BYTES the logs will just be cleared when it approaches the set size.

Some design consideration:
I found out from the docs that when LOG_MAX_BYTES and LOG_BACKUP_COUNT is set to 0, essentially RotatingFileHandler behaves like a regular FileHandler See: here

Closes #3628, closes #4844.

[codecov]

Codecov Report

Merging #5815 (b2649dd) into master (e9e1034) will increase coverage by 46.30%.
The diff coverage is 100.00%.

❗ Current head b2649dd differs from pull request most recent head 0f0bc85. Consider uploading reports for the commit 0f0bc85 to get more accurate results

@@             Coverage Diff             @@
##           master    #5815       +/-   ##
===========================================
+ Coverage   42.36%   88.66%   +46.30%     
===========================================
  Files         162      162               
  Lines       11206    11209        +3     
  Branches     1825     1825               
===========================================
+ Hits         4747     9939     +5192     
+ Misses       6103      986     -5117     
+ Partials      356      284       -72     

Impacted Files	Coverage Δ
scrapy/utils/log.py	89.89% <100.00%> (+49.27%)	⬆️

... and 140 files with indirect coverage changes

[wRAR]

The comment from the original PR about it needing a test applies here too.

[yushao2]

@wRAR I have no idea how to write unittests for the logging, so i pretty much copied from cpython's example and shoehorned it to work. I hope that's fine
https://github.com/python/cpython/blob/main/Lib/test/test_logging.py

[yushao2]

seems like checks are failing on 3.7 & 3.8, give me some time while i set up a 3.7 env and run tests locally. it seems like manual trigger of workflows are needed given that i'm a new contributor so let me try to sort this out