Take learnings from Web Almanac site and apply to main httparchive site

[tunetheweb]

We learned a lot from making the Web Almanac site and it follows a lot of best practices, particularly in terms of Accessibility, Performance, SEO and automation to ensure we stay at the top of that.

We should bring those learnings here and align the web sites, which will also make them easier to maintain together.

Making a list so I don't forget anything:

• - Move Virtual Env to .venv to prevent linking problems - Move Virtual Env folder inline with Web Almanac #241
• - Add robots.txt and sitemap.xml - Add Robots.txt and Sitemap.xml #243
• - Add GitHub SuperLinter - Add GitHub Super Linter #244
• - Rename master branch to main - A few cleanups after renaming master to main #245
• - Add CodeQL scanning for JS and Python files to supplement Super Linter - Add CodeQL scanning #246 and Fix codeql issues #247
• - Add test script and run it in an action Add test script and action #248
• - Update all dev dependencies and automate Update all dev dependencies to latest versions and add new dependabot #249
• - Look at whether to remove Babel? Is it really needed as quite chunky and adds a lot of dependencies Remove Babel #443
• - Look at whether to remove Bootstrap - do we really need it?
• - Might need another normalize CSS script if we do remove Bootstrap.
• - Add metadata for SEO reason (see also Web page metadata #34 ) - Seo fixes #252
• - Set rel="canonical" on reports to avoid query param versions getting indexed. - Seo fixes #252
• - Set noindex when not using httparchive.org domain to avoid GCP version getting indexed - Seo fixes #252
• - Fix bug where by reports are only loaded when report URL is loaded - Seo fixes #252
• - Add Lighthouse tests on pull request - Add Lighthouse automatic checks #253
• - Lighthouse review - Add Lighthouse automatic checks #253
• - Upgrade highcharts to latest version as current version has a vulnerability - Upgrade highcharts to latest version #255
• - Look at CSP
• - Look at colour contrast - Resolve a number of accessibility issues #256
• - Move to em/rem based sizes to allow rescaling.
• - Accessibility review - Resolve a number of accessibility issues #256
• - Move to same menus as web almanac? Have made the current menus keyboard accessible (though think we should change hover/focus to click) so might not be needed now.
• - Performance review
• - Self-host Google Fonts - Self-host Google Fonts #258
• - SEO review
• - Add Security Headers - Add caching and security headers #257
• - Add Cache control headers - Add caching and security headers #257
• - Add redirects for trailing slashes and other commonly mistyped names - Redirects and error tracking in GA #516
• - Add web-vitals script? - Add web vitals script #515
• - Refactor Python scripts to organise into a server directory - Move server to server directory #513
• - Add pytest (ideally with 100% coverage!) - Add 100% coverage to pytest #512
• - Add deploy scripts - Add deploy scripts similar to Web Almanac #260

Feel free to add to this as you see fit, and grab some if they seem of interest. I'll start will work my way through them anyway.

[tunetheweb]

Made quite a bit of progress with this, however the Accessibility improvements in particular have necessitated some changes:

• Some of the text had bad colour contrast, including the sub-headings:

and the table headings:

And our links:

Made them dark enough to pass colour-contrast checkers, which maybe doesn't follow the original style - the sub headings are now the same colour but I couldn't get a grey that passed:

The links also blend in a bit more to the text when darker, but I brought back the underlines (except for menu links) which helps:

Also a lot of the links had the same names ("View reports", "Learn more"...etc.) so made them a bit more descriptive.

Also added a few missing <h1>s which sometimes resulted in a slight style change.

Other than those, most of the above changes shouldn't result in any noticeable change (except to AT users and search engine crawlers!).

@rviscomi once we decide on #256 could you give this a a good going over (as quite a lot of change!) and then let's do a release?

[rviscomi]

Sounds good. Thanks for working on this—these are some long-overdue improvements! 🙌

[tunetheweb]

OK merged everything I'm gonna merge for now. If you could have a good look over it @rviscomi that would be good.