feat: add ability to provide existing file digest

[rajington]

Summary

Simulates On-Demand ISR by adding the ability to provide an existing file digest, without requiring those files to actually exist or be hashed (#297).

Rationale

Currently On-Demand ISR is not supported on the Next.js Runtime. We can emulate this capability with external CI and the Netlify CLI by simply redoing the deploy with the changed files, plus all the previous deploys' files. The Netlify CLI hashes all these files, and the Netlify API will reply that only the changed files need to be uploaded.

If there are a lot of these files, we ask our CI to restore and then Netlify CLI to hash files that we know we have no intention of uploading.

Instead, the CI could just cache only the previous deploys' file digest JSON (or re-request it using the undocumented API /api/v1/deploys/{deploy_id}/files), and enable the CLI deploy command to merge the digest of just the new files, with the previous digest.

This is a WIP and more of a PoC of a strategy that could enable performant incremental uploads with an external build system, which is likely not the direction Netlify hopes to pursue.

---
title: Normal On-Demand ISR Workflow
---
flowchart LR
    CMS -->|New Content| Netlify
    Netlify -->|New Pages| Site

---
title: CI-Based "On-Demand ISR" Workflow
---
flowchart LR
    CMS -->|New Content| CI
    CACHE -->|Previous Content| CI
    CI -->|New + Previous Pages| Site

Demo

1. Upload the base-deploy directory containing only one.html to a Netlify site

netlify deploy -d ./base-deploy

...
✔ CDN requesting 4 files
...

2. Confirm file one exists:

curl -I https://$WEBSITE_DRAFT_URL/one

HTTP/2 200
...

3. Upload the new-deploy directory containing only two.html, but pass the existing file digest JSON

netlify deploy -d ./new-deploy --existing-file-digest "$(<./base-deploy.json)"

...
✔ CDN requesting 1 files
...

4. Confirm file two exists

curl -I https://$WEBSITE_DRAFT_URL/two

HTTP/2 200
...

5. Confirm file one also still exists

curl -I https://$WEBSITE_DRAFT_URL/one

HTTP/2 200
...

---

For us to review and ship your PR efficiently, please perform the following steps:

• Open a bug/issue before writing your code 🧑‍💻. This ensures we can discuss the changes and get feedback from everyone that should be involved. If you`re fixing a typo or something that`s on fire 🔥 (e.g. incident related), you can skip this step.
• Read the contribution guidelines 📖. This ensures your code follows our style guide and
  passes our tests.
• Update or add tests (if any source code was changed or added) 🧪
• Update or add documentation (if features were changed or added) 📝
• Make sure the status checks below are successful ✅