Curio Network Optimizaton

[snadrus]

Related Issues

Network-level optimization has not been a priority. Here is a straightforward way we could implement it.

Proposed Changes

CanAccept's end (once hard requirements are met) can include a "soft requirements" section managed by taskhelp.NetworkOptimizer which skips rounds if soft requirements are too few.

Additional Info

Checklist

Before you mark the PR ready for review, please make sure that:

• Commits have a clear commit message.
• PR title is in the form of of <PR type>: <area>: <change being made>
  • example: fix: mempool: Introduce a cache for valid signatures
  • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
  • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
• If the PR affects users (e.g., new feature, bug fix, system requirements change), update the CHANGELOG.md and add details to the UNRELEASED section.
• New features have usage guidelines and / or documentation updates in
  • Lotus Documentation
  • Discussion Tutorials
• Tests exist for new functionality or change in behavior
• CI is green

[magik6k]

I quite like this approach.

Would be nice to ship this initially with some machine-resource-based funcs - e.g. "machine load over 50%" and "machine load over 80%" - then give SPs a config toggle to say "prefer spreading load - less utilized machines accept tasks first" and "prefer consolidating load - more utilized machines accept tasks first" (in the latter more of the cluster can idle/sleep when not needed)

I can see this applied to almost all heavier tasks that aren't latency-sensitive (like winningPoSt which needs to happen in <30s)