Schedule/Prioritise Content

[saul-jb]

Describe the feature you are proposing

It would be useful to have more control on the priority of content a user might add to the cluster allowing them to choose what content is downloaded on the nodes first. This is similar to the item on the roadmap: Optimistic replication: allow cluster peers to decide what content they back rather than defining allocations., it would be great to allow cluster peers to choose priorities alongside the content the back.

An example of this would be adding 1TB of low value data to the cluster then adding 1GB of high value data later - one would like the high value data done first or at a faster rate before continuing on the 1TB workload.

Additional context

This would make the cluster more useful for applications that want to use the cluster daemon over the API but needs more find-grained control (dynamically adding and changing priorities).

[hsanjuan]

There is a level of control that you can have with the priority_pin_max_age and priority_pin_max_retries options. Not exactly what you need, but they assure that if something hasn't managed to be pinned in while it will be deprioritized vs. new-arrivals.

The idea is mostly to deal with growing queues that include unretrievable content, so that this content doesn't not block progress on content that is indeed retrievable.