Allow the ProviderManager to have more paralleism

[aschmahmann]

The ProviderManager has a single event loop for managing all requests (puts, gets, etc.)

go-libp2p-kad-dht/providers/providers_manager.go

Line 113 in 06918c8

func (pm *ProviderManager) run(proc goprocess.Process) {

The major operations in the event loop such as add and get provider block the event loop and can potentially take a long time (e.g. a network based datastore lookup)

go-libp2p-kad-dht/providers/providers_manager.go

Lines 135 to 153 in 06918c8

	case np := <-pm.newprovs:
	err := pm.addProv(np.key, np.val)
	if err != nil {
	log.Error("error adding new providers: ", err)
	continue
	}
	if gcSkip != nil {
	// we have an gc, tell it to skip this provider
	// as we've updated it since the GC started.
	gcSkip[mkProvKeyFor(np.key, np.val)] = struct{}{}
	}
	case gp := <-pm.getprovs:
	provs, err := pm.getProvidersForKey(gp.key)
	if err != nil && err != ds.ErrNotFound {
	log.Error("error reading providers: ", err)
	}
	// set the cap so the user can't append to this.
	gp.resp <- provs[0:len(provs):len(provs)]

This would lead to the provide manager getting backlogged and a slow down in network responses.

If instead we allow for parallelism on the calls happening within the event loop, such as allowing many adds or gets happening at the same time, then we'd be enabling users to reduce response latencies by increasing the resources they use.

[Stebalien]

Also related: #675. But yeah, given high latency blockstore operations, this should be parallelized.

[Stebalien]

Note: parallelism is a problem for gets, not puts (ish). The datastore is an "auto batching" datastore, so puts are effectively done in parallel.

I say "ish" because we'd need to have a "put" queue to make this fully parallel, otherwise we'll block every time we flush.

[Stebalien]

I guess the main issue here is that gets are blocking puts.

[aschmahmann]

Agreed. Generally the "event loop" pattern tends to fall apart when items inside the loop are blocking, so it'd be nice to have worker pools to keep the event loop clear.