Idea: TCP Request pipelining support

[TysonAndre]

Currently, gomemcache

1. acquires a connection
2. sends a request
3. receives a response
4. Releases a connection back to the connection pool

gomemcache/memcache/memcache.go

Lines 290 to 304 in f7880d5

	func (c *Client) onItem(item *Item, fn func(*Client, *bufio.ReadWriter, *Item) error) error {
	addr, err := c.selector.PickServer(item.Key)
	if err != nil {
	return err
	}
	cn, err := c.getConn(addr)
	if err != nil {
	return err
	}
	defer cn.condRelease(&err)
	if err = fn(c, cn.rw, item); err != nil {
	return err
	}
	return nil
	}

It's possible in the memcache protocol to send requests without waiting for the response (https://en.wikipedia.org/wiki/Protocol_pipelining) and this is done in various clients/proxies. This allows for lower resource usage (one connection instead of multiple connection, fewer syscalls, etc), and higher throughput when there is higher latency (e.g. memcached in a different datacenter)

Is this something you'd be interested in a PR for?

This is something I'd implemented in https://github.com/TysonAndre/golemproxy (a hackathon project to implement something easier to customize than twemproxy, e.g. for custom business logic to replicate commands or compare results. The source was combined to make the development cycle easier since I was the only one working on that, but the client remains as an independent component with a similar api. golemproxy worked in unit/integration tests)

golemproxy is a local memcache proxy written in Golang. Like twemproxy, it supports pipelining and shards requests to multiple servers.

(Another difference is that that repo accepts bytes instead of str, but that was because it was a proxy for clients that send/receive non-utf8 data, such as serialized data)

(The gomemcache project appeared inactive when I'd worked on that support, so I hadn't created a ticket then)

[bradfitz]

Yes! Totally! This has been bugging me for 5+ years and is the reason why most PRs around connection management in this repo don't interest me too much; the core use of connections in this package just are wrong to begin with (not pipelined)

[TysonAndre]

Okay, I should have time later this week to work on this