move from robin-hood-hashing to unordered_dense library

[nilsnolde]

fixes #4551

I'll do a performance test tmrw when I have the full planet graph available.

[nilsnolde]

I benchmarked a bunch of containers in terms of performance. RAM is not so important for the CostMatrix structure, it's mostly a bunch of int64.

test setup:

• only changed this line to control which container is used:

  valhalla/src/thor/costmatrix.cc

  Line 48 in 453d3ef

  class CostMatrix::ReachedMap : public robin_hood::unordered_map<uint64_t, std::vector<uint32_t>> {};

• each row had a file with 20 random requests, either in berlin bbox or spanning mannheim/usedom (see bbox)
• server was long running, the times reported are averages over 3-4 runs and in seconds

	master	unordered_dense::segmented_map	unordered_dense::map	absl::flat_hash_map
berlin, 20 sources/targets	36	38	39	36
berlin, 50 sources/targets	118	126	124	119
mannheim-usedom, 20 sources/targets	210	220	215	215

all in all it hardly matters, though unordered_dense::segmented_map is slightly worse than the rest, which makes sense, it's adding a layer of indirection by segmenting the map.

I'd almost advocate to move to absl::flat_hash_map. it's a transitive dependency already (protobuf needs it), it's available in all package managers and we could remove one vendored dependency.

thoughts?

[nilsnolde]

it's a transitive dependency already (protobuf needs it)

hm, kinda forgot that only protobuf > 21.12 needs absl.. most major distros/package managers seems to have locked themselves to the pre-abseil protobuf version.. still, I highly doubt that protobuf will remove that dependency again, no matter how much pain it's inflicting on packagers. but that makes my main point moot..

[danpat]

RAM is not so important for the CostMatrix structure, it's mostly a bunch of int64.

Said like someone who hasn't made a 10k x 10k matrix before :-)

[nilsnolde]

With Valhalla? Hell no😄 Point was more that the affected structure is drowning compared to stuff like EdgeLabel which is 7-8 times as big.

[nilsnolde]

agreed offline to go with abseil instead of another third_party vendored library.

do note: abseil is not available via apt for ubuntu < 22.04. IMO that's fair, it can still be installed from source.

this PR should be good for review @kevinkreiser.

[nilsnolde]

Ouch.. Getting lots of ld.lld: error: undefined symbol: absl::debian3::hash_internal::MixingHashState::kSeed and related linker errors on ubuntu 22.04. Don't really understand why.. All other platforms (and my own arch linux) work fine. I'm kinda fearing that the abseil package for 22.04 wasn't built for C++17..

[kevinkreiser]

yeah i was just about to make this point. by switching to this dependency we no longer allow people to compile on certain platforms which really sucks when there is a perfectly fine header only thirdparty dep. remind why we dont like robinhood again?

[nilsnolde]

my reasons were:

• remove more vendored stuff in favor of packaged stuff, in turn mostly to reduce our own packaging requirements/"attack surface" for apt/rpm/pkgbuild repos
• it will bitrot over time and if not c++23, then maybe c++2x would break smth in robin-hood (std headers moving around etc), at which point we'd move or maintain it ourselves (pretty low effort admittedly)

I admit none of them are pressing reasons to fix what ain't broken (yet). I'm in cleanup mode, this was just a (rather far reaching) part of it:)

let's keep it here for now and not do anything about it until the time comes for any of the above reasons. I don't really want to investigate what abseil is messing up internally;)