Route choice API, link loading, path file generation

[Jake-Moss]

TODO:

• Basic path file computation
• Basic link flow computation
• Adjust existing tests to new API
• API solidification, it should mirror existing APIs. I think I'll go with a thin wrapper around the existing route choice object
• Support multiple matrix slices. Current implementation of link loading assumes a single slice, this isn't difficult to change
• Separate path file generation and link loading for now. Currently the link loading implementation uses the path files as an intermediate result
• Map all outputs back to network IDs, currently everything is in compact form
• Saving of sparse path files into parquet
• Saving of dense and sparse link flows into results database
• New tests for user facing class
• General code quality improvements, some methods are deeply intertwine with each other and generally too long, particularly the batched method, which is 252 lines
• Documentation example

[pedrocamargo]

@janzill , can you take a look at this PR sometime before next Tuesday? That would allow Jake to jump straight into it

[pedrocamargo]

@Jake-Moss , the API looks pretty good. I think that with some minor tweaks and the example notebook, it will be perfect.

[Jake-Moss]

Didn't realise dict | dict was a 3.9 feature, I'll fix that next week

[pedrocamargo]

Didn't realise dict | dict was a 3.9 feature, I'll fix that next week

We no longer support Python 3.8, though

[pedrocamargo]

@Jake-Moss , in order to save choice sets to disk, we would need to either save the relationship between compact graph link and link/direction or convert it to link_id/direction.

Compact graphs are not permanent, so this becomes dangerous when saving to disk

[Jake-Moss]

@Jake-Moss , in order to save choice sets to disk, we would need to either save the relationship between compact graph link and link/direction or convert it to link_id/direction.

Compact graphs are not permanent, so this becomes dangerous when saving to disk

What file is that from? I can't see it on the latest commit, I thought that doc string was modified, either way any doc string that mentions the compact links is out of date, except apply_link_loading, which is used by get_load_results to return both

aequilibrae/aequilibrae/paths/route_choice.py

Lines 330 to 361 in f55eac9

	if uncompressed:
	uncompressed_df = pd.DataFrame(
	{"link_id": lids}
	| {k + dir: np.zeros(lids.shape) for k in self.link_loads.keys() for dir in ["_ab", "_ba"]}
	)
	for k, v in self.link_loads.items():
	# Directional Flows
	uncompressed_df[k + "_ab"].values[m.network_ab_idx] = np.nan_to_num(v[m.graph_ab_idx])
	uncompressed_df[k + "_ba"].values[m.network_ba_idx] = np.nan_to_num(v[m.graph_ba_idx])
	# Tot Flow
	uncompressed_df[k + "_tot"] = np.nan_to_num(uncompressed_df[k + "_ab"].values) + np.nan_to_num(
	uncompressed_df[k + "_ba"].values
	)
	if compressed:
	compressed_df = pd.DataFrame(
	{"link_id": compact_lids}
	| {
	k + dir: np.zeros(compact_lids.shape)
	for k in self.compact_link_loads.keys()
	for dir in ["_ab", "_ba"]
	}
	)
	for k, v in self.compact_link_loads.items():
	compressed_df[k + "_ab"].values[m_compact.network_ab_idx] = np.nan_to_num(v[m_compact.graph_ab_idx])
	compressed_df[k + "_ba"].values[m_compact.network_ba_idx] = np.nan_to_num(v[m_compact.graph_ba_idx])
	# Tot Flow
	compressed_df[k + "_tot"] = np.nan_to_num(compressed_df[k + "_ab"].values) + np.nan_to_num(
	compressed_df[k + "_ba"].values
	)

and the translation for the results table happens during its construction

aequilibrae/aequilibrae/paths/route_choice_set.pyx

Lines 1173 to 1182 in f55eac9

	for link in deref(route):
	# Translate the compressed link IDs in route to network link IDs, this is a 1:n mapping
	network_link_begin = self.mapping_idx[link]
	network_link_end = self.mapping_idx[link + 1]
	path_builder.AppendValues(
	&self.mapping_data[network_link_begin],
	network_link_end - network_link_begin
	)
	offset += network_link_end - network_link_begin

[Jake-Moss]

I believe this is largely ready now, it is lacking select link analysis support, though that will come in a separate PR

[janzill]

looks great, one small suggestion regarding testing assignment

[Jake-Moss]

Hmm I thought the CI was fixed...

[pedrocamargo]

Latest Mac-os is now M1...

[Jake-Moss]

Latest Mac-os is now M1...

Hmm that'll do it

[Jake-Moss]

Latest Mac-os is now M1...

I'm not sure what's going on now, the logs files said that the system was build a x86_64 wheel running on arm but deallocate doesn't seem have had an update in the last couple months. I wouldn't have expected that to break it. We'll see how this CI runs, testing locally gave me nothing to go on.

It doesn't seem to be respecting the `--ignore-missing-dependencies``