joint_tour_frequency_composition column to categorical

[i-am-sijia]

joint_tour_frequency_composition column currently is a string, this PR converts it to pandas categorical.

[jpn--]

It looks like the "alt" column of the joint_tour_frequency_composition_alternatives.csv input file (at least for the SANDAG demo model) is integers. The .reindex here is potentially leaving null values, which promotes the choice column to float (at least for pandas 1.x), then we .fillna which converts the NaNs to 0. I'm not sure why the last step here is converting the string when a convert to integer would be fine. I don't think we want this to end up as categorical with the category labels as integers, that's confusing. Am I missing something?

[i-am-sijia]

That is true. We could just keep this column as int. I don't recall why this is converted to str in the last step. I just checked the ABM3 configs, it's not using this column in downstream UECs, so we should be able to actually change this to int without any impact downstream.

However, if we just do .astype(int) here, by default it will use int64, which uses more memory than if we convert it to pandas categorical. I also feel grudge against something like a hardcoded .astype(int8).

[i-am-sijia]

How about something like

households_merged["joint_tour_frequency_composition"] = pd.to_numeric(
    choices.reindex(households_merged.index).fillna(no_tours_alt),
    downcast="integer"
)