Drop duplicate rows before returning output df

[jcweaver]

I'm not sure if this is expected or not but when running pred_fl_full_name, pred_census_last_name, & pred_fl_last_name using ethnicolr2, I get returned dataframes with duplicate rows. I don't think these duplicate rows should be added.

[rajashekar]

@jcweaver - I verified with below code, I am not seeing duplicate rows, can you please share minimal code to replicate?

names = [
    {"last": "hernandez", "first": "hector"},
    {"last": "zhang", "first": "simon"},
]
df = pd.DataFrame(names)
df["fullname"] = self.df["last"] + " " + self.df["first"]
odf = pred_fl_full_name(self.df, full_name_col="fullname")
pd.set_option("display.max_colwidth", None)
print(odf)

output -

        last   first  ...     preds                                                                                                                    probs
0  hernandez  hector  ...  hispanic  {'asian': 0.0003889082, 'hispanic': 0.987706, 'nh_black': 0.0012073584, 'nh_white': 0.008855153, 'other': 0.0018425211}
1      zhang   simon  ...     asian   {'asian': 0.936533, 'hispanic': 0.0016647797, 'nh_black': 0.0012903506, 'nh_white': 0.011221227, 'other': 0.049290624}