Handle split_out=None deprecation warning to drop_duplicates

[charlesbluca]

Looks like we didn't add handling for split_out=None with #10542, causing us to get an error like:

shuffle = None, split_out = None

    def _determine_split_out_shuffle(shuffle, split_out):
        """Determine the default shuffle behavior based on split_out"""
        if shuffle is None:
>           if split_out > 1:
E           TypeError: '>' not supported between instances of 'NoneType' and 'int'

/datasets/charlesb/micromamba/envs/dask-sql-py39/lib/python3.9/site-packages/dask/dataframe/core.py:202: TypeError

This PR adds a warning for the deprecated use and falls back to the default of split_out=1

cc @rjzamora

• Closes #xxxx
• Tests added / passed
• Passes pre-commit run --all-files

[rjzamora]

Thanks for the fix @charlesbluca ! Sorry for missing this.

It looks like the original split_out=None deprecation warning was added to groupby.agg more than a year ago (#9491). Unfortunately, it seems like we have not been warning for other APIs (like drop_duplicates).

Questions:

1. Should we try to find a simple way to add a temporary deprecation warning for all APIs using split_out?
2. Should we just move ahead with the deprecation now, and start raising a clear TypeError in _determine_split_out_shuffle?

• Related question: Is there a reason that dask-sql needs to use split_out=None for now? If so, how much time do you think is needed to address the deprecation?

[charlesbluca]

Should we try to find a simple way to add a temporary deprecation warning for all APIs using split_out?

I think that makes sense - it seems like we have a few unified utility functions used to check and control shuffle/split_out, maybe we can move this logic to one of those functions so we don't have to deal with the burden of adding a unique check to each relevant API?

Related question: Is there a reason that dask-sql needs to use split_out=None for now? If so, how much time do you think is needed to address the deprecation?

Though I caught this in dask-sql's test suite, don't think this should require much work to address - this was just a single test asserting some behavior based on split_out that happened to use the old default. I'm generally okay with moving forward with the deprecation, as it has been a while, though maybe other maintainers have context on why we haven't moved forward yet (cc @jrbourbeau)