BUG: FutureWarning appears when assigning a bool list through .at

[WILDADF]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

data_BDlist = pd.DataFrame({
    "postUrl": [None],
    "postTitle": [None],
    "BDurl": [None],
    "BDvalid": [False],
})

for k in range(len(data_BDlist)):
    print("row: ",k)
    if (len(urls) > 0): 
        data_BDlist.at[k, 'BDurl'] = urls
        data_BDlist.at[k, 'BDvalid'] = arrcheck(urls) #

Issue Description

similar to this issue but I'm not sure

arrcheck(urls) is a function that returns a array of bool
usually returns [True]

python -u "main.py"
row:  0
row:  1
row:  2
row:  3
main.py:114: FutureWarning: Setting an item of incompatible dtype is deprecated and will raise an error in a future version of pandas. Value 'True' has dtype incompatible with float64, please explicitly cast to a compatible dtype first.
  data_BDlist.at[k, 'BDvalid'] = arrcheck(urls)
row:  4
row:  5

result in first 6 run

postUrl	postTitle	BDurl	BDvalid
--------	----------	----------	-------
postUrl0	postTitle0	nan
postUrl1	postTitle1	nan
postUrl2	postTitle2	nan
postUrl3	postTitle3	BDurl-arr0	True
postUrl4	postTitle4	BDurl-arr1	[True]
postUrl5	postTitle5	BDurl-arr2	[True]

Expected Behavior

No FutureWarning raised.

result in first 6 run

postUrl	postTitle	BDurl	BDvalid
--------	----------	----------	-------
postUrl0	postTitle0	nan
postUrl1	postTitle1	nan
postUrl2	postTitle2	nan
postUrl3	postTitle3	BDurl-arr0	[True]
postUrl4	postTitle4	BDurl-arr1	[True]
postUrl5	postTitle5	BDurl-arr2	[True]

Installed Versions

INSTALLED VERSIONS

commit : d9cdd2e
python : 3.10.11.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19045
machine : AMD64
processor : Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : Chinese (Traditional)_Taiwan.950

pandas : 2.2.2
numpy : 1.26.4
pytz : 2024.1
dateutil : 2.9.0.post0
setuptools : 65.5.0
pip : 24.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : 8.24.0
pandas_datareader : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : 4.12.3
bottleneck : None
dataframe-api-compat : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2024.1
qtpy : None
pyqt5 : None

[Aloqeely]

Thanks for the report! I don't get any warning when running your code sample and replacing arrcheck(urls) with [True].
Could you provide a minimal bug report that someone can run without the need of any of your internal variables (i.e. without urls and arrcheck(urls))

[WILDADF]

I'm not sure is it a minimalist bug report but here the best I can

import pandas as pd
import numpy as np

data_BDlist = pd.DataFrame({
    "postUrl": ["postUrl0", "postUrl1", "postUrl2"],
    "postTitle": ["postTitle0", "postTitle1", "postTitle2"],
})

data_BDlist.to_csv('data_BDlist_tset.csv', index=False)


def check(url) -> bool:
    return (url == "True")

def arrcheck(urls):
    bool_arr = []
    for url in urls:
        bool = check(url)
        bool_arr.append(bool)
    return bool_arr


arr_urls = [[], ["True"], ["True"]]


data_BDlist.to_csv('data_BDlist_tset.csv', index=False)

for k in range(len(data_BDlist)):
    print("row: ",k)
    urls = arr_urls[k]
    print(type(urls),urls)
    if (len(urls) > 0): 
        data_BDlist.at[k, 'BDurl'] = urls
        data_BDlist.at[k, 'BDvalid'] = arrcheck(urls)

data_BDlist.to_csv('data_BDlist_tset.csv', index=False)

postUrl	postTitle	BDurl	BDvalid
postUrl0	postTitle0	nan
postUrl1	postTitle1	['True']	True
postUrl2	postTitle2	['True']	[True]

and I'm also find a way to fix it here is

data_BDlist = pd.DataFrame({
    "postUrl": ["postUrl0", "postUrl1", "postUrl2"],
    "postTitle": ["postTitle0", "postTitle1", "postTitle2"],
    "BDurl": [None, None, None],
    "BDvalid": [None, None, None],
})

is totally my mistake

anyway thank you so much

[WILDADF]

by the way, is it correct for the type change (list->bool)

[Aloqeely]

Which type change?

[WILDADF]

postUrl	postTitle	BDurl	BDvalid
postUrl0	postTitle0	nan
postUrl1	postTitle1	['True']	True
postUrl2	postTitle2	['True']	[True]

it's supposed to be "list"

[Aloqeely]

I'm not sure, it might be related to this: https://stackoverflow.com/questions/77098113/solving-incompatible-dtype-warning-for-pandas-dataframe-when-setting-new-column
TL;DR: You should create the column before assigning the first value to it with .at, as that will make pandas think the column has a float64 value, (because np.nan is treated as float)