Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FutureWarning issued when sf_api_query() function from project readme encounters an empty related field value #691

Open
chrisolof opened this issue Oct 20, 2023 · 0 comments · Fixed by #692
Open

FutureWarning issued when sf_api_query() function from project readme encounters an empty related field value #691

chrisolof opened this issue Oct 20, 2023 · 0 comments · Fixed by #692

Comments

@chrisolof
Copy link
Contributor

FutureWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
df = pd.concat([df.drop(columns=[col]),df[col].apply(pd.Series).drop('attributes',axis=1).add_prefix(col+'.')],axis=1)

I think this is generated whenever the related field is empty, leaving pd.Series with no data to infer a data type from.

It seems that passing the column's data type along with the current value to pd.Series in the apply() call removes the data type ambiguity, which also removes the warning.

PR to follow.
chrisolof added a commit to chrisolof/simple-salesforce that referenced this issue Oct 20, 2023
@chrisolof
Pass the column datatype along with current value to pd.Series
7fd129f 
Prevents pd.Series from issuing a FutureWarning when value is empty and
it is unable to infer a data type.

simple-salesforcegh-691
@chrisolof chrisolof mentioned this issue Oct 20, 2023
Merged
jon-wobken pushed a commit that referenced this issue Oct 24, 2023
@chrisolof
Pass the column datatype along with current value to pd.Series (#692)
40f093e 
Prevents pd.Series from issuing a FutureWarning when value is empty and
it is unable to infer a data type.

gh-691
jon-wobken added a commit that referenced this issue Feb 26, 2024
@jon-wobken @calbanese11
@chrisolof @rob-avestruz-kp @prazwalp007 @jsma @rokadias @jshadle-vectr @gdrosos @Shadow-Devil @framarzar
Release/v1.12.6 (#714)
c191451 
* fix bulk2 upload and download utf-8 encoding errors. (#688)

Co-authored-by: codyalbanese <calbanese@appannie.com>

* Pass the column datatype along with current value to pd.Series (#692)

Prevents pd.Series from issuing a FutureWarning when value is empty and
it is unable to infer a data type.

gh-691

* added max retries to recursive refresh token functionality (#685)

* remove pendulum depeendency and use native datetime

* adding query_all function to fetch soft deleted records in sf (#697)

Co-authored-by: Prajwal Pokharel <prajwalpokharel@ip-192-168-4-22.ec2.internal>

* Handle non-XML responses during error conditions (#695)

```
  File "/home/web/project/env/lib/python3.10/site-packages/simple_salesforce/api.py", line 146, in __init__
    self._refresh_session()
  File "/home/web/project/env/lib/python3.10/site-packages/simple_salesforce/api.py", line 283, in _refresh_session
    self.session_id, self.sf_instance = self._salesforce_login_partial()
  File "/home/web/project/env/lib/python3.10/site-packages/simple_salesforce/login.py", line 221, in SalesforceLogin
    return soap_login(soap_url, login_soap_request_body,
  File "/home/web/project/env/lib/python3.10/site-packages/simple_salesforce/login.py", line 231, in soap_login
    except_code = getUniqueElementValueFromXmlString(
  File "/home/web/project/env/lib/python3.10/site-packages/simple_salesforce/util.py", line 21, in getUniqueElementValueFromXmlString
    xmlStringAsDom = xml.dom.minidom.parseString(xmlString)
  File "/usr/lib/python3.10/xml/dom/minidom.py", line 2000, in parseString
    return expatbuilder.parseString(string)
  File "/usr/lib/python3.10/xml/dom/expatbuilder.py", line 925, in parseString
    return builder.parseString(string)
  File "/usr/lib/python3.10/xml/dom/expatbuilder.py", line 223, in parseString
    parser.Parse(string, True)
xml.parsers.expat.ExpatError: syntax error: line 1, column 0
```

Where `string` == "upstream connect error or disconnect/reset before headers. reset reason: connection termination"

* Adding utf-8 encoding when performing bulk2 data uploads. (#683)

* add functionality to return detailed results from batch request/result

* simplify worker variable from batch_results to result

* Update bulk2.py

* pass instance_url to login (#711)

* Require cryptography as optional dependency of pyjwt (#680)

* Update .travis.yml - remove support for 3.6 and 3.7

Remove support for python 3.6 and 3.7

* Update README.rst

Remove python 3.6 and 3.7
Add python 3.12

* Add typing annotations (#660)

* Add type annotations for util.py

* Add type annotations for metadata.py

* Fix up typing issues in metadata.py

* Add type annotations for login.py

* Fix up typing issues in login.py and add errorhandling

* Add type annotations for format.py

* Add type annotations for exceptions.py

* Fix up typing issues in login.py

* Add type annotations for bulk.py

* Fix up typing issues in bulk.py

* Add type annotations for api.py

* Fix up typing issues in api.py

* Fix up unittests

* Use MutableMapping instead of dict

* Fix formatting (long lines)

* Fix up wrong typing

* Fix up typing issues

* Fixup long lines

* Fixup typing issues in login.py

* Add mypy type checking to tox testenv:static script

* Use old type annotation syntax since python 3.6 doesn't support __future__ annotations

* Use old type annotation syntax

* Add py.typed to create a PEP 561 compatible package

* Add type alias for headers

* Fix unused import

* Add type alias for proxies

* Add TypeVar T for generic functions

* Add types for BulkDataAny and BulkDataStr

* Use standard HTTPMethod StrEnum

* Use standard HTTPMethod StrEnum

* Revert "Use standard HTTPMethod StrEnum"

This reverts commit b99d34a.

* Revert "Use standard HTTPMethod StrEnum"

This reverts commit d0d1d6e.

* Make Usage and PerAppUsage more type safe

* Fix Missing class docstring

* Add typing to bulk2 api

* Finish typing of bulk2

* Fixup

* Add future import and fixup DateTime annotation

* Maybe fix imports

* Move TypedDict import to typing_extensions

* Add dependency so also python 3.6 works

* Remove from future annotations

* Use older syntax

* Adjust pipeline to use py.test instead of setup.py test (since it is deprecated)

* Fixup test dependencies and use pytest instead of setup.py test

* Fixup python 3.6 error

* Fixup python 3.6 error

* Fix formatting and add pip cache

* Fix typing issues

---------

Co-authored-by: JonWobken <49450080+jon-wobken@users.noreply.github.com>

* Update api.py

Fix formatting
Update default api version

* Update bulk.py

Fix code format
Remove unused "operation" in _get_batch_request_with_batch_results

* Update bulk2.py

fix code formatting

* Update login.py

line to long fix

* Update api.py

* Fix CI for release/v1.12.6 (#713)

* Fix pipeline

* Fix pipeline

* Include "fields" as a query param in GET requests (#705)

* Include fields as a query param in get requests

* fix line length from CI build

* Refactor to pass **kwargs

Simplifies use to **kwargs so fields can be passed as "params='fields=field1,field2,field3'"

* Update api.py

* Update api.py

---------

Co-authored-by: JonWobken <49450080+jon-wobken@users.noreply.github.com>

* Update bulk.py

added detail result argument

* Update setup.py

* Update bulk.py

* Update test_requirements.txt

* Update test_requirements.txt

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

* Update CHANGES

---------

Co-authored-by: Cody Albanese <43307196+calbanese11@users.noreply.github.com>
Co-authored-by: codyalbanese <calbanese@appannie.com>
Co-authored-by: Chris Caldwell <chrisolof@gmail.com>
Co-authored-by: Rob O <140531024+rob-avestruz-kp@users.noreply.github.com>
Co-authored-by: prazwalp007 <prazwalp007@gmail.com>
Co-authored-by: Prajwal Pokharel <prajwalpokharel@ip-192-168-4-22.ec2.internal>
Co-authored-by: John-Scott Atlakson <24574+jsma@users.noreply.github.com>
Co-authored-by: Saqib Rokadia <saqib@saqibr.com>
Co-authored-by: jshadle-vectr <132278887+jshadle-vectr@users.noreply.github.com>
Co-authored-by: Giorgos Drosos <56369797+gdrosos@users.noreply.github.com>
Co-authored-by: Felix Schnabel <fschnabel007@gmail.com>
Co-authored-by: framarzar <86966421+framarzar@users.noreply.github.com>
jon-wobken added a commit that referenced this issue Mar 26, 2024
@jon-wobken @calbanese11
@chrisolof @rob-avestruz-kp @prazwalp007 @jsma @rokadias @jshadle-vectr @gdrosos @Shadow-Devil @framarzar
Release/v1.12.6 (#721)
a9ac1ab 
* fix bulk2 upload and download utf-8 encoding errors. (#688)

Co-authored-by: codyalbanese <calbanese@appannie.com>

* Pass the column datatype along with current value to pd.Series (#692)

Prevents pd.Series from issuing a FutureWarning when value is empty and
it is unable to infer a data type.

gh-691

* added max retries to recursive refresh token functionality (#685)

* remove pendulum depeendency and use native datetime

* adding query_all function to fetch soft deleted records in sf (#697)

Co-authored-by: Prajwal Pokharel <prajwalpokharel@ip-192-168-4-22.ec2.internal>

* Handle non-XML responses during error conditions (#695)

```
  File "/home/web/project/env/lib/python3.10/site-packages/simple_salesforce/api.py", line 146, in __init__
    self._refresh_session()
  File "/home/web/project/env/lib/python3.10/site-packages/simple_salesforce/api.py", line 283, in _refresh_session
    self.session_id, self.sf_instance = self._salesforce_login_partial()
  File "/home/web/project/env/lib/python3.10/site-packages/simple_salesforce/login.py", line 221, in SalesforceLogin
    return soap_login(soap_url, login_soap_request_body,
  File "/home/web/project/env/lib/python3.10/site-packages/simple_salesforce/login.py", line 231, in soap_login
    except_code = getUniqueElementValueFromXmlString(
  File "/home/web/project/env/lib/python3.10/site-packages/simple_salesforce/util.py", line 21, in getUniqueElementValueFromXmlString
    xmlStringAsDom = xml.dom.minidom.parseString(xmlString)
  File "/usr/lib/python3.10/xml/dom/minidom.py", line 2000, in parseString
    return expatbuilder.parseString(string)
  File "/usr/lib/python3.10/xml/dom/expatbuilder.py", line 925, in parseString
    return builder.parseString(string)
  File "/usr/lib/python3.10/xml/dom/expatbuilder.py", line 223, in parseString
    parser.Parse(string, True)
xml.parsers.expat.ExpatError: syntax error: line 1, column 0
```

Where `string` == "upstream connect error or disconnect/reset before headers. reset reason: connection termination"

* Adding utf-8 encoding when performing bulk2 data uploads. (#683)

* add functionality to return detailed results from batch request/result

* simplify worker variable from batch_results to result

* Update bulk2.py

* pass instance_url to login (#711)

* Require cryptography as optional dependency of pyjwt (#680)

* Update .travis.yml - remove support for 3.6 and 3.7

Remove support for python 3.6 and 3.7

* Update README.rst

Remove python 3.6 and 3.7
Add python 3.12

* Add typing annotations (#660)

* Add type annotations for util.py

* Add type annotations for metadata.py

* Fix up typing issues in metadata.py

* Add type annotations for login.py

* Fix up typing issues in login.py and add errorhandling

* Add type annotations for format.py

* Add type annotations for exceptions.py

* Fix up typing issues in login.py

* Add type annotations for bulk.py

* Fix up typing issues in bulk.py

* Add type annotations for api.py

* Fix up typing issues in api.py

* Fix up unittests

* Use MutableMapping instead of dict

* Fix formatting (long lines)

* Fix up wrong typing

* Fix up typing issues

* Fixup long lines

* Fixup typing issues in login.py

* Add mypy type checking to tox testenv:static script

* Use old type annotation syntax since python 3.6 doesn't support __future__ annotations

* Use old type annotation syntax

* Add py.typed to create a PEP 561 compatible package

* Add type alias for headers

* Fix unused import

* Add type alias for proxies

* Add TypeVar T for generic functions

* Add types for BulkDataAny and BulkDataStr

* Use standard HTTPMethod StrEnum

* Use standard HTTPMethod StrEnum

* Revert "Use standard HTTPMethod StrEnum"

This reverts commit b99d34a.

* Revert "Use standard HTTPMethod StrEnum"

This reverts commit d0d1d6e.

* Make Usage and PerAppUsage more type safe

* Fix Missing class docstring

* Add typing to bulk2 api

* Finish typing of bulk2

* Fixup

* Add future import and fixup DateTime annotation

* Maybe fix imports

* Move TypedDict import to typing_extensions

* Add dependency so also python 3.6 works

* Remove from future annotations

* Use older syntax

* Adjust pipeline to use py.test instead of setup.py test (since it is deprecated)

* Fixup test dependencies and use pytest instead of setup.py test

* Fixup python 3.6 error

* Fixup python 3.6 error

* Fix formatting and add pip cache

* Fix typing issues

---------

Co-authored-by: JonWobken <49450080+jon-wobken@users.noreply.github.com>

* Update api.py

Fix formatting
Update default api version

* Update bulk.py

Fix code format
Remove unused "operation" in _get_batch_request_with_batch_results

* Update bulk2.py

fix code formatting

* Update login.py

line to long fix

* Update api.py

* Fix CI for release/v1.12.6 (#713)

* Fix pipeline

* Fix pipeline

* Include "fields" as a query param in GET requests (#705)

* Include fields as a query param in get requests

* fix line length from CI build

* Refactor to pass **kwargs

Simplifies use to **kwargs so fields can be passed as "params='fields=field1,field2,field3'"

* Update api.py

* Update api.py

---------

Co-authored-by: JonWobken <49450080+jon-wobken@users.noreply.github.com>

* Update bulk.py

added detail result argument

* Update setup.py

* Update bulk.py

* Update test_requirements.txt

* Update test_requirements.txt

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

* Update CHANGES

* Update __version__.py

---------

Co-authored-by: Cody Albanese <43307196+calbanese11@users.noreply.github.com>
Co-authored-by: codyalbanese <calbanese@appannie.com>
Co-authored-by: Chris Caldwell <chrisolof@gmail.com>
Co-authored-by: Rob O <140531024+rob-avestruz-kp@users.noreply.github.com>
Co-authored-by: prazwalp007 <prazwalp007@gmail.com>
Co-authored-by: Prajwal Pokharel <prajwalpokharel@ip-192-168-4-22.ec2.internal>
Co-authored-by: John-Scott Atlakson <24574+jsma@users.noreply.github.com>
Co-authored-by: Saqib Rokadia <saqib@saqibr.com>
Co-authored-by: jshadle-vectr <132278887+jshadle-vectr@users.noreply.github.com>
Co-authored-by: Giorgos Drosos <56369797+gdrosos@users.noreply.github.com>
Co-authored-by: Felix Schnabel <fschnabel007@gmail.com>
Co-authored-by: framarzar <86966421+framarzar@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Pass the column datatype along with current value to pd.Series chrisolof/simple-salesforce
1 participant
@chrisolof