Dumping a Tweet's JSON .data field sometimes reports "TypeError: Object of type datetime is not JSON serializable"

[wls]

Summary

Non-JSON serializeable field in supposed JSON object.

Reproduction Steps

According to Tweepy documentation about the Tweet class's .data field, it says this field is "The JSON data representing the Tweet."

That means, this .data object should be serializable by Python's json module, encoding it as a JSON string.

---

When the edit_controls field is passed to as a tweet_field to .get_users_tweets like so:

tweepy_client.get_users_tweets(twitterid.data.id, ..., tweet_field=[...,"edit_controls",...])

The object contains a datetime.datetime object, which results in a "TypeError: Object of type datetime is not JSON serializable" error message.

Removing the "edit_controls" causes this field not to be added to the Tweet object, and the serialization a JSON string works.

---

Note, a similar field in the object, created_at, works just fine, as it holds the date/timestamp as a string, not a datetime.datetime object.

Minimal Reproducible Example

import tweepy
import json

client = tweepy.Client(PUT_YOUR_CREDENTIALS_HERE)

response = client.get_users_tweets("8414992",
                                 max_results=5,  # Must be between 5 and 100 
                                 since_id = "1698369403052532153",
                                 until_id = "1698369403052532155",
                                 
                                 # Removing "edit_controls" works
                                 tweet_fields=["created_at","edit_controls"], # <--
                                 )
print(response.data[0].data)
print(json.dumps(response.data[0].data)) # Fails when edit_controls is present

Expected Results

I expected to have a JSON string of the Tweet's .data field produce this serialized object:

{
    "created_at": "2023-09-03T16:16:34.000Z",
    "id": "1698369403052532154",
    "text": "@MsBelleMae @darkpatterns @affinitybyserif Wow, I have found the exact opposite to be true \u2014 what happened?\n\nThis does not sound like the Affinity I\u2019ve been using for years.",
    "edit_history_tweet_ids": [
        "1698369403052532154"
    ],
    "edit_controls": {
       "edits_remaining": 5,
       "is_edit_eligible": false,
       "editable_until": "2023-09-03T17:16:34.000Z"
    }
}

NOTE: the editable_until fields should be a string, easily serialized by JSON

Actual Results

Program reported an error: TypeError: Object of type datetime is not JSON serializable

Here's what the object dump effectively looks like (notice the datetime.datetime() entry):

{
  'created_at': '2023-09-03T16:16:34.000Z', 
  'id': '1698369403052532154', 
  'edit_history_tweet_ids': ['1698369403052532154'], 
  'text': '@MsBelleMae @darkpatterns @affinitybyserif Wow, I have found the exact opposite to be true — what happened?\n\nThis does not sound like the Affinity I’ve been using for years.',
  'edit_controls': {
     'edits_remaining': 5,
     'is_edit_eligible': False,
     'editable_until': datetime.datetime(2023, 9, 3, 17, 16, 34, tzinfo=datetime.timezone.utc)
  }
}

NOTE: The editable_util field is a datetime.datetime object, while the created_at field is a string!

Twitter API Access Plan

Basic

Tweepy Version

v4.14.0

Checklist

• I have searched for duplicate issues.
• If applicable, I have shown the entire traceback.
• If applicable, I have removed any visible credentials from my code and/or screenshots.

Additional Context

I think the issue is in tweepy/tweet.py in this section of code, however created_at is working and editable_until is not. It's not immediately obvious as to why, but at debugger confirms it.

Something is coercing the assigned datetime.datetime object back to a string in one case and not in the other.

        self.created_at = data.get("created_at")
        if self.created_at is not None:
            self.created_at = parse_datetime(self.created_at)

        self.edit_controls = data.get("edit_controls")
        if self.edit_controls is not None:
            self.edit_controls["edits_remaining"] = int(
                self.edit_controls["edits_remaining"]
            )
            self.edit_controls["editable_until"] = parse_datetime(
                self.edit_controls["editable_until"]
            )

In tracing through the code, data.get("created_at") has a value of "2023-09-03T16:16:34.000Z", and data.get("edit_controls") has a value of "2023-09-03T17:16:34.000Z".

While self.created_at = ends up being a string, when assigned parse_datetime's value of datetime.datetime(2023, 9, 3, 16, 16, 34, tzinfo=datetime.timezone.utc).

The value of self.edit_controls["editable_until"] = (which appears to be an entry in a sub dictionary) ends up being a datetime.datetime object, when assigned a value of datetime.datetime(2023, 9, 3, 17, 16, 34, tzinfo=datetime.timezone.utc).

[wls]

Note, as an experiment, commented out (v4.14.0's lines 197-199 in Tweepy's tweet.py) and the JSON code ran just fine with edit_controls present:

            # self.edit_controls["editable_until"] = parse_datetime(
            #    self.edit_controls["editable_until"]
            # )

I hesitate to do this, as I don't know what problem adding those lines fixed and don't want to break any code dependent on that behavior.