Accessing blob.custom_time errors on 0 microsecond timestamp

[rconnaughton]

Environment details

• OS type and version: centos 6
• Python version: 3.7
• google-cloud-storage version: 1.34.0

Steps to reproduce

1. Load a blob
2. Set the custom_time field to a datetime object with 0 microseconds
3. patch the object with the changes, and then reload the object
4. Access the custom_time field and observe the value error

Code example

blob = Blob(mybucket, objectname)
blob.reload()
blob.custom_time = datetime(year=2000, month = 1, day = 13, hour = 13, minute = 1, second = 0, microsecond = 0)
blob.patch()
blob.reload()
blob.custom_time
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 3773, in custom_time
    return _rfc3339_to_datetime(value)_rfc3339_to_datetime
  File "/usr/local/lib/python3.7/site-packages/google/cloud/_helpers.py", line 286, in _rfc3339_to_datetime
    return datetime.datetime.strptime(dt_str, _RFC3339_MICROS).replace(tzinfo=UTC)
  File "/usr/local/lib/python3.7/_strptime.py", line 577, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/local/lib/python3.7/_strptime.py", line 359, in _strptime
    (data_string, format))
ValueError("time data '2000-01-13T13:01:00Z' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'")

As compared to a working example:

blob.custom_time = datetime(year=2000, month = 1, day = 13, hour = 14, minute = 2, second = 0, microsecond = 10)
blob.patch()
blob.reload()
blob.custom_time
datetime.datetime(2000, 1, 13, 14, 2, 0, 10, tzinfo=<UTC>)

Stack trace

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/site-packages/google/cloud/storage/blob.py", line 3773, in custom_time
    return _rfc3339_to_datetime(value)_rfc3339_to_datetime
  File "/usr/local/lib/python3.7/site-packages/google/cloud/_helpers.py", line 286, in _rfc3339_to_datetime
    return datetime.datetime.strptime(dt_str, _RFC3339_MICROS).replace(tzinfo=UTC)
  File "/usr/local/lib/python3.7/_strptime.py", line 577, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/local/lib/python3.7/_strptime.py", line 359, in _strptime
    (data_string, format))
ValueError("time data '2000-01-13T13:01:00Z' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'")

[frankyn]

Hi @rconnaughton,

Thank you for filing this issue. I was able to reproduce it and looks like Cloud Storage returns no fractional microsecond when it's 0 even though the format described in documentation is YYYY-MM-DD'T'HH:MM:SS.SS'Z'.

The issue with the client is that it always expects the returned value to include microseconds. I'm following up with the backend team to understand if this is the expectation.

I'll loop back when I learn more. Thank you for your patience!

[frankyn]

This is a bug in the library. The Storage backend team stated that when milliseconds is 0 the 0 is not returned.

I'll work on a fix this week. It will require two PRs based on my current understanding, there might be a better solution though and will research / update:

1. Add a new format to python-cloud-core.
2. Use the new format in case the format parsing with non-zero milliseconds is included in python-storage.