Blob's custom_time is lost on upload.

[oittaa]

Environment details

• OS type and version: Debian Buster
• Python version: Python 3.7.3
• pip version: pip 18.1 from /usr/lib/python3/dist-packages/pip (python 3.7)
• google-cloud-storage version: 1.35.1

Steps to reproduce

1. Create a blob.
2. Set custom_time.
3. Upload the blob .
4. Try to retrieve blob's custom_time.
5. Expected result is a datatime, but the actual result is None.

If blob's custom_time is set before uploading, it is just lost. My understanding is that the metadata could be uploaded with the object's content at the same time without needing to patch it afterwards.

https://cloud.google.com/storage/docs/uploading-objects#rest-upload-objects

Code example

import datetime
from google.cloud import storage

bucket_name = "my-test-bucket"
blob_name = "test_blob"

client = storage.Client()
bucket = client.get_bucket(bucket_name)

blob = bucket.blob(blob_name)
blob.custom_time = datetime.datetime.now(datetime.timezone.utc)
print("Before upload {}".format(blob.custom_time))
blob.upload_from_string("test")
blob.reload()
print("After upload {}".format(blob.custom_time))
blob.custom_time = datetime.datetime.now(datetime.timezone.utc)
blob.patch()
blob.reload()
print("After patching {}".format(blob.custom_time))

Stack trace

Before upload 2021-02-05 10:24:08.405494+00:00
After upload None
After patching 2021-02-05 10:24:08.814159+00:00

[oittaa]

For example content_encoding works as expected.

Code

from google.cloud import storage

bucket_name = "my-test-bucket"
blob_name = "test_blob"

client = storage.Client()
bucket = client.get_bucket(bucket_name)

blob = bucket.blob(blob_name)
blob.content_encoding = "gzip"
print("Before upload {}".format(blob.content_encoding))
blob.upload_from_string("test")
blob.reload()
print("After upload {}".format(blob.content_encoding))

Output

Before upload gzip
After upload gzip

Is the reason that customTime is missing from _WRITABLE_FIELDS? EDIT: Yes it is. Adding "customTime", line fixed the issue.

python-storage/google/cloud/storage/blob.py

Lines 92 to 103 in 5c60d24

	_WRITABLE_FIELDS = (
	"cacheControl",
	"contentDisposition",
	"contentEncoding",
	"contentLanguage",
	_CONTENT_TYPE_FIELD,
	"crc32c",
	"md5Hash",
	"metadata",
	"name",
	"storageClass",
	)

Which is used by _get_writable_metadata.

python-storage/google/cloud/storage/blob.py

Lines 1518 to 1547 in 5c60d24

	def _get_writable_metadata(self):
	"""Get the object / blob metadata which is writable.
	This is intended to be used when creating a new object / blob.
	See the `API reference docs`_ for more information, the fields
	marked as writable are:
	* ``acl``
	* ``cacheControl``
	* ``contentDisposition``
	* ``contentEncoding``
	* ``contentLanguage``
	* ``contentType``
	* ``crc32c``
	* ``md5Hash``
	* ``metadata``
	* ``name``
	* ``storageClass``
	For now, we don't support ``acl``, access control lists should be
	managed directly through :class:`ObjectACL` methods.
	"""
	# NOTE: This assumes `self.name` is unicode.
	object_metadata = {"name": self.name}
	for key in self._changes:
	if key in _WRITABLE_FIELDS:
	object_metadata[key] = self._properties[key]
	return object_metadata

And which is then used by _get_upload_arguments.

python-storage/google/cloud/storage/blob.py

Lines 1549 to 1571 in 5c60d24

	def _get_upload_arguments(self, content_type):
	"""Get required arguments for performing an upload.
	The content type returned will be determined in order of precedence:
	- The value passed in to this method (if not :data:`None`)
	- The value stored on the current blob
	- The default value ('application/octet-stream')
	:type content_type: str
	:param content_type: Type of content being uploaded (or :data:`None`).
	:rtype: tuple
	:returns: A triple of
	* A header dictionary
	* An object metadata dictionary
	* The ``content_type`` as a string (according to precedence)
	"""
	headers = _get_encryption_headers(self._encryption_key)
	object_metadata = self._get_writable_metadata()
	content_type = self._get_content_type(content_type)
	return headers, object_metadata, content_type

[tritone]

I was able to reproduce this and discovered custom_time was missing from attributes to set during blob creation. Will send a PR with a fix.