Using S3Client->headObject returns ObjectLockRetainUntilDate even when Object Lock is disabled on bucket

[tomfotherby]

Version of PHP (php -v)?

7.3.9

What issue did you see?

When I use headObject() from the PHP SDK, the ObjectLockRetainUntilDate value is output even though the bucket has Object Lock Disabled:

$S3 = \Aws\S3\S3Client::factory($config);
$headers = $S3->headObject(array(
    'Bucket' => '***',
    'Key' => 'test.txt'
))->toArray();
print_r($headers);

Output:

Array
(
    [DeleteMarker] => 
    [AcceptRanges] => bytes
    [Expiration] => 
    [Restore] => 
    [LastModified] => Aws\Api\DateTimeResult Object
        (
            [date] => 2016-04-07 12:25:51.000000
            [timezone_type] => 2
            [timezone] => GMT
        )

    [ContentLength] => 25
    [ETag] => "9152d7f1724ed8fbcd2e0c87029f193c"
    [MissingMeta] => 
    [VersionId] => 
    [CacheControl] => 
    [ContentDisposition] => 
    [ContentEncoding] => 
    [ContentLanguage] => 
    [ContentType] => text/plain
    [Expires] => Aws\Api\DateTimeResult Object
        (
            [date] => 2019-11-12 09:14:26.336281
            [timezone_type] => 3
            [timezone] => UTC
        )

    [WebsiteRedirectLocation] => 
    [ServerSideEncryption] => 
    [Metadata] => Array
        (
        )

    [SSECustomerAlgorithm] => 
    [SSECustomerKeyMD5] => 
    [SSEKMSKeyId] => 
    [StorageClass] => 
    [RequestCharged] => 
    [ReplicationStatus] => 
    [PartsCount] => 
    [ObjectLockMode] => 
    [ObjectLockRetainUntilDate] => Aws\Api\DateTimeResult Object
        (
            [date] => 2019-11-12 09:14:26.336305
            [timezone_type] => 3
            [timezone] => UTC
        )

    [ObjectLockLegalHoldStatus] => 
    [@metadata] => Array
        (
            [statusCode] => 200
            [effectiveUri] => https://redacted.s3.amazonaws.com/test.txt
            [headers] => Array
                (
                    [x-amz-id-2] => PU6fM5xwlLmzBimv2redactedTv3V0sG2yjxKGbz8l7yTTZxMta0Cxo00=
                    [x-amz-request-id] => F65EE1DC5435C72E
                    [date] => Tue, 12 Nov 2019 09:14:27 GMT
                    [last-modified] => Thu, 07 Apr 2016 12:25:51 GMT
                    [etag] => "9152d7f1724ed8fbcd2e0c87029f193c"
                    [accept-ranges] => bytes
                    [content-type] => text/plain
                    [content-length] => 25
                    [server] => AmazonS3
                )

            [transferStats] => Array
                (
                    [http] => Array
                        (
                            [0] => Array
                                (
                                )
                        )
                )
        )
)

Note, ObjectLockMode and ObjectLockLegalHoldStatus are null but ObjectLockRetainUntilDate is set:

    [ObjectLockRetainUntilDate] => Aws\Api\DateTimeResult Object
        (
            [date] => 2019-11-12 09:14:26.336305
            [timezone_type] => 3
            [timezone] => UTC
        )

Additional context

I ran into this issue because we use copyObject(), suppling Metadata for the sink object by fetching it from the source object using headObject(). This means, after upgrading our SDK we suddenly started trying to set the ObjectLockRetainUntilDate for the sink object. When Object Lock is disabled for a bucket, if you provide ObjectLockRetainUntilDate in the Metadata the copyObject throws a error:

AWS HTTP error: Client error response 
[url] https://redacted.s3.amazonaws.com/redacted2.txt 
[status code] 400 
[reason phrase] Bad Request InvalidRequest (client): Bucket is missing ObjectLockConfiguration

The workaround is to remove the ObjectLockRetainUntilDate from the headObject output, e.g.

        $objOptions = $s3Client()->headObject(['Key' => $key,'Bucket' => $bucket]);
        $arr = $objOptions->toArray();
        unset($arr['ObjectLockRetainUntilDate']);
        return array_filter($arr);

I feel it would be more logical to not have a ObjectLockRetainUntilDate output if Object Lock is disabled on the bucket. Close this issue if you dissagree, no worries.

Version of AWS SDK for PHP?

This issue happens from 3.76.0 and higher (tested up to 3.115.1). The CHANGELOG for 3.76.0 says:

S3 Object Lock enables customers to apply Write Once Read Many (WORM) protection to objects in S3 in order to prevent object deletion for a customer-defined retention period.

From 3.75.0 and lower headObject does not output ObjectLockRetainUntilDate ✔️

[diehlaws]

Hi @tomfotherby, thanks for reaching out to us about this and for the detailed issue description. The behavior you're observing is caused by a combination of the following:

1. The AWS SDK for PHP will include all modeled members of a given response object, even if they are empty.
2. ObjectLockRetainUntilDate and Expires are both DateTimeResult objects which cannot currently be initialized as null values.

DateTime objects are initialized using the time at which the object is initialized, and since there is no ObjectLockRetainUntilDate value for S3 to return in the headObject call for your object, the SDK initializes this value along with the Expires value using a DateTime object with the time at which the response was received.

Unfortunately, changing either of these behaviors would result in a breaking change for the SDK so we will not be able to fix this in the current major version of the SDK. That being said, we can look into omitting empty array members from API responses for version 4 of the AWS SDK for PHP to avoid this behavior once the next major version of the SDK is released. In the meantime, the code snippet you provided as a workaround seems like the best approach for your use case.