tf.io.gfile.rename not working for directories in S3

[j99ca]

The following code used to work in Tensorflow < 2.6. Upon Tensorflow 2.6, we had to import tensorflow_io. However, the tf.io.gfile.rename function which used to work on directories in S3 no longer works. I would like to update to a newer version of Tensorflow but this issue is preventing our organization from doing so, as some libraries we use use tf.io.gfile.rename to change folder names during training.

Code to replicate the issue:

import tensorflow as tf
import tensorflow_io as tfio

SOURCE_DIR = 's3://.../old_name/'
DEST_DIR = 's3://.../new_name/'

tf.io.gfile.rename(SOURCE_DIR, DEST_DIR)

I created an issue on the Tensorflow github repo but I haven't gotten any responses in a couple of months.