Special characters in filename causes uploads to fail

[jet10000]

Bug report

When upload "望舌诊病.pdf"

Describe the bug

[thebengeu]

Thanks for the bug report @jet10000! We'll take a look.

[rahul3v]

@thebengeu @jet10000 @alaister
As per now for both objectName and bucketName , supabase only allow s3 safe characters as per AWS guideline here

 // only allow s3 safe characters and characters which require special handling for now
 // https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html

[alaister]

S3 supports UTF-8 characters in filenames. However, at the moment, we are very strict with which filenames we allow.
I think this is a valid use case to add support for different languages in filenames.

One option is to update the isValidKey function https://github.com/supabase/storage-api/blob/9480891af024396c58045578d16c91778aae67d2/src/utils/index.ts#L76-L80 to allow everything aside from the characters outlined in https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html#object-key-guidelines-avoid-characters

[PiddePannkauga]

I would love support for å,ä,ö.

Love from Sweden

[Xuanwo]

Hi, opendal meets a similiar problem in apache/opendal#2190 that we have a test case (which passed on most storage platforms from s3, gcs, azalob to hdfs) like the following:

let path = format!("{} !@#$%^&()_+-=;',.txt", uuid::Uuid::new_v4());

Does this case make sense to you? I'm willing to help fix this. I believe that all URL-unsafe characters should be percent-encoded, and the server-side should handle the job of decoding them.

[javierfern03]

I would love support for ´ I can't use words with accents in my images, which is very common in Spanish.

[turulix]

Also ÖÄÜ in german

[orlein]

I could understand that the isValidKey function looks like because it would be safe if the system accepts only the alphabet.
As I'm Korean, there is no good way to safely convert from the Korean characters(hangul) to alphabet. It would be same for the Japanese characters and Chinese characters too.
Are there any specific reasons for the function's regex? If the function accepts the encoded characters from encodeURIComponent function, it would be great.

[li4man0v]

MacOS generates screenshot names that don't match the pattern in isValidKey. For example, "Screenshot 2024-01-24 at 12.25.39 AM".

[jingsam]

Ops! This issue seems easy to fix but have last for 2 year's. Unbelievable!

[ThaddeusJiang]

hi everyone

My solution is base64 encoding when uploading file, demo: https://github.com/ThaddeusJiang/supabase-helpers/blob/main/backup_storage_buckets.ts#L71-L73

[gitnik]

This is a big pretty big oversight. At the very least it would be nice if this was documented somewhere

[stefan-girlich]

You can upload files using any name (e.g. base64, thx @ThaddeusJiang ), store the original file name in a different table, and then use it when generating the public link:

const {
  data: { publicUrl },
} = supabase.storage
  .from('my_bucket')
  .getPublicUrl(internalFileName, { download: originalFileName })

Source (for URL access): https://supabase.com/docs/guides/storage/serving/downloads#downloading
The supabase-js docs unfortunately do not mention this yet: https://supabase.com/docs/reference/javascript/storage-from-getpublicurl