Camera plugin on iOS throws error when selected photo is not available locally (iCloud)

[alyyousuf7]

Bug Report

Plugin(s)

• @capacitor/camera version 5.0.7 - I'm pretty sure it's on old versions as well

Capacitor Version

💊   Capacitor Doctor  💊 

Latest Dependencies:

  @capacitor/cli: 5.4.1
  @capacitor/core: 5.4.1
  @capacitor/android: 5.4.1
  @capacitor/ios: 5.4.1

Installed Dependencies:

  @capacitor/cli: 5.4.0
  @capacitor/core: 5.4.0
  @capacitor/android: 5.4.0
  @capacitor/ios: 5.4.0

[success] iOS looking great! 👌
[success] Android looking great! 👌

Platform(s)

iOS

Current Behavior

When I use Camera.pickImages or Camera.getPhoto, and I select any image, that is not available locally on my iOS but saved on iCloud Photos and Optimized Storage turned on, the plugin throws an error Error loading image.

Expected Behavior

I expect that regardless the file is available locally or not, it should be loaded fine.

Similar behaviour can be witnessed on iOS when you use <input type="file" accept="image/*" /> on Safari/Chrome to pick an image, it loads the image, showing spinner, if it's not available on the iPhone locally. The same does not happen when using Capacitor.

Code Reproduction

• Make sure you have iCloud on the profile
• Make sure you have Optimize Storage feature on for Photos
• From the code below, when the native PHPicker opens up, select a photo which is not available locally on device

Camera.pickImages({ quality: 90 })

OR

Camera.getPhoto({
  allowEditing: false,
  source: CameraSource.Photos,
  resultType: CameraResultType.Uri,
})

Other Technical Details

It seems that the img.itemProvider.canLoadObject returns false when the photo is not available locally.

Additional Context

[Ionitron]

This issue needs more information before it can be addressed.
In particular, the reporter needs to provide a minimal sample app that demonstrates the issue.
If no sample app is provided within 15 days, the issue will be closed.

Please see the Contributing Guide for how to create a Sample App.

Thanks!
Ionitron 💙

[alyyousuf7]

added reproduction steps in detail

[giuseeFG]

Hi, did you find a solution? The problem is if you have a photo on iCloud and not locally, the library cannot download it.

[bpolack]

Also experiencing this issue.

[Ionitron]

This issue needs more information before it can be addressed.
In particular, the reporter needs to provide a minimal sample app that demonstrates the issue.
If no sample app is provided within 15 days, the issue will be closed.

Please see the Contributing Guide for how to create a Sample App.

Thanks!
Ionitron 💙

[giuseeFG]

I tried to upgrade to capacitor 5 with no success, the problem remains.

💊   Capacitor Doctor  💊 

Latest Dependencies:

  @capacitor/cli: 5.4.1
  @capacitor/core: 5.4.1
  @capacitor/android: 5.4.1
  @capacitor/ios: 5.4.1

Installed Dependencies:

  @capacitor/core: 5.4.1
  @capacitor/cli: 4.6.1
  @capacitor/ios: 5.4.1
  @capacitor/android: 4.6.1

[success] iOS looking great! 👌
[success] Android looking great! 👌

[eliotfrost]

Have been seeing the same issue. Seems to be coming up a lot given a lot of people with new devices (whose photos are almost entirely on iCloud)

[giuseeFG]

Have been seeing the same issue. Seems to be coming up a lot given a lot of people with new devices (whose photos are almost entirely on iCloud)

The very very strange thing is that I have other several apps in witch this bug doesn't happen! With the same phone and with the same photos. 🤷‍♂️

[alyyousuf7]

Have been seeing the same issue. Seems to be coming up a lot given a lot of people with new devices (whose photos are almost entirely on iCloud)

The very very strange thing is that I have other several apps in witch this bug doesn't happen! With the same phone and with the same photos. 🤷‍♂️

Are those apps built over capacitor as well?
In other apps (not capacitor) I have noticed that they show a spinner on the native UI (probably PHPicker?) when you select and the image is not available locally.

[giuseeFG]

Have been seeing the same issue. Seems to be coming up a lot given a lot of people with new devices (whose photos are almost entirely on iCloud)

The very very strange thing is that I have other several apps in witch this bug doesn't happen! With the same phone and with the same photos. 🤷‍♂️

Are those apps built over capacitor as well? In other apps (not capacitor) I have noticed that they show a spinner on the native UI (probably PHPicker?) when you select and the image is not available locally.

Yes, in Capacitor (4 and 5) as well.
I spent a lot of hours this week, with no success. I tried everything (downgrade/upgrade version, re-create platform, copy the platform from the good one into the "broken" one etc...).

[john-legallynoticed]

I am having the same issue. It is 100% reproducible on our end when attempting to select photos from a Shared Album. The album must have been shared by someone else. Shared Albums owned by the user work fine.

[ZakDaMack]

Hi all, we have been experiencing the same issue with iOS 16/17 devices due to the images only being available on iCloud. Has there been an update on any potential fixes/workarounds?

[benmartin88]

Hi, we are also experiencing this issue with users across iOS 16.4 onwards, and 17.0.1 onwards. This only seems to occur, from what we can tell, when a user selects a photo that is only stored on iCloud. Although we have had reports of users taking photos then having the same issue. I believe this could be because of the settings for storage.

I would assume this is affecting all apps using capacitor with the camera plugin, who have a user base on ios.

We tested replacing the camera plugin with the capawesome filepicker plugin which has a pickImages function and the issue exists there too. It looks like the exception is throw by capacitor core

[jonmbennett]

We also started experiencing this, especially with users who are on iOS 17.0.3.

If an image fails to get selected, and then you try again 15-30 seconds later, it will often work the second time. If you're trying to reproduce this issue, you may need to take a fresh photo and then immediately try using it from the Gallery. It seems that attempting to use the photo causes it to get downloaded from iCloud, or some other mechanism that makes it available to Capacitor.

In the CameraPlugin.swift code of the Capacitor Camera plugin, I attempted to adjust it to use the Photos framework's PHImageManager instead, but it didn't seem to have any effect (positive or negative). I'm not experienced with Swift so it's likely I didn't do something correctly, but I'll share this in case it helps generate some ideas.

// Fallback: Use PHImageManager if the original code fails
if let assetId = result.assetIdentifier {
  let asset = PHAsset.fetchAssets(withLocalIdentifiers: [assetId], options: nil).firstObject
  let options = PHImageRequestOptions()
  options.isNetworkAccessAllowed = true
                    
  PHImageManager.default().requestImage(for: asset!, targetSize: CGSize(width: 500, height: 500), contentMode: .aspectFit, options: options) { [weak self] (image, info) in
    if let image = image {
      if var processedImage = self?.processedImage(from: image, with: asset?.imageData) {
        processedImage.flags = .gallery
        self?.returnProcessedImage(processedImage)
        return
      }
    }
    self?.call?.reject("Error loading image")
  }
} else {
  self?.call?.reject("Error loading image")
}

This issue is also being reported on some React Native plugins, such as here. Perhaps something changed on Apple's side?

[lyubchev]

Same issue here at my company, a lot of users have been reporting this issue and we reproduce it. Any temporary fixes?

[benmartin88]

Same issue here at my company, a lot of users have been reporting this issue and we reproduce it. Any temporary fixes?

I think this will only get worse as more people upgrade their ios or device.

From what I've seen on the react forums it seems to be something to do with HEIF. Someone had written a work around to convert these to jpeg.

We did try implementing bits of it but have no experience with swift.

I wonder if collectively we could write a temp fix whilst we wait for an official patch

[tonga54]

Same issue here, any temporary fix? This is a serious bug, it affects a lot of users.

[benmartin88]

Same issue here, any temporary fix? This is a serious bug, it affects a lot of users.

We are already migrating away from capacitor as a result of continued issues beyond this one since ios 17. Disappointing to see no engagement from the ionic team

[tonga54]

EDITED

I just developed a temporary solution, basically it will always search for the images that are hosted on iCloud.

In order to use this code, you will have to replace the "picker" function inside the file "node_modules/@capacitor/camera/ios/Plugin/CameraPlugin.swift"
with the following code.

I sincerely hope that it can be useful to you.

If this was helpful to you, you can contribute with me with a donation of whatever you think is necessary, thank you very much.

@available(iOS 14, *)
extension CameraPlugin: PHPickerViewControllerDelegate {
    public func picker(_ picker: PHPickerViewController, didFinishPicking results: [PHPickerResult]) {
        picker.dismiss(animated: true, completion: nil)
        
        if results.isEmpty {
            // handle when no photo is selected
            self.call?.reject("User did not select any photo")
            return
        }

        if multiple {
            let manager = PHImageManager.default()
            let options = PHImageRequestOptions()
            options.version = .original
            options.isNetworkAccessAllowed = true
            options.deliveryMode = .highQualityFormat
            options.isSynchronous = false

            var images: [ProcessedImage] = []

            let dispatchGroup = DispatchGroup()

            results.forEach { result in
                dispatchGroup.enter()
                if let assetID = result.assetIdentifier,
                    let asset = PHAsset.fetchAssets(withLocalIdentifiers: [assetID], options: nil).firstObject {
                    manager.requestImageData(for: asset, options: options) { [weak self] (data, _, _, _) in
                        if let data = data, let image = UIImage(data: data),
                            let processedImage = self?.processedImage(from: image, with: asset.imageData) {
                            images.append(processedImage)
                        }
                        dispatchGroup.leave()
                    }
                } else {
                    dispatchGroup.leave()
                }
            }

            dispatchGroup.notify(queue: .main) {
                self.returnImages(images)
            }
        } else {
            guard let result = results.first else {
                self.call?.reject("User cancelled photos app")
                return
            }
            
            guard let assetID = result.assetIdentifier else {
                self.call?.reject("Error loading image")
                return
            }

            let assets = PHAsset.fetchAssets(withLocalIdentifiers: [assetID], options: nil)
            guard let asset = assets.firstObject else {
                self.call?.reject("Error loading image data")
                return
            }

            let manager = PHImageManager.default()
            let options = PHImageRequestOptions()
            options.version = .original
            options.isSynchronous = true
            options.isNetworkAccessAllowed = true

            manager.requestImageData(for: asset, options: options) { [weak self] (data, _, _, _) in
                guard let self = self else { return }
                if let data = data {
                    let image = UIImage(data: data)
                    var processedImage = self.processedImage(from: image!, with: asset.imageData)
                    processedImage.flags = .gallery
                    self.returnProcessedImage(processedImage)
                } else {
                    self.call?.reject("Error downloading image data")
                }
            }
        }
    }
}

[benmartin88]

I just developed a temporary solution, basically it will always search for the images that are hosted on iCloud.

In order to use this code, you will have to replace the "picker" function inside the file "node_modules/@capacitor/camera/ios/Plugin/CameraPlugin.swift" with the following code.

I sincerely hope that it can be useful to you.

If this was helpful to you, you can contribute with me with a donation of whatever you think is necessary, thank you very much.

@available(iOS 14, *)
extension CameraPlugin: PHPickerViewControllerDelegate {
    public func picker(_ picker: PHPickerViewController, didFinishPicking results: [PHPickerResult]) {
        picker.dismiss(animated: true, completion: nil)
        
        if results.isEmpty {
            // handle when no photo is selected
            self.call?.reject("User did not select any photo")
            return
        }

        if multiple {
            let manager = PHImageManager.default()
            let options = PHImageRequestOptions()
            options.version = .original
            options.isSynchronous = true
            options.isNetworkAccessAllowed = true

            var images: [ProcessedImage] = []
            var processedCount = 0
            for result in results {
                guard let assetID = result.assetIdentifier else {
                    continue
                }
                
                let assets = PHAsset.fetchAssets(withLocalIdentifiers: [assetID], options: nil)
                guard let asset = assets.firstObject else {
                    processedCount += 1
                    if processedCount == results.count {
                        self.returnImages(images)
                    }
                    continue
                }
                
                manager.requestImageData(for: asset, options: options) { [weak self] (data, _, _, _) in
                    if let data = data {
                        let image = UIImage(data: data)
                        if let processedImage = self?.processedImage(from: image!, with: asset.imageData) {
                            images.append(processedImage)
                        }
                    }
                    processedCount += 1
                    if processedCount == results.count {
                        self?.returnImages(images)
                    }
                }
            }
        } else {
            guard let result = results.first else {
                self.call?.reject("User cancelled photos app")
                return
            }
            
            guard let assetID = result.assetIdentifier else {
                self.call?.reject("Error loading image")
                return
            }

            let assets = PHAsset.fetchAssets(withLocalIdentifiers: [assetID], options: nil)
            guard let asset = assets.firstObject else {
                self.call?.reject("Error loading image data")
                return
            }

            let manager = PHImageManager.default()
            let options = PHImageRequestOptions()
            options.version = .original
            options.isSynchronous = true
            options.isNetworkAccessAllowed = true

            manager.requestImageData(for: asset, options: options) { [weak self] (data, _, _, _) in
                guard let self = self else { return }
                if let data = data {
                    let image = UIImage(data: data)
                    var processedImage = self.processedImage(from: image!, with: asset.imageData)
                    processedImage.flags = .gallery
                    self.returnProcessedImage(processedImage)
                } else {
                    self.call?.reject("Error downloading image data")
                }
            }
        }
    }
}

What happens of the image is not on icloud or the user has icloud disabled?

[tonga54]

What happens of the image is not on icloud or the user has icloud disabled?

If the image is not on iCloud or if iCloud is disabled, PhotoKit (the framework you're using for handling images) will look in the local cache on the device for the image.

When isNetworkAccessAllowed is set to true and the full resolution version of the image is not in the local cache or on iCloud (or if iCloud is disabled), the image request will fail and the requestImageData(for:options:resultHandler:) method's resultHandler completion block will be called with an error.

Therefore, in your code, you should handle these errors in the requestImageData(for:options:resultHandler:) method's resultHandler callback to provide appropriate feedback to the user. This will ensure a smooth user experience in case something goes wrong when downloading the image.

[giuseeFG]

EDITED

I just developed a temporary solution, basically it will always search for the images that are hosted on iCloud.

In order to use this code, you will have to replace the "picker" function inside the file "node_modules/@capacitor/camera/ios/Plugin/CameraPlugin.swift" with the following code.

I sincerely hope that it can be useful to you.

If this was helpful to you, you can contribute with me with a donation of whatever you think is necessary, thank you very much.

@available(iOS 14, *)
extension CameraPlugin: PHPickerViewControllerDelegate {
    public func picker(_ picker: PHPickerViewController, didFinishPicking results: [PHPickerResult]) {
        picker.dismiss(animated: true, completion: nil)
        
        if results.isEmpty {
            // handle when no photo is selected
            self.call?.reject("User did not select any photo")
            return
        }

        if multiple {
            let manager = PHImageManager.default()
            let options = PHImageRequestOptions()
            options.version = .original
            options.isNetworkAccessAllowed = true
            options.deliveryMode = .highQualityFormat
            options.isSynchronous = false

            var images: [ProcessedImage] = []

            let dispatchGroup = DispatchGroup()

            results.forEach { result in
                dispatchGroup.enter()
                if let assetID = result.assetIdentifier,
                    let asset = PHAsset.fetchAssets(withLocalIdentifiers: [assetID], options: nil).firstObject {
                    manager.requestImageData(for: asset, options: options) { [weak self] (data, _, _, _) in
                        if let data = data, let image = UIImage(data: data),
                            let processedImage = self?.processedImage(from: image, with: asset.imageData) {
                            images.append(processedImage)
                        }
                        dispatchGroup.leave()
                    }
                } else {
                    dispatchGroup.leave()
                }
            }

            dispatchGroup.notify(queue: .main) {
                self.returnImages(images)
            }
        } else {
            guard let result = results.first else {
                self.call?.reject("User cancelled photos app")
                return
            }
            
            guard let assetID = result.assetIdentifier else {
                self.call?.reject("Error loading image")
                return
            }

            let assets = PHAsset.fetchAssets(withLocalIdentifiers: [assetID], options: nil)
            guard let asset = assets.firstObject else {
                self.call?.reject("Error loading image data")
                return
            }

            let manager = PHImageManager.default()
            let options = PHImageRequestOptions()
            options.version = .original
            options.isSynchronous = true
            options.isNetworkAccessAllowed = true

            manager.requestImageData(for: asset, options: options) { [weak self] (data, _, _, _) in
                guard let self = self else { return }
                if let data = data {
                    let image = UIImage(data: data)
                    var processedImage = self.processedImage(from: image!, with: asset.imageData)
                    processedImage.flags = .gallery
                    self.returnProcessedImage(processedImage)
                } else {
                    self.call?.reject("Error downloading image data")
                }
            }
        }
    }
}

Thanks! I'll try asap. Do you know if the fix you provided, returns also exif?

[tonga54]

Thanks! I'll try asap. Do you know if the fix you provided, returns also exif?

Yeah, it returns the same parameters as the original plugin.
I hope this solution works for you!!

[giuseeFG]

Thanks! I'll try asap. Do you know if the fix you provided, returns also exif?

Yeah, it returns the same parameters as the original plugin. I hope this solution works for you!!

Mate it worked ❤️ I'll proceed with a donation!

By the way, I noticed that also Android, has a similar issue with not-local photos. Do you know anything about that?

[tonga54]

Mate it worked ❤️ I'll proceed with a donation!

By the way, I noticed that also Android, has a similar issue with not-local photos. Do you know anything about that?

Hi how are you? Thank you very much for the donation, I am very happy that it worked for you.

Regarding what you mentioned about Android, I have not seen any related reports, but if you continue experiencing this error, create an issue in this repository and I will try to resolve it with pleasure.

[john-legallynoticed]

Can we please get an update from the Capacitor team? This is a serious issue affecting every iOS user. This should not be tagged "needs reproduction" as reproduction instructions have been provided. You simply try to pick any iCloud photo not physically present on the device and the plugin fails.

[tonga54]

Can we please get an update from the Capacitor team? This is a serious issue affecting every iOS user. This should not be tagged "needs reproduction" as reproduction instructions have been provided. You simply try to pick any iCloud photo not physically present on the device and the plugin fails.

It's a very serious bug, i posted a temporary solution in the comments #1807 (comment), you can use it if you want, but we need an official one.