Huge memory spike on one HEIC file

[urthling]

Hi,

Memory usage for a 10MB (16k x 3k) is about 300MB. However, the attached file uses over 2GB.

Note, to allow this to upload, I changed the file extension to GIF but this is a HEIC

Any thoughts?

[catdad]

Hmm... I do see that the attached image uses quite a bit of memory. I see that the memory is being used by the libheif decoding code, and I am not entirely sure what is or isn't expected there.

You mentioned "a 16k x 3k image is about 300MB". Where does this value come from? Do you have one such heif image that uses 300MB and this second attached one uses 2GB? If you have a second image like that, can you please upload that as well so that I can compare them?

[urthling]

First off, thank you for putting this project out there, this is the only project I've found that works for me, so great job!

Ah, the rest of the images are actually JPGs my bad. So this was the only HEIC. My environment is restricted, so I'm going to limit HEICs to 8K for the time being.

Mmm, the process doesn't seem to spike in the same way with this decoder:
https://strukturag.github.io/libheif/

[catdad]

Interesting. This project uses the same library as your link, just a newer version. Let me see what I can find

[suan]

I'm getting the same issue. Basically any "Live" image taken from an iPhone results in a huge >1GB memory spike when being converted, even if just using the first frame.

[suan]

Could strukturag/libheif#493 have fixed this?

[mohammed-almujil]

Hi,

I appreciate the work done to put together this library.

I am getting a huge memory spike when converting HEIC to JPEG using a pic taken from an iPhone 4032x3024. I am using Node/docker with 1gb RAM, it spikes from 92mb to to 500mb and keeps on leaking causing OOM after several image conversions.

I am not too concern about the spikes but more about the OOM. I found a workaround by converting to PNG instead of JPEG. It still spikes but at least it clears to around 140mb which is not too bad and seems consistent so far.

[catdad]

Hey all. Thanks for these reports. I don't have a lot of time these days to investigate. To make it easier, having repro steps would help, including attaching any necessary images. Some further information might help, like whether you are seeing issues with all images or only specific ones. And as always, PRs are welcome!

[markoffk]

@catdad Here is example file (sample1.heif) which eats 3GB of RAM during convert from HEIF to JPG:
https://filesamples.com/formats/heif

[jalik]

Hello, I have the same issue with the sample file given by @markoffk.
I also have the following error:

Image to composite must have same dimensions or smaller {"error":{"message":"Image to composite must have same dimensions or smaller","name":"Error","stack":"Error: Image to composite must have same dimensions or smaller"}}

[yamiteru]

So basically this library is completely useless unless you want to convert heic to png. And it seems the author doesn't maintain this repo. Great.

[jalik]

So basically this library is completely useless unless you want to convert heic to png. And it seems the author doesn't maintain this repo. Great.

Maintaining one or more libraries is hard and not often rewardful (in open source world).
Don't forget that these people were kind enough to share their projects with us.
Today they may not be able to continue for whatever reason, thus you have several options:

• find a workaround (if possible) in your project until a fix is released (like converting HEIC to PNG, then to JPEG with another tool)
• investigate and submit a pull request with a fix
• fork the project and add/fix what you want yourself (don't depend on anybody, rely on yourself)
• fork the project and pay someone to do the work for you (need money)
• find an alternate library

Hope you'll find the way to go.

[bitfede]

Hey @catdad , thank you so much for sharing this library with the dev community, you saved my life! You deserve lots of good karma!! Unfortunately I'm also worried about the big memory consumption, which crashed my heroku server once and forced me to get a bigger server for now.

I have a backend app that receives emails with photo attachments of receipts to perform OCR+AI etc to extract structured JSON data from it. Yesterday the issue of HEIC files being sent from iphones came up so I rushed to write a piece of code that will convert HEIC to JPG, and you library was the easiest to integrate.

This is the HEIC image that I tested.

Here's the spike of memory in my heroku dashboard:

This happened when 4 emails with HEIC photos were sent and processed at the same time.. Scared that more requests will crash the server.

I read in the previous comments that if you convert to PNG instead of JPEG it will use significantly less memory, is that true? I will try tonight and update this comment.

Other than this, has anyone found any workaround or solutions? I am expecting a bit more traffic into the app next week so I wanted to reach out to see if any other dev had the same issue and how they ended up solving it.

Thanks in advance for any feedback!

PS: I'm open to create a small dev task force team to tackle this problem together and submit a fix! Anyone wanna join? :D

PPS: is this memory leak fix pushed in November for libheif anyhow related? 1div0/libheif@8a76ebc

[jalik]

Hello @bitfede, you could workaround the issue by using a processing queue or forcing the processing to handle images in serial.

// this is not ideal, but really simple to handle processing in serial
async function processEmails(emails) {
  for (const email of emails) {
    const inputBuffer = await getImageFromEmail(email);

    const jpegBuffer = await convert({
      buffer: inputBuffer,
      format: 'JPEG',
      quality: 1
    });
  }
}

[bitfede]

hmm unfortunately it's an expressJS app, and every email comes as a POST request to one of my route endpoints. I don't have an array of emails to process. I get the file buffer, name etc by accessing req.files[0]. Thank you for the input tho @jalik , appreciate taking the time to help.

[jalik]

hmm unfortunately it's an expressJS app, and every email comes as a POST request to one of my route endpoints.

I don't know the structure of your app, but it does not block you from storing incoming emails somewhere (database, filesystem, object-storage like S3...), then process them later with a CRON like function, it is just more complex to setup.
The thing here is to not process your email individually, but in batch, so first you need to create the batch, then looping over each email, you may need to rethink how you handle your emails.

[mohammed-almujil]

Hey @bitfede, PNG conversion was better and more stable. I ended up not using the library in my express app because the conversion operation was blocking. I made it async but my implementation was too heavy for my mono app running in real-time.

My app uploads images from iOS to the backend server, I nicely asked the mobile engineer if they were able to help convert the image on iOS to JPEG before sending it over and that's the solution that we went for. It is not the best solution but it works.