Change image processing from ImageMagick to libvips

[Gargron]

• Removes metadata
• Retains color profile
• Reads pixels for blurhash
• Reads histograms for color extraction
• Crops animated GIFs when needed

---

Fixes MAS-223

[renchap]

The Github actions will need to install libvips42 instead of imagemagick.

There are probably some ImageMagick related files to remove, for example config/imagemagick.

The Docker image will also need to be changed to install vips, @vmstan started to work on it. Maybe you can update it in this PR to simply install libvips42 in place of imagemagick, and @vmstan can open another PR later to get it installed better (because the current Debian package enabled far too many formats for our usage).

We will also need to take great care in the upgrade instructions to tell people that they need to install VIPS now (and uninstall IM?). This will also be an issue with people running on nightly. The alternative is to support both IM and VIPS for 4.3, with a config option to enable VIPS, show a deprecation warning with IM is used, and remove IM support in 4.4, but I do not know if this is worth it.

[renchap]

It is also probably a good idea to call Vips.block_untrusted (see libvips/ruby-vips#382) so only the trusted loaders (ie, that are covered by the fuzzing testing) are enabled.

Untrusted operations can be seen with vips -l | grep untrusted. There are more details in the 8.13 release notes.

[github-actions]

This pull request has merge conflicts that must be resolved before it can be merged.

[shleeable]

Worth adding libheif?

[vmstan]

Worth adding libheif?

It's included by the libvips42 package.

[vmstan]

I’ve been running this PR for the last 20 hours on vmst.io and it’s been largely successful. I have had no user reported issues uploading and processing image content, but noticed a few entries in Sidekiq dead queue triggered by a few remote posts.

extract_area: bad extract area

https://plasmatrap.com/notes/9t1auqkbl0
(There are many entries all related to this note via replies, etc, but this is the source post)

Related backtrace:

/usr/local/bundle/gems/ruby-vips-2.2.1/lib/vips/operation.rb:228:in `build'
/usr/local/bundle/gems/ruby-vips-2.2.1/lib/vips/operation.rb:481:in `call'
/usr/local/bundle/gems/ruby-vips-2.2.1/lib/vips/image.rb:229:in `method_missing'

hist_find_ndim: image is not 1 - 3 bands

https://graphics.social/users/metin/statuses/112404645439099091
https://mastodon.online/users/spocko/statuses/112401772350190886
https://mastodon.social/users/wearenew_public/statuses/112401409861904050
https://grafana.social/@grafana/112406569119122877

Related backtrace:

/usr/local/bundle/gems/ruby-vips-2.2.1/lib/vips/operation.rb:228:in `build'
/usr/local/bundle/gems/ruby-vips-2.2.1/lib/vips/operation.rb:481:in `call'
/usr/local/bundle/gems/ruby-vips-2.2.1/lib/vips/image.rb:229:in `method_missing'

VipsForeignSave: "/tmp/c0ddd4cf006863154a6ba8d7b30396bd20240507-7-ivyc3w.jfif" is not a known file format

https://mastodon.art/users/catrionaroberts/statuses/112400639207525684

Related backtrace:

/usr/local/bundle/gems/ruby-vips-2.2.1/lib/vips/image.rb:597:in `write_to_file'
/opt/mastodon/lib/paperclip/lazy_thumbnail.rb:47:in `make'
/usr/local/bundle/gems/kt-paperclip-7.2.2/lib/paperclip/processor.rb:33:in `make'

The JFIF file format error appears to be related to libvips/libvips#3775 which was fixed by a later version of libvips than what comes from the Debian repo. After this error appeared I switched to using the latest version of libvips compiled into the Docker container using vmstan#27 which will be submitted as a change after this PR has merged.

[vmstan]

As far as performance data is concerned, so far I would say libvips has led to slightly lower container CPU utilization. I don't have any fine grained metrics on image processing performance, but comparing the last day worth of traffic to the same time period in the previous week, which had similar amounts of Sidekiq traffic processed:

Tuesday 7 into Wednesday 8 May 2024

Tuesday 30 April into Wednesday 1 May 2024

Average CPU usage (blue line) appears consistently lower, as do all load averages.

These are on three, 4x dedicated vCPU droplets at Digital Ocean. Average Sidekiq jobs per day approx 1.4 million.

[vmstan]

I've run into an issue where uploading a PNG file that exceeds the allowed dimensions/megapixels results in an error of 422 Validation failed: File must be less than 16 MB, File file size must be less than 16 MB even if the file itself is less than 16 MB.

Uploading a file that is exactly 2160x3840 works, but a file that is 2161x3842 will result in the above error. In both cases the file is roughly 4.6 MB in size. So not only is the wrong type of error being triggered, it doesn't seem like any error should be triggered if the file was rescaled by vips.

I do not see this issue with JPG or HEIF files.

[LeoEurope]

I’ve been running this PR for the last 20 hours on vmst.io and it’s been largely successful.

@vmstan As a switch to libvips supposedly allows WebP and AVIF files to be rendered in link previews (closing #27370 and #14983, and probably many more) do you see that many of the issues listed here are resolved?

[vmstan]

I’ve been running this PR for the last 20 hours on vmst.io and it’s been largely successful.

@vmstan As a switch to libvips supposedly allows WebP and AVIF files to be rendered in link previews (closing #27370 and #14983, and probably many more) do you see that many of the issues listed here are resolved?

I test posted all of the links shared in that comment and they looked the same on my instance running with vips as they do on mastodon.online running ImageMagick and similar build of 4.3.

[LeoEurope]

I’ve been running this PR for the last 20 hours on vmst.io and it’s been largely successful.

@vmstan As a switch to libvips supposedly allows WebP and AVIF files to be rendered in link previews (closing #27370 and #14983, and probably many more) do you see that many of the issues listed here are resolved?

I test posted all of the links shared in that comment and they looked the same on my instance running with vips as they do on mastodon.online running ImageMagick and similar build of 4.3.

OK, that's odd. As I've already bug spammed this ticket enough and this is a libvips implementation ticket let's get to the bottom of the issue using ticket #14983 and consider using ticket #27370 as a potential stopgap until the system accepts WebP and AVIF. Once libvips has properly landed I can be more helpful with troubleshooting.