Lazy download images for source plugins

[KyleAMathews]

A lot of source plugins use createRemoteFileNode to download images and make them available to process locally. E.g. a CMS allows a user to upload images and the developer wants to make sure they're minimized and that the site is loading optimally sized thumbnails.

The problem with this is that it can mean that hooking up Gatsby to e.g. a wordpress site can get stalled as 1000s of images need to be downloaded before Gatsby's bootstrap can finish.

This makes it hard to evaluate using Gatsby & day-to-day development painful.

Ideally we'd lazy download files when a graphql query references one.

[laurenskling]

I'm downloading the same 3000 images from wordpress everytime I start a project of mine, and most of the time it fails after 10 minutes when using a bad internet connection. It's really killing for developing.

I was looking for a solution to create development fallback images, but lazy might be even better 🌈

[jamessimone]

@laurenskling there are plugins like gatsby-remark-images-anywhere that cache the downloaded images, meaning that unless you change your package file, gatsby node file, or gatsby config file, you’ll have the images downloaded any time you start up. I also wrote a blog post on optimizing images that you might find interesting if looking, like me, for something a little more specific than the plugin I linked. My adventure travel blog currently has nearly 1000 photos and downloading them and optimizing the first time I start gatsby up takes several minutes, but subsequent startups take a minute or less.

Long story short - if you want to download your remote images and cache them between gatsby runs, it’s totally possible ... you just have to write / use plugins at the moment! Hope that helps.

[laurenskling]

I found this (https://thoughtsandstuff.com/gatsby-with-wordpress-caching-downloaded-media-images-to-reduce-build-time/) article somewhere; this worked perfectly for my wordpress setup! Wouldn't something like this be great to be included in gatsby-source-wordpress?

edit: I think the above was a false positive :/ It seems like my project is not dowloading the files again when I restart development right now

[jamessimone]

@laurenskling that article’s caching implementation is not properly implemented anymore. There was a change made to Gatsby relatively recently - and perhaps unintentionally - that invalidates the cache in situations where the node is simply touched. It’s possible I can submit a PR to fix this for the gatsby source Wordpress plugin if I have some free time this week, because it’s difficult to explain exactly what needs to be done here (other than that I don’t think the createResolvers way is going to work).

[laurenskling]

Yeah, I think I ran into a false positive using this article, it might have been a gatsby or gatsby-source-wordpress that is making my downloads cache recently :) This is really nice for my DX <3 no more waiting 10 minutes before i can start developing.

But to get back on topic, if lazy loading would be possible, it would be even better :)