[gatsby-plugin-sharp] SVG traced placeholders #2456

fk · 2017-10-14T03:14:59Z

(See #2435) Using node-potrace (and Jimp…) and SVGO.
Only working with resolutions right now, also didn’t check SVG size yet … and still have to look at gatsby-image too. 🤧

turdSize, optTolerance and turnPolicy defaults for potrace follow https://twitter.com/Martin_Adams/status/918481948217049088

Todo:

add to sizes()
support all node-potrace parameters
memoize
update README
Fix for IE – https://codepen.io/tigt/post/optimizing-svgs-in-data-uris

Using node-potrace (and Jimp…) and SVGO. Only working with `resolutions` right now, also didn’t check SVG size yet.

KyleAMathews · 2017-10-14T03:16:20Z

Deploy preview failed.

Built with commit 80e1bd5

https://app.netlify.com/sites/using-glamor/deploys/59e7d078a6188f031e24c654

gatsbybot · 2017-10-14T03:21:21Z

Deploy preview ready!

Built with commit 8e32e7b

https://deploy-preview-2456--gatsbygram.netlify.com

KyleAMathews · 2017-10-14T06:12:48Z

Hot dang! That's super cool! Umm wow, @fk we're going to anoint you king graphics wizard or something soon :-)

The size is 24kb uncompressed, 16kb gzipped (compared html size on master image-processing vs this).

This works shockingly well.

Should be really straightforward now to add this as an option to gatsby-image 🎉

KyleAMathews · 2017-10-14T06:13:51Z

Specifically:

Add it as a fragment option
In gatsby-image, check for the key and use it instead of base64/backgroundColor

sebastienfi · 2017-10-14T08:17:06Z

The option would be to choose between blurred image and svg placeholder ? I like this

sebastienfi · 2017-10-14T08:20:33Z

packages/gatsby-plugin-sharp/src/index.js

@@ -461,10 +466,15 @@ async function resolutions({ file, args = {} }) {
    pngCompressionLevel: 9,
    grayscale: false,
    duotone: false,
+    trace: {


Would be awesome to implement background, color, threshold config parameters and expose these so user can customize the styling of the svg.

👍 👍 Planning to expose all of those: https://github.com/tooolbox/node-potrace#parameters – background and color are already working:

KyleAMathews · 2017-10-14T16:24:53Z

We gotta for sure enable this on all the avatars on gatsbyjs.org :-D

fk · 2017-10-14T18:00:43Z

Add it as a fragment option

Does adding

GatsbyImageSharpResolutions_tracedSVG
GatsbyImageSharpSizes_tracedSVG
GatsbyContentfulResolutions_tracedSVG
GatsbyContentfulSizes_tracedSVG

to the current ones sounds good for a first shot?

KyleAMathews · 2017-10-14T18:02:29Z

Yup! Except we can't add this to Contentful as it doesn't support traced svgs (though in theory someone could add it there too).

KyleAMathews · 2017-10-14T18:03:15Z

For Contentful, the image GraphQL schema is actually just creating URLs for their image api.

fk · 2017-10-14T18:04:21Z

👍 Thanks for the heads up regarding contentful! Probably would have bitten my teeth off :-D

fk · 2017-10-14T20:34:32Z

KyleAMathews · 2017-10-14T20:40:50Z

🎉🎉🎉

Want to do one from a full color image too real quick?

fk · 2017-10-14T21:06:41Z

Ah sorry didn't see until now. Gimme a minute!

fk · 2017-10-14T21:24:22Z

… so much for "a minute":

Following to https://twitter.com/Martin_Adams/status/918481948217049088

fk · 2017-10-15T00:18:39Z

Will wrap it up for today/night 🤧, but I think this is ready for a first review!

only updated the gatsby-image README with very basic info for now
the exposed Potrace options (trace: { … }) aren't mentioned anywhere yet
GraphQL types for some Potrace option fields aren't super thorough (threshold, color, background)
examples/image-processing features a little too much "tracedSVG"

KyleAMathews

Looking super solid! Love how cleanly all these things fit together ❤️

KyleAMathews · 2017-10-15T02:04:58Z

packages/gatsby-plugin-sharp/src/index.js

@@ -9,10 +9,15 @@ const imagemin = require(`imagemin`)
 const imageminPngquant = require(`imagemin-pngquant`)
 const queue = require(`async/queue`)
 const path = require(`path`)
+const potrace = require(`potrace`)


Let's lazy load these so they're only read off disk if someone is using them.

Struggling! 👋 😄 Happy to explain in detail what I'm trying to do, but maybe this is enough to makes sense already: How do I find out if the query contains the tracedSVG field?

Just move the requires inside the function for generating the traces.

Yep! My problem is determining if the traced SVG should be generated at all – if I set defaultValue for the "tracedSVG" field, the traced SVG will always be generated; if I don't, tracedSVG will only be returned if one or more "trace" arguments are set. Or am I 🤒 again?

Let me try and explain my issue again (still 😷🤒) – let's see if this amounts to anything more understandable then the previous attempt:

We want to lazy load potrace/SVGO only if needed. In the case of duotone, this would be easy – if there are duotone options set, apply the duotone effect – the image is always generated, anyway, and duotone options are required to generate a duotone image.

In the case of the "tracedSVG" however, I can't make the decision based on the "tracedSVG" options (trace:{color…}) being set or not, because that would require the user to always set options. Ideally, I could make the decision based on wether the "tracedSVG" is present in the query at all.

Ahhh gotcha. Yeah so what you want to do here is add a resolve function to the actual field which is where you do the require and generate the svg. A field resolve function is only called if that field is actually queried so is the perfect place to put expensive work.

😬😊🙏

This reverts commit da19d06. Default color of traced SVG now matches default „backgroundColor“ of the gatsby-image placeholder.

fk · 2017-10-19T00:04:35Z

Yes, that makes a lot of sense of course. Was wondering about this when I gave this thing the first shot last weekend. So we would generate the traced SVG based on the maxWidth for sizes() and width for resolutions()?

Seeing how I struggled to pass those "trace" arguments down to the "tracedSVG" child just now, I wonder how long it would take me … 😳 💔

KyleAMathews · 2017-10-19T00:23:32Z

Haha :-) You've got this!

So I'm assuming node-potrace needs the file written out? So probably we should wait until a file is written to disk and then trace that.

fk · 2017-10-19T00:31:24Z

node-potrace uses Jimp and also accepts a (Jimp?!) buffer: https://github.com/tooolbox/node-potrace/blob/0e98e95626abda922130e3c3323b3cb2c3e29007/lib/Potrace.js#L1018-L1050

fk · 2017-10-19T01:01:06Z

packages/gatsby-transformer-sharp/src/extend-node-type.js

 module.exports = ({ type, pathPrefix, getNodeAndSavePathDependency }) => {
  if (type.name !== `ImageSharp`) {
    return {}
  }

+  const getTracedSVG = parent => {
+    const promise = traceSVG({
+      file: { absolutePath: path.join(process.cwd(), `public`, parent.src) },


So I just wondered if what I'm currently doing is actually close to what you suggested …
The reason I wondered about this is that I didn't know how to actually reach the original image path from the subfield, so I just used what was there, namely parent.src…

And after coincidentally deleting my public folder, it turns out that parent.src isn't necessarily available at this point in time, meaning all that I thought would work up to now really isn't ;-/ … so much for "you got this". 😿

fk · 2017-10-19T22:49:50Z

examples/image-processing/src/pages/index.js

@@ -306,14 +306,19 @@ export const pageQuery = graphql`
    sizes: imageSharp(id: { regex: "/fecolormatrix-kanye-west.jpg/" }) {
      sizes(
        duotone: { highlight: "#f00e2e", shadow: "#192550" }
+        trace: {


@KyleAMathews Unsure about arg naming – should this rather be traceSVG or even matching the field name (tracedSVG … which I dislike for no apparent reason ;-))?

it should be a noun — so tracedSVG. GraphQL is declarative. You say — "I want X!" And X is given to you.

👍 (even if in this case it's "I want to configure X"?)

Oh my bad — this is the arguments not the field. Yeah, traceSVG then is right.

… currently generating SVGs up to 1.4MB :/ ;)

KyleAMathews · 2017-10-20T03:38:03Z

Well lookie at that https://deploy-preview-2456--using-gatsby-image.netlify.com/traced-svg/

fk · 2017-10-20T07:46:44Z

🎉 🎉 🎉

EmilTholin · 2017-10-20T09:04:01Z

packages/gatsby-plugin-sharp/src/index.js

+
+const optimize = svg => {
+  const SVGO = require(`svgo`)
+  const svgo = new SVGO()


This resize technique above is super awesome!
It might also be worth looking into multipass, to compress the svg even further:

const svgo = new SVGO({ multipass: true, floatPrecision: 1 })

Awww thanks Emil! 🤗 🙌

Demo images on a diet after that (missed checking one, but here's the before/after for two of the three at https://deploy-preview-2456--using-gatsby-image.netlify.com/traced-svg/):

7.401 -> 4.506 Byte
13.629 -> 8.562 Byte

Thank you again Emil!

(Hey I could swear I didn't see it in https://github.com/EmilTholin/image-trace-loader when I checked … ah yes, only saw the initial version. We're lucky you are watching here!)

Looking at image-trace-loader again, I hadn't noticed https://github.com/EmilTholin/image-trace-loader/blob/master/lib/index.js#L32-L57 either 😊 ❤️ … watchu think about that @KyleAMathews? (Been 👀 https://github.com/akfish/node-vibrant btw., uses Jimp too)

Remove unnecessary/confusing options

fk · 2017-10-20T14:52:28Z

Looking gooood: https://deploy-preview-2456--using-gatsby-image.netlify.com/traced-svg/
https://deploy-preview-2456--image-processing.netlify.com/

KyleAMathews · 2017-10-20T16:27:33Z

Looking good indeed!!!

Merging and releasing 🎉

fk · 2017-10-20T17:06:30Z

🙌

sebastienfi · 2017-10-27T01:18:50Z

Congrats

fk · 2017-10-27T15:27:07Z

Thank you @sebastienfi! 🤗

First shot at SVG „traced placeholders“

96a236b

Using node-potrace (and Jimp…) and SVGO. Only working with `resolutions` right now, also didn’t check SVG size yet.

Merge branch 'master' into topics/2435-traced-placeholder

7d5756b

sebastienfi reviewed Oct 14, 2017

View reviewed changes

fk added 2 commits October 14, 2017 20:14

Add fragments

819de3f

Yo!

00b69c0

fk added 5 commits October 15, 2017 00:12

Memoize

1c2fbc3

Support sizes()

ce59c9a

Expose potrace options

8389ea3

Add turnPolicy default

795e02d

Following to https://twitter.com/Martin_Adams/status/918481948217049088

First shot at readme

c214ea9

KyleAMathews suggested changes Oct 15, 2017

View reviewed changes

fk added 4 commits October 15, 2017 18:41

Add missing defaultValue „turnPolicy“

beaa8bc

Remove invalid „;utf8“ param

89f9d10

Remove default „trace“

da19d06

Revert ‚Remove default „trace“’, fix default color of traced SVG

6f050d0

This reverts commit da19d06. Default color of traced SVG now matches default „backgroundColor“ of the gatsby-image placeholder.

Pass „trace“ args to „tracedSVG“ subfield

fc334bc

fk commented Oct 19, 2017

View reviewed changes

Brute-force fix things

2029956

fk commented Oct 19, 2017

View reviewed changes

fk and others added 6 commits October 20, 2017 01:27

Rename arg „trace“ -> „traceSVG“

ce8635b

READMEs

74d77c0

Minor README change

3cd3930

Add example page for tracedSVG

0778ee1

… currently generating SVGs up to 1.4MB :/ ;)

Resize images before tracing the SVG

8e32e7b

Temp copy fragments

53d7b4f

Make the default background color light gray

ff3c6b4

EmilTholin reviewed Oct 20, 2017

View reviewed changes

fk added 4 commits October 20, 2017 11:20

Bring back trace args/defaults

750d702

Demo color + background

ede2e3c

Remove unnecessary/confusing options

Enable multipass, set float precision 🙏 @EmilTholin

c0fd0ca

Remove disclaimer from README

eb5c7b9

KyleAMathews changed the title ~~[WIP][gatsby-plugin-sharp] SVG traced placeholders~~ [gatsby-plugin-sharp] SVG traced placeholders Oct 20, 2017

KyleAMathews approved these changes Oct 20, 2017

View reviewed changes

KyleAMathews merged commit 0906635 into gatsbyjs:master Oct 20, 2017

fk deleted the topics/2435-traced-placeholder branch November 14, 2017 01:21

[gatsby-plugin-sharp] SVG traced placeholders #2456

[gatsby-plugin-sharp] SVG traced placeholders #2456

Conversation

fk commented Oct 14, 2017 • edited

KyleAMathews commented Oct 14, 2017 • edited

gatsbybot commented Oct 14, 2017 • edited

KyleAMathews commented Oct 14, 2017

KyleAMathews commented Oct 14, 2017

sebastienfi commented Oct 14, 2017

sebastienfi Oct 14, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KyleAMathews commented Oct 14, 2017

fk commented Oct 14, 2017

KyleAMathews commented Oct 14, 2017

KyleAMathews commented Oct 14, 2017

fk commented Oct 14, 2017

fk commented Oct 14, 2017

KyleAMathews commented Oct 14, 2017

fk commented Oct 14, 2017

fk commented Oct 14, 2017 • edited

fk commented Oct 15, 2017 • edited

KyleAMathews left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fk commented Oct 19, 2017

KyleAMathews commented Oct 19, 2017

fk commented Oct 19, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fk Oct 19, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KyleAMathews commented Oct 20, 2017

fk commented Oct 20, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fk Oct 20, 2017 • edited

Choose a reason for hiding this comment

fk commented Oct 20, 2017

KyleAMathews commented Oct 20, 2017

fk commented Oct 20, 2017

sebastienfi commented Oct 27, 2017

fk commented Oct 27, 2017

fk commented Oct 14, 2017 •

edited

KyleAMathews commented Oct 14, 2017 •

edited

gatsbybot commented Oct 14, 2017 •

edited

sebastienfi Oct 14, 2017 •

edited

fk commented Oct 14, 2017 •

edited

fk commented Oct 15, 2017 •

edited

fk Oct 19, 2017 •

edited

fk Oct 20, 2017 •

edited