-
-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[p5.js 2.0 RFC Proposal]: Shader Hooks #7031
Comments
I'd love to make shaders more accessible to beginners – they're neat. A few questions come to mind:
|
None so far, but I'll keep this updated if anything new comes up.
I don't think these are mutually exclusive approaches actually. I think the best case scenario is you have both, as they tackle separate bits of complexity. Hooks address the fact that a shader as a whole entity requires a lot of knowledge about the whole shader pipeline and data flow through it, letting you focus on just one piece at a time for less cognitive load. Js-to-glsl addresses the fact that you also need to know a whole separate language in order to write shaders, which is also additional complexity. I think the hooks problem is the easier one to tackle first for a few reasons. There are a lot of features in GLSL since it's a whole programming language, so there's just a lot of surface area to cover to be able to write anything you'd want to do in GLSL via JavaScript. I don't think you need full coverage for it to be useful, but if you have the option of writing actual GLSL too, then you've got a nice fallback for edge cases outside of what we cover. I imagine in the future, rather than just writing a GLSL string, you'd call some p5 methods that at also support One other point I'll mention is that we've put more time thinking about the API for hooks already, and probably the riskiest part was the API planning so that we make something we can support going forward. We haven't done that thinking and de-risking yet for js-to-glsl, so that will add to its development time. Definitely something to start thinking about, but I think probably not actionable yet in the near term for 2.0.
I have a functional prototype with a slightly less refined interface, where it probably is only a day or two of development to get the API to match the proposal. The longer chunk of work, I think, will be documenting all the hooks added into the core shaders. I think it's also OK for that to grow over time, maybe only needing examples for the most useful hooks, and just documenting the types + a description for the other ones at first. I'd like to spend maybe a week getting those high priority ones looking good, and then we could ask for community help on adding to the rest. For a js-to-glsl project, I think it'll take a bit of R&D at first. There are a lot of different approaches one could take that all have different tradeoffs. A Shader Park style system has you write what looks like a real js function, which it then reads as a source code string, parses, and recompiles. You could also go for a
You could make something like this outside of core p5 and it would work, but it means that the shaders go out of date whenever core p5 changes, and the addon is responsible for copy and pasting source code whenever the p5 source changes. That's currently what p5.warp does, and it means the library breaks slightly with every new version. One of the reason why I think hooks would be the part to do in 2.0 rather than js-to-glsl is that I think js-to-glsl is maybe a better fit for an addon. As long as core accepts a glsl string, then that's an interchange format between core and the addon that the addon can build for, and since GLSL will not be changing like p5 does, it doesn't have the same problems with going out of date. |
Increasing access
Which types of changes would be made?
Most appropriate sub-area of p5.js?
What's the problem?
tl;dr: if you want to write a shader for p5.js, you have to write the whole thing yourself.
What's the solution?
This has been discussed a bit in #6144 (comment), but here's a slightly more refined API:
We provide accessors for our default shaders, such as
materialShader
,normalShader
, etc.We add a method to
p5.Shader
that lets you override different aspects of its behaviour, one function at a time. I'm going to refer to each function as a "hook." An example might begetWorldPosition()
, which returns the coordinate of a vertex in world space. If you wanted to modify the default material to make all the vertices wiggle up and down, it would look like this:The
modify
function takes in an object where entries contain GLSL implementations of whichever hooks you want to override.You would then use your modified shader like a normal shader:
I've made a prototype with a similar API here as a proof of concept: https://editor.p5js.org/davepagurek/sketches/FAHwrLMAD
We update the
createShader
API to allow creation of these hooks. I'm thinking that will be done by passing an options object as a final parameter, which includes the default implementations of hooks. This API, being for core and library developers, is more verbose thanmodify
: you have to specify which hooks go in the vertex vs the fragment shader, and you have to specify the return type of the functions too.In your shader source, you can then assume that the function
HOOK_hookName
exists. For example:Under the hood,
modify()
will look through the vertex and fragment hooks to see which one matches the (return type free) name. This hopefully frees up some cognitive load from end users.Since the shader may want to only call the hook if it exists rather than relying on a function call that does nothing, I'm thinking it'd be useful to also track when a user has called
modify
and add a#define HOOK_MODIFIED_hookName
to the shader too.We create and document hooks for relevant parts of all our default shaders. I've done that for the default material shader in this demo sketch. On the docs for
materialShader
and the other default shaders, we can then list all the available hooks, and show interesting examples of what you might do with them.Pros (updated based on community comments)
Cons (updated based on community comments)
Proposal status
Under review
The text was updated successfully, but these errors were encountered: