Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluate and Implement HTML Macros #33

Open
rrmckinley opened this issue Apr 27, 2020 · 17 comments
Open

Evaluate and Implement HTML Macros #33

rrmckinley opened this issue Apr 27, 2020 · 17 comments

Comments

@rrmckinley
Copy link

rrmckinley commented Apr 27, 2020

I want to evaluate these HTML macro systems and any others. @Pauan and others, let me know if you have thoughts on a good html macro system for rust to enable JSX style in Dominator, thanks.

typed-html is a wonderful library. Unfortunately, it focused its power in strictness of the HTML spec itself, and doesn't allow arbitrary compositions of custom elements.

render takes a different approach. For now, HTML is not typed at all. It can get any key and get any string value. The main focus is custom components, so you can create a composable and declarative template with no runtime errors.

@rrmckinley
Copy link
Author

rrmckinley commented Apr 27, 2020

Interesting server/client context switching

Hop is multitier. That is client-side codes are also implemented in Hop. The ~{ mark switches from server-side context to client-side context:

service hello() {
  return <html><div onclick=~{ alert( "world" ) }>hello</div></html>;
}

Hop client-side code and server-side can also be mixed using the ${ mark:

service hello( { name: who } ) {
  return <html><div onclick=~{ alert( "Hi " + ${who} + "!") }>hello</div></html>;
}

@Pauan
Copy link
Owner

Pauan commented May 4, 2020

I've never liked JSX, but even ignoring my own personal tastes, there are some major issues with it.

One big issue is that JSX does not distinguish between static, dynamic, and computed properties. This is because JSX was designed for vdom, which only has static and dynamic properties.

What is the difference?

  • static properties are fixed and unchanging, such as .attribute("foo", "bar")

  • dynamic properties change over time, such as .attribute_signal("foo", ...)

  • computed properties are similar to static except the value is an expression, such as .attribute("foo", format!("{} + {}", bar, qux))

The problem with JSX is that it cannot represent this difference. If you use <div foo="bar"> then that's a static property. If you use <div foo={bar}> then that's a dynamic property. But how would you represent computed properties?

Similarly, JSX cannot distinguish between children, child_signal, text, text_signal, and children_signal_vec: JSX only has static children and dynamic children. This would make children a lot clunkier to use, because child_signal and text_signal couldn't be used.

JSX would also force dominator to create a custom component system, which is very limiting. With dominator you can just create regular Rust structs (such as using Foo::new()), and then call the Foo::render(foo) method. But with JSX it would instead need to use some weird custom component system which doesn't compose well with Rust structs, and also doesn't work well with Rust lifetimes.

JSX also has problems with events: you need to use hacky systems like onclick={...} (which hardcodes the on prefix for events), and it doesn't allow you to use multiple of the same event. In addition, dominator uses static types for events, but JSX cannot handle that, so it would lose static typing.

Another problem is that JSX does not distinguish between properties and attributes. That's really important! Most SVG requires attributes, whereas properties are sometimes needed for HTML (such as value or checked).

Another problem is: how would JSX handle custom methods? Dominator has a lot of methods like with_node, before_inserted, after_inserted, after_removed, apply, and future. Similarly, JSX cannot handle mixins, or global_event, or the shadow DOM.

Another issue is formatting. It's awkward when your JSX becomes too large to fit onto one line (which happens a lot):

<div
    foo="bar"
    qux="corge"
    onclick={|e| { ... }}
></div>

Can all these problems be fixed? Maybe, but it would require a lot of changes to JSX which would make it very different from HTML, which defeats the point of JSX (which is to be similar to HTML).

Fundamentally, HTML was never designed for making apps, it was always designed for static text. If you have a lot of text and only a few tags then HTML works great. But web apps use lots of tags and very little text, so HTML just doesn't work well.

And JSX in particular was always designed specifically for vdom, I don't think it will ever work good for FRP systems (like dominator).

Rather than trying to shoehorn HTML into web apps, I think it would be better to create a new macro syntax from scratch which fixes the above issues.

Though personally I quite like dominator's existing syntax, I spent a lot of time making it readable and maintainable (at the cost of some verbosity). I designed dominator's syntax for large real apps, not toy code examples or code golfing. In real apps you end up using a lot of signals (and computed values), so the extra verbosity of typing out .attribute or .property or .style is actually beneficial.

@Pauan
Copy link
Owner

Pauan commented May 4, 2020

P.S. The idea of having special syntax to distinguish between the client and server is cool, however there isn't any need for that in Rust, because Rust has conditional compilation (with cfg):

#[cfg(feature = "client")]
html!("div", { ... })

#[cfg(feature = "server")]
html!("div", { ... })

I could easily add a new method which would make it even easier:

html!("div", {
    .with_cfg!(feature = "client", {
        .event(|e: events::Click| {
            ...
        })
    })

    .with_cfg!(feature = "server", {
        ...
    })
})

@dakom
Copy link

dakom commented May 5, 2020

@Pauan - the following may be a small detour but I think it's coming from exactly the same "big picture" problem, and is worth sharing...

The last relatively large Typescript+React project I built was a mess. Not because of Typescript exactly or React exactly - but because once things started getting bigger and needed to go outside the typical TodoMVC structure, it just didn't scale. I don't mean in terms of "number of components" but more like scale to deal with "different kinds of problems". And the transpiler didn't save me as much as I wanted it to. Fwiw I had experimented with a lot of ways to do things - and also used XState for a lot of the state management stuff. All in all the project was fine and worked and all, but I'm embarrassed by the codebase. There's just all these weird useEffect hacks and casting to any and... well, I'm just glad its over.

My point isn't about that other tech, and I don't want to debate its pros and cons - my point is that what you wrote here is crucial:

I designed dominator's syntax for large real apps, not toy code examples or code golfing. In real apps you end up using a lot of signals (and computed values), so the extra verbosity of typing out .attribute or .property or .style is actually beneficial.

Generally speaking - I think you're moving the inevitable complexity of large real apps into the right place, where it's typechecked and clear. This is the main reason I'm interested in Dominator tbh... performance is icing on the cake (within a reasonable threshhold).

So why am I commenting on this particular issue?

The other thing that happens with large real apps is, often enough, the need to work with a team. That team, on the web frontend, right now, will likely be very allergic to Rust. In a non-toy non-personal project setting, telling other frontend devs to learn Rust, or even enough Rust to get by, might not be realistic for all sorts of reasons (not always technical).

So, I don't know what the solution for this is. Lets assume for the sake of convenience that "them" here is someone who is focused solely on html/js/css, and the "me" here is much happier working in Rust. Let's also assume that "them" is focused solely on presentation, the JS they would write is very minimal. Some ideas:

  1. Have them write web components in their JS framework of choice, and then I target those web components from Dominator (changing their properties via signals, passing callbacks down, etc.). I've been experimenting with this in a few different ways... it works quite well in small proofs of concept, but my spidey sense is going off like crazy that it's not going to scale - for exactly the reasons you mentioned! At first it's beautiful to have "presentational" logic and state totally separate from "app/business" logic and state - but it gets muddy due to the particular overlap here, and I don't think it's actually going to scale well. I'd anticipate a lot of friction about what should be done where. In theory it would be clear - but in reality, I don't think it will be.

  2. The inverse: don't use Dominator - rather, do the frontend DOM work in, whatever, and then call into Rust libraries for logic/state management. I've been experimenting with this - but again, it is setting off my spidey sense that while it works great for small toy examples, it's not going to scale well and will require a lot of careful communication to keep things in sync. Already I'm hitting issues of needing to call things like get_user() from different places on the JS side, and serializing through JsValue means the typescript doesn't come through. There's a lot of manual bookkeeping.

  3. Have them write clean, non-stateful components in plain html/css - perhaps in a visual reference library like storybook, and then re-create that from scratch in Dominator (sortof the way they will base their code on a design). This requires keeping them in sync, and is creating some double-work. I'm starting to lean toward this frankly because I've exhausted the above two options and this is the only one that's left. However, this route has its own problems, like if its possible to at least re-use the .css files (and respecting the encapsulation).

  4. Some HTML macro thing that lets them edit HTML and CSS in my Rust source without having to know Rust. There can be a few simple "syntax" rules they can learn - like the need to quote strings... but, it can't be too different. They should basically be able to copy/paste from codepen.

I'm not sure there's a perfect answer here... but I figured it's worth commenting since it's very much on my mind. I'm currently leaning toward option 3 though. I'm skeptical that 4 will really be able to address everything, but very happy if it can!

@dakom
Copy link

dakom commented May 5, 2020

The more I think about it, the more option 3 makes sense.

It's not unlike how a design will often progress from static wireframes to rough mockups to complete but static references to interactive, perhaps even responsive references.

This is just adding another step to the design phase where the programmer or, let's say "technical layout artist", is bringing it to its next incarnation. These could even be user tested and vetted independently before going into full-on production. With a bit of JS sprinkled in they could probably even be done very close to the final thing (from a user's perspective).

Yes there's a bit of double-work, but we accept that when going between other design phases because the tradeoff is worth it, and I think the same idea applies here.

That then leaves the app code itself to be 100% Rust, with a pure focus, and the html/css is also done in its focused sandbox.

With that in mind, I don't think I personally have a need for html macros - though I'm happy to re-evaluate if/when you add them :)

@Pauan
Copy link
Owner

Pauan commented May 5, 2020

@dakom At first it's beautiful to have "presentational" logic and state totally separate from "app/business" logic and state - but it gets muddy due to the particular overlap here, and I don't think it's actually going to scale well.

I agree, I think it's a mistake to try and separate presentation from logic. Logic separation made sense for websites, which were primarily text content with some progressive enhancement, but it makes no sense at all for web apps.

I don't know of any non-web GUI framework that has a clean separation between presentation and logic, in fact they're usually intrinsically tied together. That's because in reality they are tied together, and there's just no way to cleanly separate them.

The inverse: don't use Dominator - rather, do the frontend DOM work in

I don't think that's a good idea. It's hard enough to do state management with all the tools available to you (Rust, dominator, Signals), it's even harder when you have to juggle between JS and Rust.

Have them write clean, non-stateful components in plain html/css - perhaps in a visual reference library like storybook, and then re-create that from scratch in Dominator (sortof the way they will base their code on a design).

I think that's a reasonable idea. Many designs don't even start with HTML, they start off with rough paper sketches, then more polished ink sketches, then HTML. So the idea of creating the design in stages makes a lot of sense.

However, this route has its own problems, like if its possible to at least re-use the .css files (and respecting the encapsulation).

That's not a problem, dominator can do everything the web can, so of course it can load .css files. It's not idiomatic to do that, since class! gives better encapsulation, but you don't need to use class!, you can use regular .css just fine (in fact, that's what the TodoMVC example does).

Some HTML macro thing that lets them edit HTML and CSS in my Rust source without having to know Rust.

Rather than having a macro, I think a better idea is to have some sort of CLI tool that can take in HTML and spit out some Rust code. Then you would just need to add interactivity (with signals/events) to the generated Rust code.

Now they can create their design entirely in Codepen (or wherever) using regular HTML + CSS, then just run the tool which will generate dominator code from it.

@dakom
Copy link

dakom commented May 5, 2020

Now they can create their design entirely in Codepen (or wherever) using regular HTML + CSS, then just run the tool which will generate dominator code from it.

That is a really interesting idea! Could be like the way diesel does its thing... (write sql, run a cli tool, it generates rust macros, which then generate rust structs)

@Pauan
Copy link
Owner

Pauan commented May 5, 2020

@dakom Yes, and that means they can use whatever tools they are familiar with (HTML and CSS obviously, but also Codepen, etc.) They don't need to learn anything new, their existing workflow can stay the same.

And since the HTML is static, there aren't any computed or dynamic attributes, and there aren't any events, so all of the downsides I mentioned above don't apply.

But you still get the benefits of having everything done in dominator (which gives you the benefits of signals, events, futures, streams, static typing, etc.)

@dakom
Copy link

dakom commented May 6, 2020

I'm having a bit of trouble envisioning exactly how this would work - since I'd still need to take the result of the cli tool and add in all the functionality (like changing properties on signals, nesting dynamic children, etc.)

Excited to see where this goes though! :)

@Pauan
Copy link
Owner

Pauan commented May 6, 2020

since I'd still need to take the result of the cli tool and add in all the functionality (like changing properties on signals, nesting dynamic children, etc.)

Well of course, the designer isn't adding in any of that, so you will have to. The point isn't to automate everything (ultimately you still have to do the porting by hand), the point is to avoid the tedious work of manually converting HTML tags, classes, and attributes into dominator. The actual tricky parts will still have to be done by hand. So it doesn't replace option 3, it instead just makes option 3 a bit easier and faster.

@dakom
Copy link

dakom commented May 6, 2020

ok cool, and for unsupported things it will just ignore them? For example, given this:

<div class="menu">
  <ul class="left big">
    <li onclick="do_something(1)">child 1</li>
    <li onclick="do_something(2)">child 2</li>
  </ul>
</div>

would it ignore the onclick and generate something like this?

html!("div", {
  .class("menu")
  .children(&mut [
    html!("ul", {
      .class("left")
      .class("big")
      .children(&mut [
        html!("li", {
          .text("child 1")
        }),
        html!("li", {
          .text("child 2")
        })
      ])
    })
  ])
})

Or, alternatively - since it's not meant to be a perfect drop-in replacement, maybe it could add in the .event() stubs?

@dakom
Copy link

dakom commented May 6, 2020

Another idea - is to create this as a npm/js package (whether it's developed in Rust or not).

That way it could be added as a plugin to Storybook...

So, designer/html person creates things in storybook, and then anyone can hit a button to "generate DomBuilder code"

@dakom
Copy link

dakom commented May 6, 2020

Ok, check it out: https://github.com/dakom/storybook-for-dominator-boilerplate

This could be a really nice workflow :)

For the HTML->Dominator string conversion, I created a new npm package... since it's running in the DOM anyway it takes advantage of that and just walks through an ad-hoc element.

The code for that is not very elegant, but it's a start - and upgrades to the package won't break Storybook just work since it's just String -> String

@L1lith
Copy link

L1lith commented Oct 11, 2021

@dakom At first it's beautiful to have "presentational" logic and state totally separate from "app/business" logic and state - but it gets muddy due to the particular overlap here, and I don't think it's actually going to scale well.

I agree, I think it's a mistake to try and separate presentation from logic. Logic separation made sense for websites, which were primarily text content with some progressive enhancement, but it makes no sense at all for web apps.

I don't know of any non-web GUI framework that has a clean separation between presentation and logic, in fact they're usually intrinsically tied together. That's because in reality they are tied together, and there's just no way to cleanly separate them.

The inverse: don't use Dominator - rather, do the frontend DOM work in

I don't think that's a good idea. It's hard enough to do state management with all the tools available to you (Rust, dominator, Signals), it's even harder when you have to juggle between JS and Rust.

Have them write clean, non-stateful components in plain html/css - perhaps in a visual reference library like storybook, and then re-create that from scratch in Dominator (sortof the way they will base their code on a design).

I think that's a reasonable idea. Many designs don't even start with HTML, they start off with rough paper sketches, then more polished ink sketches, then HTML. So the idea of creating the design in stages makes a lot of sense.

However, this route has its own problems, like if its possible to at least re-use the .css files (and respecting the encapsulation).

That's not a problem, dominator can do everything the web can, so of course it can load .css files. It's not idiomatic to do that, since class! gives better encapsulation, but you don't need to use class!, you can use regular .css just fine (in fact, that's what the TodoMVC example does).

Some HTML macro thing that lets them edit HTML and CSS in my Rust source without having to know Rust.

Rather than having a macro, I think a better idea is to have some sort of CLI tool that can take in HTML and spit out some Rust code. Then you would just need to add interactivity (with signals/events) to the generated Rust code.

Now they can create their design entirely in Codepen (or wherever) using regular HTML + CSS, then just run the tool which will generate dominator code from it.

I've been studying "stack" architecture while developing in the JS ecosphere. To me there are a few features that make app development stand out.

Performance vs Development Ease

I'm a giant fan of Javascript, but I think it needs to be used in efficient ways. I still get triggered by jQuery, and that's why I like the premise of JS minimalism (something Dominator seems to value). One of my favorite things about the JS language and tooling is how expressive it is, but it comes at a cost (specifically bundle size and evaluation time when loading React and other dependencies). This part of Dominators docs was exciting to me

It does not use VDOM, instead it uses raw DOM nodes for maximum performance. It is close to the metal and has almost no overhead: everything is inlined to raw DOM operations.

One of the reasons I've been recently looking at the Rust language is because of how performant and flexible it is. From what I understand macros enable us to use metaprogramming features to allow us to introduce "language extensions" which are then parsed into code at compile time. When you consider the JSX syntax I believe this could provide amazing potential in terms of ease of expression, while skipping out on the majority of the performance implications (as the syntax can be translated at compile time into raw HTML & JS).

Syntax Comparison

Here's what our Dominator component looks like

html!("div", {
            .class(&*CLASS)

            .text_signal(app.message.signal_cloned().map(|message| {
                format!("Message: {}", message)
            }))

            .event(clone!(app => move |_: events::Click| {
                app.message.set_neq("Goodbye!".to_string());
            }))
        })

This takes an overwhelming number of parenthesis and brackets, let's imagine something similar in JSX style. It might have roughly this structure

jsx!(<div class="ourClass" onClick={listener here}>Message: {message}</div>)

It seems like the HTML like syntax is a much cleaner way to represent what is ultimately a bunch of DOM nodes. I know you had some concerns about hydrations as well but I think these issues could be dealt with.

True Isomorphism

One issue would be that it's usually dependant on React. While it normally would need React to render, it doesn't actually need React at all. JSX simply represents a syntax tree, which can be broken down and completely rewritten as barebones HTML and JS (or even any other language). This is a great thing for helping developers write less code during development and reduce redundancy in their codebase.

The Svelte JS framework employs some particularly neat tricks in order to push the limits of minification (you can read a write-up about it here), but basically it skips the virtual dom just like Dominator. Astro.build is also an incredible framework that includes react, but prerenders the entire page as HTML from that same react code then hydrates it after.

I think these frameworks provide evidence that the strongest approach for web frameworks are ones that maximize performance while simultaneously providing seamless ease of development and integration. React got popular for a reason despite the performance flaws, but I think there's still a lot of room to improve, especially with the help of Rust macros. That doesn't have to mean JSX specificially but just generally integrating HTML, JS (or RS), and CSS syntax. I hope that Rust developers will head more in this direction in the future.

Small tangent on bundle generation

Small tangent: Earlier you were talking about how you couldn't separate the dynamic code (like how JSX has the dynamically rendered {} syntax). From my understanding this isn't necessarily the case. We simply evaluate the JS (or the RS macro) at compile time, and anything that is asynchronous is loaded after the initial page render. The expression <div>{2 + 2}</div> has no qualms being evaluated and turned into <div>4</div> before the HTML is even generated. However, the expression jsx!(<div class="carousel" onClick={handler}/>) would need to be split into the HTML and the deferred JS/RS bundle. The HTML bundle would look something like <div id="targetElement" class="carousel"></div> and the JS could be converted into "raw JS" looking something like document.getElementById("targetElement").addEventListener('click', handler /* our handler stripped out of the JSX*/). This doesn't necessarily have to be the specific implementation, but it's rather just a general idea how it could be written isomorphically in Rust using Macros. We can automatically detect which parts of the application need to be either compiled or run on load by determining whether they are synchronous, or asynchronous javascript calls (or maybe Rust calls idk).

Thanks for the consideration 😊 I'm kind of tired while writing this so I apologize for any redundancies

@Pauan
Copy link
Owner

Pauan commented Oct 14, 2021

@L1lith When you consider the JSX syntax I believe this could provide amazing potential in terms of ease of expression, while skipping out on the majority of the performance implications (as the syntax can be translated at compile time into raw HTML & JS).

Yes, that is true, and that's also true of the html! macro in dominator, it expands to raw DOM operations, so there is no cost.

It seems like the HTML like syntax is a much cleaner way to represent what is ultimately a bunch of DOM nodes.

Dominator is not designed for code golfing, my goal was not to have the shortest possible syntax. My goal was to create syntax which makes it possible to maintain large applications (100,000+ lines of code).

When creating tiny programs, the syntax does not matter, anything will work just fine. But when creating big programs, now the syntax makes a big difference.

JSX only works well for short one-liners, but in practice you will have many attributes, event listeners, styles, signals, etc. As soon as you need to split JSX onto multiple lines, it loses its appeal.

Also, your comparison is not fair, because the JSX code is not using signals. So this is a fair comparison:

html!("div", {
    .class(&*CLASS)
    .event(clone!(app => move |_: events::Click| {
        app.message.set_neq("Goodbye!".to_string());
    }))
    .text_signal(app.message.signal_cloned().map(|message| {
        format!("Message: {}", message)
    }))
})
jsx!(<div
    class={&*CLASS}
    onClick={clone!(app => move |_| {
        app.message.set_neq("Goodbye!".to_string());
    })}>
    {app.message.signal_cloned().map(|message| {
        format!("Message: {}", message)
    })}
</div>)

Or if you prefer one-liners...

html!("div", { .class("ourClass") .event(listener here) .text(format!("Message: {message}")) })
jsx!(<div class="ourClass" onClick={listener here}>Message: {message}</div>)

I think the dominator syntax is much clearer, much more understandable, and much easier to maintain. And as the number of attributes increases, the dominator syntax becomes even better.

JSX only wins in extremely short static one-liners. As soon as you add in signals, events, etc. then it becomes a lot worse.

Here is an example of a real app that I wrote. Notice how it has 19 method calls for a single DOM node. Also notice that almost everything is a signal, not static. Also notice how clear and easy it is to understand: you know exactly what each method call is doing, and everything is formatted nicely. Writing it in JSX would look far worse.

In practice I basically never create tiny one-liner DOM nodes, and even when I do it's not a big deal, it's just a few lines of code. Dominator is not designed to make a 3-line DOM node shorter, dominator is designed to make a 50-line DOM node clearer and more maintainable.

The Svelte JS framework employs some particularly neat tricks in order to push the limits of minification (you can read a write-up about it here), but basically it skips the virtual dom just like Dominator. Astro.build is also an incredible framework that includes react, but prerenders the entire page as HTML from that same react code then hydrates it after.

Neither of those frameworks are FRP. Also, my goal was not to copy existing JS frameworks, my goal was to create the best possible Rust framework, and Rust imposes a lot of restrictions and design choices which JS doesn't have. You can't just copy a JS framework (or JS program) into Rust and expect it to work.

That doesn't have to mean JSX specificially but just generally integrating HTML, JS (or RS), and CSS syntax.

As I explained previously, I don't think that's the right approach. HTML, JS, and CSS were not designed for creating web apps, trying to use them to create web apps results in a lot of problems, especially with an FRP framework.

That's why every FRP framework has created its own non-HTML syntax (e.g. Cycle.js, Elm, Reflex, Conductance, etc.)

We simply evaluate the JS (or the RS macro) at compile time, and anything that is asynchronous is loaded after the initial page render.

I don't think you are understanding the problem. We are writing Rust, not JS. Rust is statically typed, which means there is a fundamental type difference between a u32 and a Signal<Item = u32>, and so they cannot be handled with the same API (at least not without a lot of trait magic). That's why it has to be distinguished in the syntax (or in the case of dominator, with a different method call).

The same thing happens with children, children_signal_vec, child, child_signal, text, text_signal, etc. They are all different static types, and so they need to use different syntax or different APIs.

Also, I really like having a clear distinction between computed and dynamic values, because it means that you can tell at a glance what is changing and what is not changing. This helps a lot with maintainability. JSX (and JS frameworks in general) don't have that benefit.

@Slyker
Copy link

Slyker commented Jan 15, 2024

Hi guys why not using something like Svelte Compiler that could compile html file with rust script directly into Dominator compliant scripts

@Pauan
Copy link
Owner

Pauan commented Jan 15, 2024

@Slyker Svelte needs a special compiler, because JavaScript is very limited. But Rust has macros, which are very powerful, so it's not necessary to have a special compiler, because macros do the same thing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants