Project: Auto Generating WASM Demo

[ashu26jha]

Originally suggested by @QnnOkabayashi, also present in GSoC Project Ideas.

There is an actual need to have the demo be auto generated, currently we only have 3 demos which is not a good coverage. Here is how I could think about it:

1. We could possibly take input in form of a JSON file, which defines input fields and thier types.
2. Need to add setter functions for those inputs mentioned above
3. We could setup a frontend site which will generate JSON for the developers (select number of input fields)

Also discuss with: @sffc @Manishearth @robertbastian

[Manishearth]

The JSON model is a good first pass. We were hoping that eventually Diplomat can have a way to do this, but that's a much trickier project.

[ashu26jha]

Basically we need to call those generated bindings and interact with them in frontend?

@Manishearth thanks for brief but I do have couple of questions:

1. Using JSON for annotations seems fine but are there any other methods which could be used?
2. Any projects which tries to achieve similar goals would definitely help?

A small suggestion please review it:

1. Also I think we could add a test on default inputs to check if it is working or not (Maybe adding to CI)

Anything else to look upon that could help?

[Manishearth]

Using JSON for annotations seems fine but are there any other methods which could be used?

No, the annotations would be Diplomat attributes. You'd first have to come up with a complex model for what annotations need to exist.

Any projects which tries to achieve similar goals would definitely help?

I don't know what you mean by this

Also I think we could add a test on default inputs to check if it is working or not (Maybe adding to CI)

Also not sure what you mean by this

[ashu26jha]

Apologies for my unclear comment,

1. Is there any open source project that Auto Generates demos like this? (Anything could work)

2. We could add a check whether the generated Demo is working as expected or not.

[Manishearth]

I don't think such a project exists.

Tests could be added, testing websites are tricky.

[ashu26jha]

Even more tricky if they are not static 🙃🙂

[sffc]

Hey @ashu26jha @ayushkoli772 @man-myth @yatigautam and anyone else interested in WASM: I realized that a few days ago we merged a PR that broke the WASM Demo build, which I just fixed in #4737. Also see #4738 for a potential fix if you are running Node 18 (note that in CI we use Node 16).

[ashu26jha]

I think firstly we need to discuss about the annotation model / schema. For now let's currently not focus on how these annotations will be generated.

The annotation schema can be broken down into two pieces:

1. Default/Boiler Annotations: This may include Locale, DisplayFn, DataProvider. The idea is no matter whatever component of icu4x we use, the above will always be used in every demo.
2. Specific Annotations: This will include special input fields, output fields etc.

We will add special tags (#[auto_update]) to the annotations specified above that if these are modify, update our formatter and re-render the output (just like our current WASM demo). Example for further clarification:

Lets take FixedDecimal for example:

#[demo(
    title = "Decimal Format",
    description = "Demonstrates formatting numbers with different grouping strategies"
)] 

pub struct DecimalFormatDemo {

    // Default Annotations

    #[auto_update] // Changes to this will call update_formatter and re-render the output
    #[demo(input(label = "Locale", type = "dropdown")]
    locale: ICU4XLocale,

    // Specific Annotations
 
    #[demo(input(label = "Number"))]
    number: f64

    #[auto_update] // Changes to this will call update_formatter and re-render the output
    #[demo(
        input(label = "Grouping Strategy", type = "dropdown", options = "Default, OnDigit, Min2")
    )]
    grouping_strategy: ICU4XNumberGroupingStrategy,

    #[demo(formatter(inputs = ["data_provider", "locale", "grouping_strategy"]))] 
    #[formater_function] // #[auto_update] should call these function in case those values are changed
    formatter: Result<ICU4XNumberFormat>, 

    #[demo(output(type = "text"))] 
    formatted_value: Result<ICU4XFixedDecimal> | null,
}

CC @Manishearth @sffc

If this looks good enough, I think about how can we generate these and eventually have parsing algorithms in place

[Manishearth]

I do not think we should start with a model of how to do this as annotations on Rust code until you have a clearer understanding of Diplomat's attributes system, which will take time. The above example would not work well with that; creating a new Demo object would not really fit in the Diplomat model.

I think we should start with having a separate way of driving demo generation that may consume the Diplomat HIR if it needs, but does not function as a Diplomat backend.

[ashu26jha]

Could you suggest some starting points or references?

PS: I don't think I would be able to draft GSoC proposal in time, anyway lets forget about that and lets focus on the broader goal of the project shall we?

[Manishearth]

I don't think I have any starting points: I think the data model you propose above is decent it just needs to not be in the form of annotations to Rust code

[sffc]

I don't think we need to make an MVP be attribute-based. All it needs to do is give a way to construct the arguments required to call any function, which may involve calling other functions. Basically a graphical REPL. The attributes could be added later to abstract over certain common patterns.

[sffc]

@ashu26jha @Manishearth

What I had in mind was basically a web interface to drive the whole ICU4X FFI. Every API would be generated with UI elements for controlling it, like dropdowns for enums, text boxes for strings, etc. I think one of the more interesting challenges would be figuring out how to pass opaques into functions and also how to set the receiver (self variable). But, I think this can be mostly achieved by writing a Diplomat target on top of HIR, hopefully without too much hacking necessary in the Diplomat core framework.

[ashu26jha]

This is going to be a long one:

1. We need to make the process of adding demos easier i.e. process of creating the JSON must be easy and user friendly. To create the JSON which will consist of Diplomat Attributes using which our demos will be built. We can select any number of inputs/outputs the type of input can be selected from a dropdown menu which will have multiple options from d.ts files. This is probably the easiest of all the steps involved and shouldn't take a lot of time.

2. Now once we have the JSON (a sample JSON has been attached as well). We can use this metadata to generate input and output codes. It will consist of some boilerplate / repeated code as well, such as Locales, DataProvider etc.

3. Just like our current demos, changing input fields re-render the output, we do need to make sure our project reflects that as well. Every input will have a boolean attribute auto_update any change to this would re-render the output. We can add IDs to UI by following a simple convention: demo_name + input_name.

I think we should start by building UI for basic (easier) components like Collator followed by Decimal Formatter etc. Fallback to improving our JSON model whenever necessary. Trying to reduce upstream changes in Diplomat.

When this project will be completed, the final JSON model would act as foundation stone for adding this feature in the Diplomat itself (using Rust based annotation demonstrated above). At that point we will be comfortable writing such code.

Here is the canva link, (I am on my way to learn Figma 🙂)