MLIR support

[lutzroeder]

• JavaScript MLIR parser
• Object Model

Examples:
1043.mlir.zip
examples.mnist_xla.mlir.zip

[tucan9389]

@lutzroeder
First of all, thanks for your issue. And thanks for the repo as a person who has used this tool for almost 4 years.

And I'm personally interested in this issue.

Would you happen to have any good references to make some progress on this?
I'm trying to understand the overall code structure and the detailed implementation (especially for tflite and coreml), but it's pretty hard to understand at once. Would you give some guides or tips? I think that good reference commits might be helpful for understanding the things that I should implement.

[lutzroeder]

Initially this is likely about writing a JavaScript parser for MLIR to tokenize the file and extract the module structure. onnx.Reader.text as a starting point of what such code looks like for a similar text format.

There is already a mlir.ModelFactory which gets invoked when dragging an .mlir file into a view. The rest of the work is about mapping the parsed MLIR module into the object model the viewer consumes. CONTRIBUTING.md has steps to launch a debugger. node ./test/models.js mlir to run the existing test files. dnn.js is a format with a short implementation hat might be easier to follow to learn about the basic graph, node, attribute and parameter types in the object model.

[tucan9389]

@lutzroeder
Gently reminder. Could you review #1088?

And I'm planning quite a big update for testing with the .mlir files on Github and reinforcement of the mlir.Tokenizer and mlir.Parser based on the #1088. If you think we should have a #1088 review first and move forward, please tell me ASAP.

Thanks :)

[lutzroeder]

@tucan9389 fell free to continue iterating on the pull request. Sorry I wasn't clear, by E2E I meant once its in a good shape to move towards adding it to a release.

[tucan9389]

@lutzroeder
Oh, I see. The MLIR parser can be a massive complex. Before it gets too big, I thought we could merge or get some reviews. I'm not sure if this contribution adds value to this repository.

Do you happen to have a useful guide for releasing this feature or any test cases to merge into the main branch?

I would prefer to get reviews in between rather than getting big reviews at once, but if it's difficult for you now, could you please let me know the minimum requirements for merging? I wonder if you have thought about the work unit to merge.

[tucan9389]

@lutzroeder
Hi, I made some additional progress.
Please check tucan9389#1 out.

I think this can be merged into the main repo. Can you review and pull my implementation?
If you think there are remaining works, please let me know.

This is my side project, so the next progress could take some time.

Anyway, thanks for your comment and advice :)

[tucan9389]

@lutzroeder
Hi, I found that my previous implementation (to parse a text MLIR file in js level) was not a good way to achieve sustainability.
I'm looking for a more proper way to use WebAssembly in netron to use MLIR parser directly. This can make parse MLIR graphs with ease and with less error. I'll comment if I make some progress.

Here the keywords are that I'm looking at:

• WebAssembly
• Emscripten (emcc command to generate *.wsam file)
• mlir/Parser.h
• How to deploy it in netron environment
• etc.

[lutzroeder]

@tucan9389 do you have some additional context what you mean by "achieve sustainability"? The goal is to avoid dependencies, run in many environments and support a unified debugging experience. Improving the JavaScript parser implementation seems the right path to get there.