MLIR Integration in Pyccel: Unleashing Low-Level Power

[mkaddani]

Context

MLIR
(Multi-level Intermediate Representation) is a modern compiler infrastructure which is reusable and extensible. It reduces the cost of building domain-specfic compilers by facilitating the design and implementation of code generators, translators, and optimizers at different abstraction levels.
MLIR is a subproject of the LLVM project, and has many similarities to the LLVM compiler infrastructure. ¹

PYCCEL
A Python library which acts as a transpiler by translating
Python code to either Fortran or C code, and as an accelerator by making the generated code
callable from Python once again. ²

General

• Pyccel processes Python code and convert it into C (or Fortran) code. Following this, a compiler (GCC or Clang or Flang), acting as a backend, is utilized to compile the generated C (or Fortran) code into an executable (Binary) form.
• Within Pyccel, C (or Fortran) generated code can be regarded as an Intermediate Representation (IR).
• Both C and Fortran are considered as low-level languages; however, when viewed from the perspective of MLIR/LLVM, they are still classified as high-level languages that require further lowering and optimization.
• On the other hand, MLIR provide lot of features and controll and Optimization Passes, that unfortunately cannot be fully achieved through compiler flags like -O3.
• There is currently no C/C++ or another language/programming model frontend that emits MLIR. ³

Objective

The objective of this discussion is to initiate a conversation and collect ideas about the potential requirement of MLIR for Pyccel. and to explore the possible ways of integrating MLIR as an extension for Pyccel (considering the implementation aspect).

Footnotes

1. Compiling ONNX Neural Network Models Using
   MLIR ↩

2. Pyccel: a Python-to-X transpiler for scientific
   high-performance computing ↩

3. High Performance Code Generation in MLIR: an Early
   Case Study With Gemm ↩

[mkaddani]

• List of Projects that use MLIR.

[saidctb]

In my opinion, MLIR is the future as more projects rely on it, the big question is how we can support it. it would be nice to have a small example of Python code with some benchmarks where we show all the necessary steps, from the Python code to the MLIR representation and then we compile it and call it back from Python.