Understand oneDNN graph compiler

[akote123]

Hi,
I wanted to understand from framework like Tensorflow or Pytorch how we can enable graph compiler.

cc:@ZhennanQin

Thank you

[ZhennanQin]

Hi Aruna,

The framework integration is WIP, you may check the status from framework RFC. Currently, the recommended way to try graph compiler with pytorch, is to use IPEX. Graph compiler is enabled by default in its INT8 path, for other data types like float32 or bfloat16, you need to manually turn on oneDNN graph by inserting the below line in the beginning of your model script:

ipex._C.set_llga_fp32_bf16_enabled(True)

[akote123]

@ZhennanQin Thank you

[akote123]

@ZhennanQin , I was going through the https://arxiv.org/abs/2301.01333, after TensorIR how it calls batch reduce kernel and also I want understand where exactly the microkernel file location in oneDNN repository.

[ZhennanQin]

The brgemm interface location: https://github.com/oneapi-src/oneDNN/blob/main/src/cpu/x64/brgemm/brgemm.hpp

[akote123]

@ZhennanQin ,Graph compiler calls microkernel from primitive APIs?
How Tensor IR is connected to this?

[ZhennanQin]

Graph compiler shares the same microkernel implementation with primitive. The brgemm abstraction in graph compiler can be found at: https://github.com/oneapi-src/oneDNN/blob/main/src/graph/backend/graph_compiler/core/src/runtime/microkernel/cpu/microkernel.hpp

[akote123]

@ZhennanQin , Thank you.