DaCe workflow

[bartokon]

Hello!

So basically you can write some program in high level API NumPy or matlab (I don't see any examples in matlab?) and auto generate some Sdfg aka the working graph.

Alternatively you could directly program Sdfg by using low level api, add nodes, operations, variables.

Next step is to use diode to change graph behaviour and optimise operations while targeting some device architecture like gpu or fpga and/or perform auto optimisations.

Last step use Sdfg to generate some code like .Cpp .Cuda .opencl?

I'm getting this right?

Thanks for info.

[tbennun]

Yes, that's right!

You write your code in a high-level language, mostly in Python and NumPy (our MATLAB/Octave support is highly experimental at the moment, but there are a few examples here). Alternatively, if you know your exact dataflow or are writing a new frontend (translation from another language into SDFG), you can manually create the graph itself.

Then, you can either try to call the function from Python as-is, use the auto_optimize flag on the @dace.program decorator to try and automatically get a performance boost, or manually transform the graph. The workflow is unique in that you can see exactly what optimizations have been applied and can be applied, as well as tune all the performance aspects directly, without changing the high-level code again. You can do that by:

• Calling the Python transformation API
• Using the Visual Studio Code extension (there are some nice example gifs in the link)

DIODE is now deprecated in favor of the VSCode extension, since the new one can transform just as well and also edit the graphs directly.

Lastly, you can compile the resulting graph through the Python API (sdfg.compile()) or through the CLI tool sdfgcc, which just loads the SDFG and compiles it with the API. Compiled SDFGs create .so/.dll/.dylib files with a C API that you can use in existing programs, or you can load the graph with the Python API and call it within Python directly.

Hope this helps! Please let us know if there are any further questions.

[bartokon]

Hi!

How can I load sdfg file from python interface? I need to perform some manual fpga opt as local arrays are:

1. Stored in global memory (fix: fpga_auto_opt.fpga_global_to_local(sdfg) it works sometimes as I have BIG boolean array...)
2. Not partitioned (is there a way to do that in python API?
   like: var = dace.define_local(shape=(nb_of_photos, size), dtype=dace.bool))
   var.partition(complete) or smth like Xilinx pragmas? or var.fpga_local_reg?

Lastly, you can compile the resulting graph through the Python API (sdfg.compile()) or through the CLI tool sdfgcc.

For example:
Dir="/home/test/program.Sdfg"
Raw_sdfg=sdfg.load(dir)?
Something like that.

Thanks for help :)

[tbennun]

Loading: sdfg = dace.SDFG.from_file('/path/to/program.sdfg')
Saving: sdfg.save('/path/to/program.sdfg')

As for your "not partitioned" case, perhaps it would be better to start another discussion, because more details are necessary :)

[bartokon]

Take a look at issue CUDA/FPGA :)

[bartokon]

Is there any example how to use generated source files? I have generated some DaCe function but what now?
I have some hidden folder .dacecache and my function in it, some makefiles some library dependencies. How can I use these files for example to use it in Xilinx project? How to set-up that project?

[tbennun]

You can also directly obtain the build folder of each graph by calling sdfg.build_folder.

The folder contains several subfolders that you can use for that purpose:

• build: Linked library, cmake build files, and makefiles. You can get the library from there.
• src: The various generated source codes. In your case, it also includes the Xilinx host and device code (or RTL) as two separate subfolders.
• map: Includes source mapping information (e.g., between Python, SDFG, and the generated code). This is used in our VSCode plugin to jump between source code and generated code.
• include: Contains an include file that can be used in existing C/C++ code, in order to link with the library in build
• sample: Contains a sample file that shows how the library could be called in the context of a program
• perf: This folder gets populated with (Chrome Tracing format) JSON files that have performance results if you enable instrumentation

So you could use the library in an existing program by including the include file and linking with the library in build, or take the files in src/ directly and use them in conjunction with an HLS program.

Note that if you use RTL, the DaCe build system generates and invokes several helper scripts and .ini files to link everything automatically.