-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[MLIR] Enable multi threaded compilation #655
base: main
Are you sure you want to change the base?
Conversation
7493f15
to
5143cad
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #655 +/- ##
==========================================
- Coverage 99.95% 98.10% -1.85%
==========================================
Files 20 69 +49
Lines 4444 9556 +5112
Branches 0 747 +747
==========================================
+ Hits 4442 9375 +4933
- Misses 2 147 +145
- Partials 0 34 +34 ☔ View full report in Codecov by Sentry. |
Hello. You may have forgotten to update the changelog!
|
Did you find any? |
Co-authored-by: erick-xanadu <110487834+erick-xanadu@users.noreply.github.com>
Nope, everything seems to be fine so far. |
Do you want me to do this or are you going to check? |
@erick-xanadu I believe it is off by default currently |
@@ -303,7 +302,7 @@ def circuit(): | |||
assert "While processing 'TestPass' pass of the 'PipelineB' pipeline" in e.value.args[0] | |||
assert "PipelineA" not in e.value.args[0] | |||
assert "Trace" not in e.value.args[0] | |||
assert isfile(os.path.join(str(compiled.workspace), "2_PipelineB_FAILED.mlir")) | |||
assert isfile(os.path.join(str(compiled.workspace), "3_1_PipelineB_FAILED.mlir")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are there two numbers now? E.g., 3_1
vs 2
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- The first number iterates over hand-coded stages that we have. We increase it manually, e.g. here. Here we rely on the fact that compiler driver itself is not multi-threaded.
- The second number is the index of the pipeline in the list of pipelines provided by the user. These dumps might happen from different threads, but the file names will be different.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand it fully, but is there a way to just enumerate them with a single number instead of this tuple? I don't think there is a difference between the pipelines provided by us and the user.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is possible of course, but it would complicate the design with the arithmetic which I think is not necessary. We would need to add a number of pipelines to the counter at some point. But in reality we only want the filenames to be sorted.
// Install signal handler to catch user interrupts (e.g. CTRL-C). | ||
signal(SIGINT, | ||
[](int code) { throw std::runtime_error("KeyboardInterrupt (SIGINT)"); }); | ||
|
||
std::unique_ptr<CompilerOutput> output(new CompilerOutput()); | ||
std::unique_ptr<CompilerOutput> output(new CompilerOutput(initDumpCounter(workspace))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, here is where you create it. How about instead of having the initDumpCounter
function, just have run_compiler_driver
take an int pararameter which defaults to 1. And have the canonicalizer set it to 0.
I would like to avoid having the initDumpCounter
function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@erick-xanadu in this case we will would need to write a Python analog of initDumpCounter
counting the canonicalizer's artifacts and setting the integer to the right value. I'm ok with this solution, but do you think it is worth the time to implement? Or did you mean something else?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The canonicalizer will always have 1 single output. I don't think we need to count anything.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not? What if someone change it?
Looking into MLIR's documentation:
and later below
I believe we have not been following this. Specifically in gradients and other passes, we add functions to the module. I think we should not make this the default yet. But if we are willing to increase testing time, we could add small set of compilation tests every so often to see if we catch any errors. I'll be running this PR in a bit looking for potential issues (or a reason why they haven't been seen). Thanks! |
It looks like one of the reasons why we are not taking advantage of parallelism is because all our passes act on a module scope:
I think in order to take advantage, we would need to change these |
I am currently running into a weird error 😅 All test pass except one for printing the IR. When the IR is printed, it gets printed nine times. Even if it was not printed nine times, the test would fail. Can you reproduce it locally? I've re-triggered a run of the code.
|
This reverts commit b918d13.
Oh, it turns out I broke the PR by my 'simplification' commit. I have reverted it and added comments. The problem happened because the dump handler was triggered by all passes instead of passes which end a pipeline. |
Context: MLIR supports multi-threaded compilation which was disabled with a Todo notice.
Description of the Change: By this PR we add an
qjit
option enabling the MLIR multi-threaded compilation. We also make some precautions and infrastructure updates:Note: measurement results (internal document)
Benefits:
Possible Drawbacks: Multi-threading issues might be uncovered(?)
Related GitHub Issues:
[sc-59511]