Replies: 2 comments 4 replies
-
|
So here is what I did: Apply patch to sphinx: diff --git a/sphinx/cmd/build.py b/sphinx/cmd/build.py
index 8de24a364..31698126b 100644
--- a/sphinx/cmd/build.py
+++ b/sphinx/cmd/build.py
@@ -292,11 +292,22 @@ def main(argv: List[str] = sys.argv[1:]) -> int:
sphinx.locale.setlocale(locale.LC_ALL, '')
sphinx.locale.init_console(os.path.join(package_dir, 'locale'), 'sphinx')
+ import cProfile, pstats, io
+ from pstats import SortKey
+
+ pr = cProfile.Profile()
+
+ pr.enable()
if argv[:1] == ['-M']:
- return make_main(argv)
+ ret = make_main(argv)
else:
- return build_main(argv)
-
+ ret = build_main(argv)
+ pr.disable()
+ s = io.StringIO()
+ sortby = SortKey.CUMULATIVE
+ ps = pstats.Stats(pr, stream=s).sort_stats(sortby)
+ ps.print_stats()
+ print(s.getvalue())
if __name__ == '__main__':
sys.exit(main(sys.argv[1:]))and run EDIT: Here are the top functions: |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for raising this. People have talked about this in the past though I am out of touch with the current status. Some caching was add a long time ago and I think things have been quiet since then. That said, 18 minutes is clearly crazy! What is your interpretation of the output? It seems like a lot of the time is being spent in the sphinx CPP domain. That doesn't mean that we can't do things from here to improve it but maybe it most obviously something taking ages in the Breathe code itself. What is your read? Seems to be related to IDs. I don't know if there is room for Breathe to either generate fewer or somehow "better" IDs. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the patch to profile Sphinx, and the profile it self! My primary concern for the C++ (and C) domains has been correctness and functionality, and large-scale profiling never really happened. |
Beta Was this translation helpful? Give feedback.
-
Well I don't know the internals enough to say if breathe could do something more clever or if sphinx needs some tweaks.
The quick approach for profiling would be:
I've commited the doxygen created xml files and already ran breathe. The full documentation building from scratch is:
Now to your fixes. They already go in the right direction! Before: Now: and that is also with profiling turned on. Most of the time sphinx is not using all available cores but just one. And the profile traces are now: |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the instructions, the specific branch did the trick :-). |
Beta Was this translation helpful? Give feedback.
-
|
Did you get a change to look into that some more? Or is something already in newer sphinx versions available? |
Beta Was this translation helpful? Give feedback.
-
I'm using doxygen/breathe/sphinx at https://github.com/AllenInstitute/MIES to generate documentation.
First of all, it's working flawlessy with the latest versions and is also really nice to look at. Thanks for your effort put into breathe.
One problem which keeps persisting is that the build times are absurdely long. On my developer machine (Win 10, 6 cores, 64GB RAM) it takes around 18 minutes. When I remove the breathe plugin from conf.py it takes around 1.5 minutes. The timing does not change if I pass
-j autoto sphinx-build or not.Now we are doing a lot of stuff, even our programming lanuage is not understood by doxygen, but translated to something doxygen understands. But still 18 minutes is way too long.
Now my usual approach to find out more would be to do some profiling. Is there some documentation how to profile breathe/sphinx?
Beta Was this translation helpful? Give feedback.
All reactions