SCF parallelization with xTB Hamiltonian

[steinmig]

I compiled DFTB+ from the current master with openMP support. Running an energy calculation with the xTB Hamiltonian gave me virtual identical wall times of ~2 minutes for both 1 and 32 cores. I could confirm that multiple threads (the CPU usage never reached 32, but definitely > 1) were started and also the DFTB+ output gives me a different CPU time.

Is the reason that

• tblite does not support multiple threads for the SCF?
• the openMP support for DFTB+ is not forwarded to tblite?
• I am doing something wrong in the compilation?

To Reproduce

• Current master with gcc-9.2, all cmake defaults + WITH_TBLITE=ON
• Input file is attached (same as in NaN / Infinity gradients and general question concerning gradients #871)

dftb_in.hsd.txt

[awvwgk]

The library should use be able to exploit OpenMP parallelization. We propagate the OpenMP setting (hopefully correct) here:

dftbplus/external/tblite/CMakeLists.txt

Lines 3 to 4 in 7455efc

	# Propagate OpenMP option correctly to subproject
	set(WITH_OpenMP ${WITH_OMP})

You might be able to improve the threading performance by setting OMP_SCHEDULE=dynamic. Since most OpenMP regions use a runtime schedule (both in DFTB+ and tblite), which will usually default to a static schedule and therefore lead to suboptimal scheduling for a symmetric neighbour list (should fix this at least in tblite). Performance with default schedule seems to be better than with dynamic in tblite (4 threads).

[steinmig]

But do you get a significant speed-up or is it more or less the same wall-time?

[chemistza]

I have noticed the same thing. xTB run thrugh DFTB+ doesn't seem to scale with increasing OMP threads, whereas running the same calculation through the native tblite interface scales well.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[steinmig]

Any updates on this?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[steinmig]

I think it is worth keeping this open

[aradi]

I agree. I'll try to have a look at it. Is this also valid with the recent version of DFTB+?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[steinmig]

I'll try to test it this month

[stefanoferr]

I am experiencing this behaviour in the DFTB+ 23.1 version too.
Do you have any updates on this topic?

[francescalb]

Has this been solved? I also experience problems with parallell performance when runnin xTB

[aradi]

OK. Could you just add the following lines to the input and check the system above with 1 and 2 threads? (The system is rather small).

Options {
  TimingVerbosity = -1
}

When I do it on my laptop with the Conda version of 24.1, I see, that the diagonalization, where the program spends the most time, takes almost identical wall-clock times in both cases, that means, the matrix is too small for the diagonalization library (openblas in my case) to profit from multiple threads. If that is also the case here, you could try to use MKL instead of OpenBLAS, but otherwise there is not much we can do.