Benchmarking

[bilalshaikh42]

Total Number of runs: 33
Total time: 0:04:40.849398
Total Number of pending runs: 0
Total Number of passed runs: 33
Total Number of timed out runs: 0
Total Number of failed runs: 0

[bilalshaikh42]

Total Number of runs: 66
Total time: 0:04:28.187527
Total Number of pending runs: 0
Total Number of passed runs: 65
Total Number of timed out runs: 0
Total Number of failed runs: 1

Failure is boolnet due to #4127

[jonrkarr]

I'm inclined to think the BoolNet and NEURON issues relate to /tmp not being available or large enough when run through singularity run. I think this can be addressed by explicitly binding a local directory to /tmp in the container (add --bind option to singularity run ...).

[bilalshaikh42]

seems to happen intermittently, which makes me think one of the nodes is just out of space on /tmp. It passes when the job is assigned to a different node and fails when it is on a node out of space

[bilalshaikh42]

Total Number of runs: 99
Total time: 0:03:24.333168
Total Number of pending runs: 0
Total Number of passed runs: 99
Total Number of timed out runs: 0
Total Number of failed runs: 0

[jonrkarr]

Yeah. That's kind of annoying that there are no guarantees on /tmp. I guess we have to live with that.

We might not have seen this earlier because we were using a dedicated partition. Now, we have access to more machines. But, we're also sharing machines.

[bilalshaikh42]

Total Number of runs: 132
Total time: 0:03:11.464150
Total Number of pending runs: 0
Total Number of passed runs: 132
Total Number of timed out runs: 0
Total Number of failed runs: 0

[bilalshaikh42]

On the first attempt of 132 runs- All 127 that were submitted seemed to have succeeded fine. Ironically, the submit script failed to connect to Github on run 127

[bilalshaikh42]

Total Number of runs: 165
Total time: 0:03:26.514650
Total Number of pending runs: 0
Total Number of passed runs: 165
Total Number of timed out runs: 0
Total Number of failed runs: 0

[jonrkarr]

Vastly improved! Other than the minor issues you recently created, I think we're ready to move on.