[Bug report] Weird numpy.vstack behavior

[brockho]

Describe the bug
In a folder with the bbob-noisy data sets BFGS and DASA from 2009, I get the following when running the cocopp.bestalg.custom_generate function:

In [1]: import cocopp

In [2]: cocopp.bestalg.custom_generate(["BFGS", "DASA"])
  Data consistent according to consistency_check() in pproc.DataSet
  using: BFGS
  Data consistent according to consistency_check() in pproc.DataSet
  using: DASA
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-89fcb24091c7> in <module>
----> 1 cocopp.bestalg.custom_generate(["BFGS", "DASA"])

~\anaconda3\lib\site-packages\cocopp-2.6.3.81-py3.9.egg\cocopp\bestalg.py in custom_generate(args, algId, suite)
    535             print('Folder %s was created.' % output_dir)
    536
--> 537     result = generate(dictAlg, algId)
    538
    539     create_data_files(output_dir, result, suite)

~\anaconda3\lib\site-packages\cocopp-2.6.3.81-py3.9.egg\cocopp\bestalg.py in generate(dict_alg, algId)
    463     for f, i in pproc.dictAlgByFun(dict_alg).items():
    464         for d, j in pproc.dictAlgByDim(i).items():
--> 465             tmp = BestAlgSet(j, algId)
    466             res[(d, f)] = tmp
    467     return res

~\anaconda3\lib\site-packages\cocopp-2.6.3.81-py3.9.egg\cocopp\bestalg.py in __init__(self, dict_alg, algId)
    150
    151         # Align ERT
--> 152         erts = list(np.transpose(np.vstack([dict_alg[i].target, dict_alg[i].ert]))
    153                     for i in sortedAlgs)
    154         res = readalign.alignArrayData(readalign.HArrayMultiReader(erts))

~\anaconda3\lib\site-packages\cocopp-2.6.3.81-py3.9.egg\cocopp\bestalg.py in <genexpr>(.0)
    150
    151         # Align ERT
--> 152         erts = list(np.transpose(np.vstack([dict_alg[i].target, dict_alg[i].ert]))
    153                     for i in sortedAlgs)
    154         res = readalign.alignArrayData(readalign.HArrayMultiReader(erts))

~\anaconda3\lib\site-packages\numpy\core\shape_base.py in vstack(tup, dtype, casting)
    287     if not isinstance(arrs, list):
    288         arrs = [arrs]
--> 289     return _nx.concatenate(arrs, 0, dtype=dtype, casting=casting)
    290
    291

ValueError: all the input array dimensions except for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 11 and the array at index 1 has size 48

Together with @olafmersmann and @nikohansen, we checked last week that the input in line 152 of bestalg looks correct (i.e. both .target and .ert lists have the same length and that only inside the vstack, the lists are altered:

In [3]: %debug
> c:\users\dimo\anaconda3\lib\site-packages\numpy\core\shape_base.py(289)vstack()
    287     if not isinstance(arrs, list):
    288         arrs = [arrs]
--> 289     return _nx.concatenate(arrs, 0, dtype=dtype, casting=casting)
    290
    291

ipdb> arrs
[array([[2.51188643e+00, 1.58489319e+00, 2.51188643e-01, 1.58489319e-01,
        1.00000000e-01, 6.30957344e-02, 3.98107171e-02, 3.98107171e-03,
        1.00000000e-03, 6.30957344e-08, 9.99000999e-09]]), array([[1.00000000e+00, 2.73333333e+00, 8.93333333e+01, 1.62000000e+02,
        5.08400000e+02, 7.54000000e+02, 3.88706667e+03, 6.88560000e+03,
        1.05727333e+04, 1.48236000e+04, 1.53521333e+04, 1.64294667e+04,
        1.84377333e+04, 1.88570667e+04, 1.90506000e+04, 1.95013333e+04,
        1.95556000e+04, 2.26037333e+04, 2.36124667e+04, 2.36502667e+04,
        2.75878667e+04, 2.75899333e+04, 2.92073333e+04, 3.24266667e+04,
        3.31683333e+04, 3.40392667e+04, 3.55434667e+04, 4.11047333e+04,
        4.12185333e+04, 5.90658667e+04, 6.50416667e+04, 7.38728667e+04,
        1.05319267e+05, 1.30880800e+05, 1.43887867e+05, 1.43894400e+05,
        1.74510800e+05, 1.74554533e+05, 2.08605200e+05, 2.08621533e+05,
        2.19923400e+05, 2.19970533e+05, 2.48365267e+05, 2.48390200e+05,
        3.11187600e+05, 3.53213000e+05, 3.53219800e+05, 4.44252929e+05]])]
ipdb> up
> c:\users\dimo\anaconda3\lib\site-packages\cocopp-2.6.3.81-py3.9.egg\cocopp\bestalg.py(152)<genexpr>()
    150
    151         # Align ERT
--> 152         erts = list(np.transpose(np.vstack([dict_alg[i].target, dict_alg[i].ert]))
    153                     for i in sortedAlgs)
    154         res = readalign.alignArrayData(readalign.HArrayMultiReader(erts))

ipdb> [dict_alg[i].target, dict_alg[i].ert]
[array([2.51188643e+01, 1.58489319e+01, 1.00000000e+01, 6.30957344e+00,
       3.98107171e+00, 2.51188643e+00, 1.58489319e+00, 1.00000000e+00,
       6.30957344e-01, 3.98107171e-01, 2.51188643e-01, 1.58489319e-01,
       1.00000000e-01, 6.30957344e-02, 3.98107171e-02, 2.51188643e-02,
       1.58489319e-02, 1.00000000e-02, 6.30957344e-03, 3.98107171e-03,
       2.51188643e-03, 1.58489319e-03, 1.00000000e-03, 6.30957344e-04,
       3.98107171e-04, 2.51188643e-04, 1.58489319e-04, 1.00000000e-04,
       6.30957344e-05, 3.98107171e-05, 2.51188643e-05, 1.58489319e-05,
       1.00000000e-05, 6.30957344e-06, 3.98107171e-06, 2.51188643e-06,
       1.58489319e-06, 1.00000000e-06, 6.30957344e-07, 3.98107171e-07,
       2.51188643e-07, 1.58489319e-07, 1.00000000e-07, 6.30957344e-08,
       3.98107171e-08, 2.51188643e-08, 1.58489319e-08, 1.00000000e-08]), array([1.00000000e+00, 2.73333333e+00, 8.93333333e+01, 1.62000000e+02,
       5.08400000e+02, 7.54000000e+02, 3.88706667e+03, 6.88560000e+03,
       1.05727333e+04, 1.48236000e+04, 1.53521333e+04, 1.64294667e+04,
       1.84377333e+04, 1.88570667e+04, 1.90506000e+04, 1.95013333e+04,
       1.95556000e+04, 2.26037333e+04, 2.36124667e+04, 2.36502667e+04,
       2.75878667e+04, 2.75899333e+04, 2.92073333e+04, 3.24266667e+04,
       3.31683333e+04, 3.40392667e+04, 3.55434667e+04, 4.11047333e+04,
       4.12185333e+04, 5.90658667e+04, 6.50416667e+04, 7.38728667e+04,
       1.05319267e+05, 1.30880800e+05, 1.43887867e+05, 1.43894400e+05,
       1.74510800e+05, 1.74554533e+05, 2.08605200e+05, 2.08621533e+05,
       2.19923400e+05, 2.19970533e+05, 2.48365267e+05, 2.48390200e+05,
       3.11187600e+05, 3.53213000e+05, 3.53219800e+05, 4.44252929e+05])]

Even weirder is the behaviour if I simply print the two lists before calling the vstack:

print([[dict_alg[i].target, dict_alg[i].ert]
                    for i in sortedAlgs])

In this case, everything goes through without error!

Your System (please complete the following information when it seems relevant):

• Operating system: Windows 10
• COCO version: cocopp.__version__ is '2.6.3.81' (current development branch)

[brockho]

Add on: the same problem originally occurred also when the 2009 bbob data set for BFGS is used in the above call.