Warm start: results.json and configs.json (from live json_result_logger) do not contain results from previous runs

[rosea-tf]

Hi, here's an abbreviated example of the problem described in the topic:

for run in range(3):
     loaded_from_json = hpres.logged_results_to_HBS_result(...) if run > 0 else None
     logger = hpres.json_result_logger(...)
     bohb = BOHB(..., result_logger=logger, previous_result=loaded_from_json)
     result = bohb.run(...)

• Run 0: result contains results from run 0, *.json files contain results from run 0
• Run 1: result contains results from run 0 and 1, *.json files contain results from run 1
• Run 2: result contains results from run 1 and 2, *.json files contain results from run 2

So by run 2, the results from run 0 have been lost.

If instead we pass the Result object directly (or via a pickle) instead of using the live-logged JSONs, then the results from run 0 can be retained:

for run in range(3):
     previous_result = result if run > 0 else None
     logger = hpres.json_result_logger(...)
     bohb = BOHB(..., result_logger=logger, previous_result=previous_result)
     result = bohb.run(...)

• Run 0: result contains results from run 0, *.json files contain results from run 0
• Run 1: result contains results from run 0 and 1, *.json files contain results from run 1
• Run 2: result contains results from run 0 and 1 and 2, *.json files contain results from run 2

However, this won't work if the any of the runs crash or need to be aborted part way through, which makes the process kind of fragile.

Expected behaviour: the previous runs are included (with shifted timestamps and iteration stamps) in the JSONs, the same as in the Result object.

[gui-miotto]

I noticed this problem too. It would be great to have #94 merged.

Dear @sfalkner @mfeurer : What is the current maintenance status of HpBandSter? The code hasn't received much attention since the end of 2018. Has HpBandSter / BOHB been discontinued?

[mfeurer]

Sorry, I cannot help here as I don't work on HpBandSter.

CC @frank-hutter

[gui-miotto]

Thanks for the reply, @mfeurer !