Malet release v0.1.13

Changes

• Cannot pass int split as string total_splits and curr_splits in Experiment.

Features

• Add option to filter out certain values alongside with filtering in at malet-plot (add '!' at the end of the field as '-filters field! v1 v2 ...').

Bug Fixes

• Add error message for overlapping fields in static and grid configs in ConfigIter. (some functionality in ExperimentLog breaks without this assertion.)
• Fix error when total_splits and curr_splits are passed in as int in Experiment.

Malet release v0.1.12

Bug Fixes

• Fix experiment skipping failed cases and executing already running cases.

Malet release v0.1.11

🚨 Fatal bug (removed from pypi)

• Experiment skipping failed cases and executing already running cases, which should be (and has been) the other way around.

Changes

• Change method name get_log_checkpoint into get_metric_info in Experiment, which now returns empty dictionaries if the log of matching config is not in the experiment log.

Features

• Add method to set all status R' in the tsv file’ into 'F' in Experiment.
• Add option to set status in Experiment.update_log.
• Add RunInfo class for various time tracking for Experiment.
• Add x, y scale option in malet-plot.
• Annotate standard deviation as mean±std in malet-plot.

Bug Fixes

• Fix run duration tracking not updated when calling update_log.
• Fix NaN checking in Experiment.get_metric_info (formerly Experiment.get_log_checkpoint).

Malet release v0.1.10

Features

• Add error message for non-matching static_config between ConfigIter and ExperimentLog in Experiment.

Bug Fixes

• Fix duration aggregation in Experiment.run
• Fix NaN checking in Experiement.get_log_checkpoint

Malet release v0.1.9

Changes

• Change arguments in Experiment.update_log from (metric_dict, config) to (config, **metric_dict).
• Change arguments and return value in Experiment.get_log_checkpoint from (config, empty_metric) → metric_dict to (config) → (metric_dict, info_dict).

Features

• Add train duration information and git info in experiment.Experiment.

Bug Fixes

• Fix Experiment not recording datetime and status.
• Fix Experiment not to delete metrics when checkpointing.
• Reflect ExperimentLog.add_result argument change to Experiment.update_log.

Malet release v0.1.8

Changes

• Remove info_field attribute in ExperimentLog due to redundancy. Please migrate all info_field to metric_field.
• Change method name get_metric_and_info → get_metric in ExperimentLog
• Remove kwarg: infos and change type of kwarg: metric from dict to unpacked keyword arguments.

Bug Fixes

• Fix str2value to process inf and return string if literal_eval throws an exception.
• Fix ExperimentLog.add_result for cases when there are two grid fields. (Dataframe.loc and multiindex edge case.)
• Fix isin to only check matching config in static configs.

Malet release v0.1.7

Changes

• Change method name explode_and_melt_metric → melt_and_explode_metric and its kwarg epoch → step in ExperimentLog.
• Compute function (arg: fn) is no longer mapped onto list fields when using ExperimentLog.add_conputed_metric.
• Change module structure of malet.plot_utils.

Features

• Can now parse metric of any nested-structure of standard python types (strings, bytes, numbers, tuples, lists, dicts, sets, booleans, None and Ellipsis) from tsv file in ExperimentLog using ast.literal_eval.
• Add mode (curve_best) for plotting star marker for best performing among x.
• Add annotations to heatmap in malet-plot.
• Filter epoch by range in malet-plot.
• malet-plot can process list fields with various lengths.
• Automatic markevery setting depending on number of x-values in curve modes in malet-plot.
• Add arguments for title, x-label, y-label, font size in malet-plot.
• Filter log.df prior to melt_and_expand_metric for faster processing in plot.draw_metric.
• Add kwarg save to ExperimentLog.merge_folder.
• Better error messages
  • ExperimentLog: error message for overlapping field names when instantiating
  • plot_utils.data_processor.select_df: error messages for non-existent field, field_value, and empty dataframe before/after filter.

Bug Fixes

• Plotting when x-field is 'step' (previously 'epoch')
  • Fix from plotting only last step to all steps (wrong argument in explode_and_melt_metric call).
  • Fix error when 'best_ref_x_fields' is not specified.
• Annotate only above the marks in curve mode in malet-plot.
• Apply str2value to indices (grid fields) when loading tsv in ExperimentLog.from_tsv.
• Fix ExperimentLog.add_derived_index to handle tuples.

Malet release v0.1.5

Changes

• Experiment.resplit_logs now automatically finds log and config path from experiment folder.
• malet-plot accepts space-seperated lists in multi_line_fields, x_fields, and best_ref_x_fields.
• Change in ExperimentLog.merge from using pd.merge to `pd.concat’.

Features

• Better command line messages for malet-plot with rich library.
• Add bar plot and heatmap in malet-plot
• Cast list configs into tuple when adding configs to dataframe in ExperimentLog for hashablilty.
• Add option parse_str in ExperimentLog.parse_tsv to parse string in the pd.DataFrame entry into lists and tuples or keep it as string.
• Add method (ExperimentLog.add_computed₎_metric, ExperimentLog.add_derived_index) to compute other metric and indices derived from existing metric and indices.
• Add remove_metric and remove_index in `ExperimentLog’.
• Remove str, NaN and change metric_value column to float in ExperimentLog.explode_and_melt_metric. This allows more flexable config gridding.
• Add resplitting for ExperimentLogs.
• Add option to specify annotate field, figure size, and plot style in flags for malet-plot .
• Add save_path kwarg in merge methods in ExperimentLog.

Bug Fixes

• Fix ExperimentLog.merge_tsv by not parsing strings into list when reading tsv.
• Fix str2value to process scientific notations (e.g. 1.200004e+12) and nan
• Fix ExperimentLog.explode_and_melt_metric to stop raising exceptions when there are no list fields, and handle list fields with NaN entries
• Fix malet-plot throwing errors from having multiple metric fields when using best_ref_…_field.

Malet release v0.1.4

Features

• Automatic fail status logging using try-catch
• Merge add_configs_only method with add_result in ExperimentLog
• Improved shell messages when plotting (plot info, saved path)
• Better plot save names (add various plot configs to prevent overwrites)
• Add plot option to select optimal value direction between min or max

Bug Fixes

• Proper sorting when plotting over list-valued field
• Fix str2value integer and float error
• Fix ExperimentLog.from_tsv method call in Experiment

Malet release v0.1.3

Features

• Shell message during malet-plot notifying how the fields in the dataframe are handled (key, non-key: specified / averaged / optimized) to guide users for field handling
• Add error message when user-specified value does not exist in the field when using plot_utils.metric_drawer:select_df

Bug Fixes

• Add str2value when handling filter in plot:draw_metric
• Fix 'cannot concat list with dict′ error in plot_utils.utils:merge_dict