Request for training logs and detailed settings

[BotMan-Sz]

Dear authors,
Recently I have been studying your paper and code, however, even with your proceesed dataset, I can not repeat your reported results in your paper. Take the HAR dataset as example, there comes the following issues.

1. When I choose the random_init training, the results(77.9) is much better than the scores you reported in Table 2(57.89±5.13). I don't know what random seed you used in your paper, but for 5 seed i tested, they all higher than your report.
   
2. When I use only 5% of trainining data for supervised trainining, the accuracy and macro F1(>0.89) is much higher than the reported scores Fig.2(MF1<0.55). Below are the results for training with 5% training dataset (with batch size of 8).
   
3. All thereported results are hard to reproduce.

Hope you could share me your experiment setting and training logs to remove doubts.

[BotMan-Sz]

logs_10_12_2024_07_00_53.log
run1_random_init_seed_0_classification_report.xlsx
logs_10_12_2024_07_07_03(supervised with 1% HAR dataset).log
logs_10_12_2024_07_21_37(supervised with 5% HAR dataset).log
run1_supervised_seed_0_classification_report(supervised with 1% HAR dataset).xlsx
run1_supervised_seed_0_classification_report(supervised with 5% HAR dataset).xlsx

[BotMan-Sz]

Add this in dataloader/dataloder.py/Load_Dataset() function to obtain certain ratio of tainining samples.

[emadeldeen24]

Hi,
thanks for experimenting with our code. Unfortunately, I don't have the logs anymore.
But I recall using the same parameters as yours, maybe the dropout was 0.5, but everything is the same. I'm not sure why you can not reproduce the results.

[BotMan-Sz]

Hi,
Thanks for your response. As the authors, reproducing your own code should not be difficult, I hope you take a little time to recall your parameter configuration, etc., and run the code again, which is crucial for the reliability of your work.

[BotMan-Sz]

If needed, I can update my projects with logs to github, and i didn't make any changes to the core codes. It seems that even the Random Initialization and supervised results are not reliable, not to say other baselines.