SameFileError when processing with StartMaja

[BartoszJaroszkowski]

Given a workplan

               DATE |       TILE |     MODE |                                                             L1-PRODUCT | ADDITIONAL INFO
2021-10-10 10:09:41 |      32UQU | BACKWARD |      S2A_MSIL1C_20211010T100941_N0301_R022_T32UQU_20211010T122329.SAFE | Backward of 8 products
2021-10-10 10:09:41 |      32UQU |  NOMINAL |      S2A_MSIL1C_20211010T100941_N0301_R022_T32UQU_20211010T110450.SAFE | L2 from previous
2021-10-15 10:10:29 |      32UQU |  NOMINAL |      S2B_MSIL1C_20211015T101029_N0301_R022_T32UQU_20211015T121549.SAFE | L2 from previous
2021-10-23 10:21:01 |      32UQU |  NOMINAL |      S2A_MSIL1C_20211023T102101_N0301_R065_T32UQU_20211023T110926.SAFE | L2 from previous
2021-10-30 10:11:41 |      32UQU |  NOMINAL |      S2A_MSIL1C_20211030T101141_N0301_R022_T32UQU_20211030T110921.SAFE | L2 from previous
2021-11-04 10:11:09 |      32UQU |  NOMINAL |      S2B_MSIL1C_20211104T101109_N0301_R022_T32UQU_20211104T121548.SAFE | L2 from previous
2021-11-14 10:11:59 |      32UQU |  NOMINAL |      S2B_MSIL1C_20211114T101159_N0301_R022_T32UQU_20211114T105348.SAFE | L2 from previous
2021-11-19 10:13:21 |      32UQU |  NOMINAL |      S2A_MSIL1C_20211119T101321_N0301_R022_T32UQU_20211119T110331.SAFE | L2 from previous

I got the following error

Traceback (most recent call last):
  File "/maja/lib/python/orchestrator/launcher/maja.py", line 100, in <module>
    main()
  File "/maja/lib/python/orchestrator/launcher/maja.py", line 81, in main
    lprocessor.scientific_processing()
  File "/maja/lib/python/orchestrator/processor/l2_nominal_processor.py", line 216, in scientific_processing
    l2_image_file_writer, checking_conditional_clouds)
  File "/maja/lib/python/orchestrator/processor/l2_processor.py", line 828, in process_one_product
    self._cams_status, l_IgnoreCurrentLTC)
  File "/maja/lib/python/orchestrator/processor/l2_processor_header_writer_setup.py", line 157, in setup_l2_header_writer
    l_L2HeaderFileWriter.write()
  File "/maja/lib/python/orchestrator/plugins/common/muscate/maja_muscate_l2_header_writer.py", line 75, in write
    self.pre_processing()
  File "/maja/lib/python/orchestrator/plugins/common/base/maja_l2_header_writer_base.py", line 124, in pre_processing
    self.outputl2privateimagefilenamesprovider.get_ltc_image_dir_filename())
  File "/maja/lib/python/orchestrator/common/file_utils.py", line 129, in copy_tree
    copy_file(src_filename, dst_filename)
  File "/maja/lib/python/orchestrator/common/file_utils.py", line 112, in copy_file
    shutil.copyfile(source, dest)
  File "/maja/lib/python3.7/shutil.py", line 104, in copyfile
    raise SameFileError("{!r} and {!r} are the same file".format(src, dst))
shutil.SameFileError: '/mnt/vol/tmp/Start_maja_e73b1f2b4319aeef75c481cf23529f75/SENTINEL2A_20211010-101729-643_L2A_T32UQU_C_V1-0/DATA/SENTINEL2A_20211010-101729-643_L2A_T32UQU_C_V1
-0_PVD_ALL/LTC/20211104.mha' and '/mnt/vol/input/L2A-Products/32UQU/SENTINEL2A_20211010-101729-643_L2A_T32UQU_C_V1-0/DATA/SENTINEL2A_20211010-101729-643_L2A_T32UQU_C_V1-0_PVD_ALL/L
TC/20211104.mha' are the same file

It seems to be happening rather randomly - sometimes reprocessing with the same exact inputs works as intended, sometimes it fails again. Looking at the traceback it seems pretty easy to mitigate (either with try/except block or by checking if the file exists beforehand) - the question is whether it is not a symptom of some underlying issue.

[jerome-colin]

Hi Bartosz,
Thanks for reporting this issue. I duplicated it on the active GitLab repository used by Maja's dev team.

This duplicated repo on both Github and GitLab is confusing, we'll simplify this soon. In the meantime, you'll see the follow-up of your bug report here:

https://gitlab.orfeo-toolbox.org/maja/maja/-/issues/268

Jerome