hub uses cache to download RGI

[fmaussion]

When building an image with https://github.com/OGGM/r2d, the default paths are used by each user:

oggm.cfg.PATHS

PathOrderedDict([('dl_cache_dir', '/home/data/download'),
                 ('tmp_dir', '/home/jovyan/.oggm_cache/tmp'),
                 ('cru_dir', '/home/jovyan/.oggm_cache/cru'),
                 ('rgi_dir', '/home/jovyan/.oggm_cache/rgi'),
                 ('test_dir', '/home/jovyan/.oggm_cache/OGGM_tests'),
                 ('working_dir', 'WaterResources-Proj-Astore'),
                 ('dem_file', ''),
                 ('climate_file', '')])

/home/jovyan/.oggm_cache/rgi becomes > 1GB large if people download RGI version 60 and 62 and the intersects.

So this takes quite a bit of their 10Gb just like that. I think rgi_dir could be shared accross users.

[TimoRoth]

Is it not on /home/data/download, so all it'd take would be to adjust the path?

…

On 22.03.2023 16:54, Fabien Maussion wrote: When building an image with https://github.com/OGGM/r2d <https://github.com/OGGM/r2d>, the default paths are used by each user: |oggm.cfg.PATHS PathOrderedDict([('dl_cache_dir', '/home/data/download'), ('tmp_dir', '/home/jovyan/.oggm_cache/tmp'), ('cru_dir', '/home/jovyan/.oggm_cache/cru'), ('rgi_dir', '/home/jovyan/.oggm_cache/rgi'), ('test_dir', '/home/jovyan/.oggm_cache/OGGM_tests'), ('working_dir', 'WaterResources-Proj-Astore'), ('dem_file', ''), ('climate_file', '')]) | |/home/jovyan/.oggm_cache/rgi| becomes > 1GB large if people download RGI version 60 and 62 and the intersects. So this takes quite a bit of their 10Gb just like that. I think |rgi_dir| could be shared accross users. — Reply to this email directly, view it on GitHub <#1551>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEA5CMW4OIRGKRCRPO4Q4TTW5MODFANCNFSM6AAAAAAWEAQQYQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[fmaussion]

It's a bit more complicated than that - OGGM downloads and extracts the RGI files in a folder (quite old design decisions). The compressed, global files are in /home/data/download yes.

On hub and classroom there is a shared folder, I think it should rather be prepared there