[CLI]: wandb/artifacts/staging not getting cleaned up

[tfriedel]

Describe the bug

I use the artifact model registry for all my training runs.
I regularly run into the issue that my disk is getting filled and it needs to be cleaned up.
For this purpose I run "wandb artifact cache cleanup 20GB" which frees up some space.
However there's still currently 70GB of data in
~/.local/share/wandb/artifacts/staging

I would have expected this to get cleaned, because it's more than 20GB.

In my mental model this folder is possibly different from the "cache" as all the files have names like tmpXXXXXX. Maybe those files are files that were supposed to get uploaded but for some reason didn't, e.g. because wandb service or the training run were stopped?

Additional Files

No response

Environment

WandB version: 0.16.5

OS: ubuntu 24.04

Python version: 3.11.0

Versions of relevant libraries:

Additional Context

No response

[umakrishnaswamy]

hey @tfriedel - the .local/share/wandb/artifacts/staging directory is different from the .cache directory. wandb artifact cache cleanup only cleans the .cache directory. would it be possible to manually clear the folders from the staging directory? you can do this with something like find /specified/path_to_directory -type d -name "tmp" -exec rm -r {} +

[tfriedel]

@umakrishnaswamy
Yes that's what I'm doing now.
It may be good if wandb somehow cleans it up on it's own, e.g. with this artifact cache cleanup command or otherwise.
I'm not sure if it's safe to delete the files, especially if there's some runs still running.

So I modified the command to only delete files older than 24 hours:

find ~/.local/share/wandb/artifacts/staging -type f -mmin +1440 -exec rm -f {} \;

[umakrishnaswamy]

thanks for the update @tfriedel ! I can make a feature request for adding that functionality, but glad the above command worked for your case