Fail to sync repo with Git LFS: "directory is not empty"

[brandonaut]

Running peru sync fails when syncing a repo that uses Git LFS. It seems that the clone into the .peru/tmp directory succeeds, but when it tries to delete the tmp directory, it fails because some of the files are still in the process of being deleted. A few seconds after I get this error message, I can see that all the files in the tmp directory get deleted and the directory is, in fact, empty. Maybe this is a threading issue?

Approximate peru.yaml:

imports:
  normalrepo: normalrepo
  gitlfsrepo: gitlfsrepo

git module normalrepo:
  url: https://mygithubenterprise/myorg/normalrepo
  rev: ***

git module gitlfsrepo:
  url: https://mygithubenterprise/myorg/gitlfsrepo
  rev: ***

Stacktrace:

> peru sync -v
=== started gitlfsrepo===
=== finished gitlfsrepo===
Traceback (most recent call last):
  File "c:\python38\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python38\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\me\.local\bin\peru.exe\__main__.py", line 7, in <module>
  File "c:\users\me\.local\pipx\venvs\peru\lib\site-packages\peru\main.py", line 370, in main
    run_task(command_fn(params))
  File "c:\users\me\.local\pipx\venvs\peru\lib\site-packages\peru\async_helpers.py", line 29, in run_task
    return asyncio.get_event_loop().run_until_complete(coro)
  File "c:\python38\lib\asyncio\base_events.py", line 616, in run_until_complete
    return future.result()
  File "c:\users\me\.local\pipx\venvs\peru\lib\site-packages\peru\main.py", line 91, in do_sync
    await imports.checkout(params.runtime, params.scope, params.imports,
  File "c:\users\me\.local\pipx\venvs\peru\lib\site-packages\peru\imports.py", line 11, in checkout
    imports_tree = await get_imports_tree(runtime, scope, imports)
  File "c:\users\me\.local\pipx\venvs\peru\lib\site-packages\peru\imports.py", line 24, in get_imports_tree
    target_trees = await get_trees(runtime, scope, imports.keys())
  File "c:\users\me\.local\pipx\venvs\peru\lib\site-packages\peru\imports.py", line 32, in get_trees
    trees = await gather_coalescing_exceptions(
  File "c:\users\me\.local\pipx\venvs\peru\lib\site-packages\peru\async_helpers.py", line 92, in gather_coalescing_exceptions
    raise GatheredExceptions(exceptions, reprs)
peru.async_helpers.GatheredExceptions: Traceback (most recent call last):
  File "c:\users\me\.local\pipx\venvs\peru\lib\site-packages\peru\async_helpers.py", line 71, in catching_wrapper
    return (await coro)
  File "c:\users\me\.local\pipx\venvs\peru\lib\site-packages\peru\imports.py", line 41, in get_tree
    tree = await module.get_tree(runtime)
  File "c:\users\me\.local\pipx\venvs\peru\lib\site-packages\peru\module.py", line 80, in get_tree
    base_tree = await self._get_base_tree(runtime)
  File "c:\users\me\.local\pipx\venvs\peru\lib\site-packages\peru\module.py", line 67, in _get_base_tree
    tree = await runtime.cache.import_tree(tmp_dir)
  File "c:\python38\lib\tempfile.py", line 827, in __exit__
    self.cleanup()
  File "c:\python38\lib\tempfile.py", line 831, in cleanup
    self._rmtree(self.name)
  File "c:\python38\lib\tempfile.py", line 813, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "c:\python38\lib\shutil.py", line 737, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\python38\lib\shutil.py", line 619, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "c:\python38\lib\shutil.py", line 617, in _rmtree_unsafe
    os.rmdir(path)
OSError: [WinError 145] The directory is not empty: 'C:\\myrepo\\.peru\\tmp\\tmpo7fdz5_h'

If I only sync modules that don't use Git LFS, it succeeds.

Environment

• Windows 10 (1809)
• Python 3.8.6
• Peru 1.2.0 installed with pipx

[oconnor663]

Thanks for putting so much context in this report. I wonder if this is going to turn out to be a Windows-specific problem. Specifically, is it that files are getting created in the background, causing directory deletion to fail? (Should repro on both Windows and Unix in this case.) Or is it that file deletion is failing, because files are somehow open in the background. (Might repro only on Windows in this case.)

Would you have time to create a repro example using public GitHub repos? I haven't used LFS before myself, and a public repro would make it much easier to fix the bug.

[brandonaut]

Oops, somehow I got really mixed up. Neither repo is using Git LFS, so I'll have to figure out what other differences they have. One of the repos syncs fine, and the other gets the "directory is not empty" problem. I haven't been able to reproduce it in a public repo so far, but I'll keep you posted.

BTW, thanks for the quick response!