Exclude __pycache__ directories from backups using CACHEDIR.TAG

[jstasiak]

BPO	41081
Nosy	@ericvsmith, @jstasiak
PRs	gh-85253: Exclude __pycache__ directories from backups using CACHEDIR.TAG #21060

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-06-22.19:39:42.956>
labels = ['type-feature', 'library', '3.10']
title = 'Exclude __pycache__ directories from backups using CACHEDIR.TAG'
updated_at = <Date 2020-06-22.20:54:25.799>
user = 'https://github.com/jstasiak'

bugs.python.org fields:

activity = <Date 2020-06-22.20:54:25.799>
actor = 'eric.smith'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-06-22.19:39:42.956>
creator = 'jstasiak'
dependencies = []
files = []
hgrepos = []
issue_num = 41081
keywords = ['patch']
message_count = 2.0
messages = ['372109', '372113']
nosy_count = 2.0
nosy_names = ['eric.smith', 'jstasiak']
pr_nums = ['21060']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue41081'
versions = ['Python 3.10']

[jstasiak]

It'd be nice of __pycache__ directories didn't pollute backups. Granted, one can add __pycache__ directory to their backup-tool-of-choice exclusion list, but those lists are ever growing and maybe it'd be good to help the tools and the users.

There's a Cache Directory Tagging Specification[1] which some backup tools like Borg, restic, GNU Tar and attic use out of the box (well, with a switch) and other tools (like rsync, Bacula, rdiff-backup and I imagine others) can be made to use it with a generic exclude-directories-with-this-file-present option (partially, just the existence of the tag file is used, not its content).

I wasn't sure what to select in Components, so I went with Library.
[1] https://bford.info/cachedir/

[ericvsmith]

To spoil it for other readers: the linked page says to create a file named CACHEDIR.TAG with a specific first line.

[brettcannon]

I'm not sure what I think about this. I get the desire, but the file isn't a standard either.

[merwok]

What are the downsides of doing this?

[jstasiak]

• Extra code to run at import-time, including IO -> in principle slower imports
• Extra code to maintain for the benefit of a set of users that may be considered too small to be worth it (people running backup software that supports this marker)
• Extra noise in the filesystem

It's not a standard in an RFC kind of sense but it's somewhat recognizable in the backup software world – I know that Borg, rustic and restic support it.

[RazerM]

Some other tools which follow this already are mypy, pytest, pipx, cargo (Rust).

I think it's a good idea, things like this are a real pain point when using backup software even if the impact of __pycache__ isn't particularly egregious.

[AA-Turner]

It doesn't seem to hurt, beyond the obvious "code to maintain" burden, and seems to have adoption as a de facto standard, so I'd be +1. @jstasiak would you be able to benchmark if the difference of writing an additional file is noticeable for import time? If it has a measurable impact beyond the noise, we may consider an opt-out config setting for those who don't care about writing the file and want imports as fast as possible.

A

[jstasiak]

Yes, I'll try to run some tests.

[brettcannon]

So I brought this up on the core dev Discord server and either got objections or lukewarm reaction. The biggest sticking point is this is no way a standard and us doing this would make it a de-facto one.

As such, I'm afraid I need to close this until some standards body picks it up.