Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Full user data export #215

Open
opatut opened this issue Mar 8, 2022 · 1 comment
Open

Full user data export #215

opatut opened this issue Mar 8, 2022 · 1 comment

Comments

@opatut
Copy link
Member

opatut commented Mar 8, 2022

I would like to see an export feature for my account, for moving between instances, or just having a backup of my data. This is required anyway for DSGVO inquiries, so why not add it?

It needs to contain, for now:

  • all raw files of my tracks
  • all derived files of my tracks
  • a JSON representation of all my track's details
  • all comments I wrote
  • my profile data

I'd say we make a "user_export" job table in the database, scan it the worker, generate a zipfile for each pending job, store this in a temporary folder, then allow the user to download this zipfile from the profile page if the export job is complete. The worker can also delete old export jobs and their related files after a while (say, 7 days or so).

@gluap
Copy link
Contributor

gluap commented May 19, 2023

A data-intensive part of this is now available in #321 (that mega-pr also contains bulk download of tracks). It works in real time. I believe adding "all my comments" as well as "all my track details" will still be possible in real time. It would save us having some extra queues and workers - we would only have to add a new api endpoint for the export without having workers, a queue and tables for shuffling around data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants