Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal to use Git LFS for large files #274

Open
bendichter opened this issue Jan 10, 2024 · 5 comments
Open

proposal to use Git LFS for large files #274

bendichter opened this issue Jan 10, 2024 · 5 comments
Assignees

Comments

@bendichter
Copy link
Contributor

Problem

With the inclusion of several large PDF and ZIP files, this repo is getting quite large. That has caused problems for me pulling and pushing, and I expect the problem to get worse as we add more reports and possibly other types of large files.

Proposed Solution

I propose we try using Git LFS. There are some nice guides for how to:

  1. Install Git LFS: https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage
  2. Configure Git LFS for specific file types: https://docs.github.com/en/repositories/working-with-files/managing-large-files/configuring-git-large-file-storage
  3. Move existing files in your repository to Git LFS storage: https://docs.github.com/en/repositories/working-with-files/managing-large-files/moving-a-file-in-your-repository-to-git-large-file-storage
@CodyCBakerPhD
Copy link
Collaborator

+1

@bendichter
Copy link
Contributor Author

@stephprince , want to take this?

@stephprince
Copy link
Contributor

sure!

@stephprince stephprince self-assigned this Jan 10, 2024
@stephprince
Copy link
Contributor

It seems Git LFS is not supported with GitHub pages site (see bottom of page here).

There is a potential workaround if we switch from deploying from a branch to deploying with GitHub actions - see discussion and related LFS/Pages issues here. From what I understand, this workaround means every workflow run will fetch the LFS data. If that sounds ok I can work on setting that up?

@oruebel
Copy link
Contributor

oruebel commented Jan 12, 2024

The PDF and Zip files that are causing the issue I believe are the reports from the events and as such are a) not necessary for the build since they are just pointed to as download links and b) don't really need Git-based version control. I'm wondering, whether it may be easier to simply create a shared GoogleDrive for web assets where we can store those files and update the links on the websites to point to the public files in the drive. For the PDF's (and Zip) files of the reports we could also just upload them to Zenodo to get DOI's for the reports. In this way we could still remove them from the repo but not need to deal with the complications that GitLFS brings with it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants