Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

End-to-end automated Dockerfile #78

Open
lidel opened this issue Feb 15, 2021 · 1 comment
Open

End-to-end automated Dockerfile #78

lidel opened this issue Feb 15, 2021 · 1 comment
Labels
dif/expert Extensive knowledge (implications, ramifications) required effort/days Estimated to take multiple days, but less than a week help wanted kind/maintenance Work required to avoid breaking changes or harm to project's status quo P2 Medium: Good to have, but can wait until someone steps up snapshots issues related to snapshot creation and updates

Comments

@lidel
Copy link
Member

lidel commented Feb 15, 2021

The version from #77 creates a basic environment with all dependencies for dev and debugging, but is not useful for end-to-end automation like one described in #58

The complexity involved is way too big for anyone sane to dig into this, so if we want to build snapshots more often than every 3-6 months, we must simplify the entire process and produce Docker image that:

  • takes inputs
    • a ZIM file
    • file system path for persisting IPFS_PATH with the output
  • returns
    • CID of produced snapshot
    • populates IPFS_PATH with all blocks

This way IPFS_PATH is independent of the Docker image and can be persisted/managed by other means (could be an ephemeral repo, or a permanent one).

@lidel lidel added help wanted dif/expert Extensive knowledge (implications, ramifications) required P2 Medium: Good to have, but can wait until someone steps up snapshots issues related to snapshot creation and updates kind/maintenance Work required to avoid breaking changes or harm to project's status quo effort/days Estimated to take multiple days, but less than a week labels Feb 15, 2021
@alzinging
Copy link

I can do this, with some changes ... this month. Faster if I see interest.

How often is Wikipedia snapshotred, I'd like to "ideally" see this done on the same schedule.

This thing is fragile. It needs love.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dif/expert Extensive knowledge (implications, ramifications) required effort/days Estimated to take multiple days, but less than a week help wanted kind/maintenance Work required to avoid breaking changes or harm to project's status quo P2 Medium: Good to have, but can wait until someone steps up snapshots issues related to snapshot creation and updates
Projects
None yet
Development

No branches or pull requests

2 participants