Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

3748 large file upload #4584

Draft
wants to merge 49 commits into
base: develop
Choose a base branch
from
Draft

3748 large file upload #4584

wants to merge 49 commits into from

Conversation

alvacouch
Copy link
Contributor

This is a prototype of large file upload. It includes:

  1. a separate page that manages the upload
  2. the ability to point the upload at any directory in a resource.
  3. A simple chooser for the file to upload. Local name of the file is preserved.

What this does not do:

  1. It uses the standard onboarding into iRODS without change.
  2. It does progress reporting on the file transfer but not on ingestion.
  3. It does not limit file size.

Pull Request Checklist:

  • Positive Test Case Written by Dev
  • Automated Testing
  • Sufficient User and Developer Documentation
  • Passing Jenkins Build
  • Peer Code review and approval

Positive Test Case

  1. [Enter positive test case here]

@alvacouch
First integration phase: tusd runs in the hydroshare stack.
c4d802e
@alvacouch
tusd gets along with HydroShare stack.
4198420
@alvacouch
NGINX and initial DJango configuration -- first draft
2b744f5
@alvacouch
Working version of tus script setup.
3d6acd3
@alvacouch
Eliminating django authentication from the tusd transfer itself
bcff4ea
@alvacouch
Move upload to new app ./hs_upload
6334630
@alvacouch
Completed port to ./hs_upload
d4aac45
@alvacouch
More tuning of hs_upload
b5d5a9a
@alvacouch
Fix urls.py to pre-built state.
3b5fe4a
@alvacouch
First working logging of tusd
9a392cf
@alvacouch
First version with irods update working.
560b444
@alvacouch
Change protocol for more security
05f9e33
@alvacouch
change location of tusd hooks in Hydroshare
a036d6c
@alvacouch
Merge branch 'develop' into 3748-nginx-upload
f09253c
@alvacouch
First version that checks uploads in progress
77f067e
@alvacouch
Version before shifting gears
e090b29
@alvacouch
First version under new security rules. Everything but django ingesti…
abd6516 
…on and cleanup.
@alvacouch
Deletion of temporary file works.
e6799aa
@alvacouch
First version with all components
a07036d
@alvacouch alvacouch added Performance File System Resource Landing Page resource landing page issues 3rd Party File Exchange Pulling files from or putting files in a separate system labels Apr 10, 2022
@alvacouch alvacouch self-assigned this Apr 10, 2022
@alvacouch alvacouch marked this pull request as draft April 10, 2022 18:05
@hydrocheck
Copy link
Collaborator

METRIC VALUE

https://sonarqube.cuahsi-workstation.com:9000/dashboard?id=hydroshare-4584

@alvacouch
Pass flake8
56f7da6
@alvacouch
Better messages in popup window
47c3a4e
@alvacouch
Extra window to handle upload handoffs from downstream file chooser.
5e39858
@alvacouch
First version with uppy
8aaa087
@alvacouch
First version with no synchronous actions.
dfda82c
@alvacouch
Version that works, albeit with poor error handling.
7577db5
@alvacouch
First version with working file upload and working journal.
a6cb0e6
@alvacouch
UPPY button looks like "add files".
a2f5b77
@alvacouch
Management command unwedges stuck uploads caused by JS crashes.
85147a1
@alvacouch
Renamed variables; straightened out Upload.delete --> Upload.remove
7b2e70d
@alvacouch
Separate out validation of file target.
0dc66d1
@alvacouch
Avoid race condition when entering file format metadata.
5e78219
@alvacouch
Version with window interaction.
d916477
@alvacouch
Clean up routines so they can be used for all uploads.
66bb97d
@alvacouch
First version of uppy companion, not yet fully functional.
8c7309e
@alvacouch
Fix small bug with debugging write.
62d7825
@alvacouch
Actually talks to Google now, but there's an error.
3057424
@alvacouch
Non-working version with correct responses from uppy, but communicati…
365f138 
…on to third parties is garbled.
@alvacouch
Everything except final transfer stage. Seems to need me to limit fil…
0e5d0c5 
…e sizes.
@alvacouch
A non-working version that is closer.
7d03f28
@alvacouch
Still not working. File upload only.
8d53da0
@alvacouch
Closer to working but references paths with an incorrect protocol.
2cd4239
@alvacouch
First version in which uppy runs.
7f0d17d
@alvacouch
Commiting debugging resources temporarily.
8b02d31
@alvacouch
Merge branch 'develop' into 3748-nginx-upload
ac0e5df
@alvacouch
Adapt to python 3
29f433f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@alvacouch alvacouch

Labels
3rd Party File Exchange Pulling files from or putting files in a separate system File System Performance Resource Landing Page resource landing page issues
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@alvacouch @hydrocheck