Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hosted Raw Videos #315

Open
nickatnight opened this issue Jun 6, 2020 · 11 comments
Open

Hosted Raw Videos #315

nickatnight opened this issue Jun 6, 2020 · 11 comments
Assignees
Labels
Engineering Changes our tools and data pipeline

Comments

@nickatnight
Copy link

I was able to get the raw videos hosted and linked to instances in my api on this repo (sourced from all-locations.json and stored in mongodb):

https://github.com/nickatnight/policebrutality.io

eg: https://prod-uploads-policebrutality.sfo2.digitaloceanspaces.com/1267647898365427714.mp4

Is there a plan to add this in your pipeline? You guys would probably need to manage a DO or AWS account. I could write an action to convert/download videos into some bucket, just need to know where. @pb-files @ubershmekel eh? You wouldn't even need to host any video files in the repo.

@yo-mike
Copy link
Collaborator

yo-mike commented Jun 6, 2020

Way to go @nickatnight. I’m looking forward to more updates from you.

@ubershmekel ubershmekel self-assigned this Jun 6, 2020
@ubershmekel ubershmekel added the Engineering Changes our tools and data pipeline label Jun 6, 2020
@EndingPoliceBrutality
Copy link

Hey everyone, I am a AWS Solutions Architect and I'd love to help on this issue or any way I can with this amazing project. I can setup an S3 backup of videos...maybe store them in Glacier for a long time. If you'd like this help or anything else cloud related on the project please let me know

@ubershmekel ubershmekel assigned bonedaddy and unassigned ubershmekel Jun 7, 2020
@ubershmekel
Copy link
Collaborator

ubershmekel commented Jun 7, 2020

Thank you @EndingPoliceBrutality we're still scratching our heads on how to organize all the back ups, images, videos, texts. There are a few issues on this topic:

The problem is we want to have official backups of our own, to empower others to make their own back ups, and we want to make the videos and images easy to access via the data api which is currently a process that parses the markdown images and makes a json, csv, and md file at https://github.com/2020PB/police-brutality/tree/data_build for every push to the repo.

So ideas there might be useful, and also if you just want to build a front end to the data using the above API that would be valuable too.

@yarikoptic
Copy link

yarikoptic commented Jun 7, 2020

As was suggested on datalad/datalad#4612 (comment) , https://git-annex.branchable.com/ with or without datalad could be used to orchestrate backup, annotation, dissemination etc of the materials. Content could be linked using git annex to original sources (urls, you tube, etc) and then backed up access a wide range of supported hosting platforms and personal computers and websites.

@danShumway
Copy link

danShumway commented Jun 7, 2020

@nickatnight do you feel comfortable with other people pulling and/or mirroring the API you have set up?

I'm putting together a front-end to try and get across the scale of the incidents, and I'm running into the same problem -- I'm heavily reliant on being able to actually host/serve the videos. I was planning on looping over the official API every couple of days and just seeing what youtube-dl could do, but if there's a more stable API I could hit or mirror in one pass without needing to worry about stuff like transcoding, that would save a lot of time.

@nickatnight
Copy link
Author

nickatnight commented Jun 7, 2020

@danShumway Go for it...that's what I made it, for other people to use :)

Edit:
I'll keep it up until the mods have an alternative. Would make sense to have everything in one place

@nickatnight
Copy link
Author

@EndingPoliceBrutality I 100% agree with an AWS solution. I'm not sure what the mods have in mind though

@ghost
Copy link

ghost commented Jun 8, 2020

Would this be something you could make a torrent file for on the server? I could grab it on my own and seed.

@yo-mike
Copy link
Collaborator

yo-mike commented Jun 9, 2020

@EndingPoliceBrutality - no need to wait. You are welcome to take initiative.

@ubershmekel
Copy link
Collaborator

Right now we're trying to keep everything in the markdown to make it easy for the public to edit. If people want more fields, more links, etc, in the official JSON - then you/we can implement that. What is missing right now? Is it mp4 file links?

@nickatnight
Copy link
Author

@ubershmekel raw links, thumbnails, duration. I know @949mac had some other ideas in mind

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Engineering Changes our tools and data pipeline
Projects
None yet
Development

No branches or pull requests

7 participants