Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Size limit for file #21

Open
saboughton opened this issue Apr 12, 2020 · 6 comments
Open

Size limit for file #21

saboughton opened this issue Apr 12, 2020 · 6 comments

Comments

@saboughton
Copy link

I was trying to open my xml file by uploading but it doesnt seem to do anything. it is 3.5GB, is there a size limit?

@MatthewMartino
Copy link

Same here, but mine is only 1.5 gigs, looking to do the same thing.

@GoTeamScotch
Copy link

Same issue. 775MB XML failed to load in both Chrome and Firefox (both current versions). Console not showing any useful errors.

@obesechess
Copy link

obesechess commented Dec 9, 2020

Same here but my file is an absolute whopper. Has this project been abandoned? I don't show that the dev has added anything since 2018. Has anyone been able to get this to work?

EDIT: well I guess he logged in haha

@devadvance
Copy link
Owner

Hi folks, there's no explicit file size limit, as it depends on the browser, operating system, and machine. I previously explored changes to support larger files, but that resulted in extremely slow file loading. I have been looking at alternative technologies outside of the browser to handle these scenarios, but none have the level of cross-platform support we need.

@obesechess
Copy link

Hi folks, there's no explicit file size limit, as it depends on the browser, operating system, and machine. I previously explored changes to support larger files, but that resulted in extremely slow file loading. I have been looking at alternative technologies outside of the browser to handle these scenarios, but none have the level of cross-platform support we need.

Thank you for the response! I’m playing around with a truly massive (approx 10GB) decrypted Signal backup and this seemed like my best shot for getting that stuff viewable outside of just buying a cheap Android phone to pop an SD Card into.

@TonyGravagno
Copy link

Just found and tried this. XML is only 340MB and it loads quickly on a laptop with i7 and 32GB RAM.

Maybe the answer for larger files is to find a way to split files up first and then load pieces separately. There are few applications that require simultaneous access to every byte of a multi-GB dataset.

We might be able to split a file by conversation size, like all contacts with < 2000 messages get put into a single file, and all other conversations are put into individual files.

If the load is still too large, like for a single conversation with a huge number of MMS, further split the files, remove the MMS, and make the images links or thumbnails. This can be done easily - Under tags mms>parts, for any 'part' element with a size > nMB and text="null", just replace the 'data' node with something tiny so that there is a marker. Create or replace the text node and add the value of 'name'. Another program might be created that scans a file for all 'part' nodes with 'ct' = image format, export those as files, and the generated web page can link to those. So this becomes (at least) a two-phase process specifically to deal with huge files.

Note, as a side benefit of the above, we get the image export that many of us want too. Export the images info a folder structure that corresponds to the conversation, date, originator phone ('address'), 'contact_name', etc. Can code from the Java v1 project be used for this?

Different people with different needs and preferences will have varying opinions on this. It's possible that there might be one solution that fits 90% of use cases, and the final 10% of users/cases will need to wait for a more advanced solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants