Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

streaming >4GB-files doesn't come to an end. #193

Open
felixniemeyer opened this issue Apr 9, 2019 · 0 comments
Open

streaming >4GB-files doesn't come to an end. #193

felixniemeyer opened this issue Apr 9, 2019 · 0 comments

Comments

@felixniemeyer
Copy link

I'm trying to stream a compressed torrent file and pipe it to a decompressor.
node stream.js | bzip2 -dkc

It works fine for some files but fails for some other files - conspicuously it fails for files with sizes > 4GB:
The failure is, that the readStream never finishes. The readStream never emits end and the file in /tmp/torrent-stream/... remains incomplete. Only very little data is missing at the end of the file.
No error get's emitted by the readStream.

You can run this script to reproduce (the torrent is about reddit comments):

var torrentStream = require('torrent-stream');
var fs = require('fs')

var magnetLink = process.argv[2] || 'magnet:?xt=urn:btih:7690f71ea949b868080401c749e878f98de34d3d&dn=reddit%5Fdata&tr=http%3A%2F%2Ftracker.pushshift.io%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80'
var fileName = process.argv[3] || 'RC_2014-01.bz2'

var engine = torrentStream(magnetLink);

engine.on('ready', () => {
  var hasMatch = false;
	engine.files.forEach(file => {
    if(file.name == fileName) {
      hasMatch = true; 
      stream = file.createReadStream()      
      stream.pipe(process.stdout)
      stream.on('end', () => {
        engine.remove(false, () => {
          engine.destroy()
        }) 
      })
    }
	});
  if(!hasMatch) engine.destroy()
});

I tried to download the problematic file with a torrent client and it worked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant