Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download of a ZIP seems to just concatenate content files #134

Open
dbenjaminmiller opened this issue Dec 13, 2021 · 7 comments · May be fixed by #169
Open

Download of a ZIP seems to just concatenate content files #134

dbenjaminmiller opened this issue Dec 13, 2021 · 7 comments · May be fixed by #169
Labels
bug Something isn't working

Comments

@dbenjaminmiller
Copy link

Currently using ffsend in Termux on Android to try to download some ZIPs. Seems to be the case that when I upload a ZIP, it downloads fine in the browser, but ffsend delivers the content files within the ZIP in a single unzipped file.

For instance, upload 1.jpg and 2.jpg in the browser. The resultant download in ffsend is a .zip which is actually just the two JPEGs concatenated. This amounts to being a valid JPEG as one of the two JPEGs, but which contains the data for both and is the according size.

Don't know if this is a problem on other platforms. But it's odd.

@timvisee
Copy link
Owner

timvisee commented Dec 13, 2021

I'm not quite sure what you're trying to explain here, sorry.

Send only supports uploading a single file. That's why, when choosing multiple files in the Send interface, or when selecting multiple files with ffsend, an archive is created which is then uploaded.

Are you saying that something breaks when extracting such archive?

ffsend archives to the TAR format by default. Which is not much more than concatenating files together along with metadata. Extracting this TAR should give you the original files back.

An image viewer might succeed in reading an image from the TAR archive. But you should extract the archive first instead.

@dbenjaminmiller
Copy link
Author

dbenjaminmiller commented Dec 13, 2021

OK, let me make it a bit clearer. Here's the chronology to reproduce.

  1. Use the browser, not FFsend, to send multiple files, which results in them being sent as a ZIP.
  2. Download the ZIP with the browser and observe that it is actually a ZIP.
  3. Download the ZIP with ffsend (on Android, at least), and find that it is not an actual ZIP, but a concatenation of the files inside the ZIP.

It's very odd.

This only seems to apply if the ZIP was compressed in the browser rather than an existing ZIP being uploaded.

@timvisee
Copy link
Owner

I see. Does a file uploaded in this way fail to extract (on other platforms as well)?

ffsend does not do any file modifications, except for extracting an archive it if you tell it to. With that I mean that you should get the exact same binary file (the ZIP archive) when you download it through a browser as well.

It has been a while since I worked last with ZIP internals. I don't recall if there's a mode to concatenate files without applying compression, to simply merge all into a single file. There probably is.

In any case, I'll take a look into it.

@timvisee
Copy link
Owner

timvisee commented Dec 13, 2021

Woah, you're right!

I was able to reproduce this. I'm seeing the same on my Linux machine. This is super weird.

I don't understand yet how something like this can happen.

@timvisee timvisee added the bug Something isn't working label Dec 13, 2021
@spl
Copy link

spl commented Jan 7, 2022

I believe I'm running into the same problem. Person A uploads a number of files to https://send.vis.ee/. Person B tries downloading the link from https://send.vis.ee/, but it reports an error after downloading. Person B tries using ffsend, which saves a file named Send-Archive.zip, but that file is reported as corrupt. Person B finds this issue, inspects the Send-Archive.zip, and finds that it does indeed appear to be the concatenation of the files uploaded by person A. Person B is using ffsend 0.2.74 on macOS from Homebrew.

@YDX-2147483647
Copy link

YDX-2147483647 commented Sep 29, 2023

I am inspecting https://github.com/timvisee/send/tree/6ad2885a168148fb996d3983457bc39527c7c8e5/ (current master)…

Send

Send supports uploading multiple files. If so, it'll concatenate the contents into a stream, call it Send-Archive.zip, and annotate it as a send-archive-type file. Note that the stream is just raw bytes (not a real zip), although the file name is *.zip.

https://github.com/timvisee/send/blob/6ad2885a168148fb996d3983457bc39527c7c8e5/app/archive.js#L26-L32

export default class Archive {
  get name() {
    return this.files.length > 1 ? 'Send-Archive.zip' : this.files[0].name;
  }
  get type() {
    return this.files.length > 1 ? 'send-archive' : this.files[0].type;
  }
  // …
}

Receive

After downloading & decrypting the raw stream, the browser will check the manifest. If it's a send-archive-type file, a zip will be created from the raw stream. (This is done at the client side)

https://github.com/timvisee/send/blob/6ad2885a168148fb996d3983457bc39527c7c8e5/app/fileReceiver.js#L95-L101

      let plainStream = this.keychain.decryptStream(blobStream(ciphertext));
      if (this.fileInfo.type === 'send-archive') {
        const zip = new Zip(this.fileInfo.manifest, plainStream);
        plainStream = zip.stream;
        size = zip.size;
      }
      // And convert `plainStream` to array buffer, …

I search this repo and https://github.com/timvisee/ffsend-api, and find nothing related to send-archive or Send-Archive.zip. Therefore I guess it's not implemented yet.

API responses

Example metadata:

let metadata = ApiMetadata::new(&file, password.clone(), false).invoke(&client)?;

Multiple files:

MetadataResponse {
    metadata: V3 {
        name: "Send-Archive.zip",
        mime: "send-archive",
        size: 52,
        manifest: Manifest {
            files: [
                ManifestFile {
                    name: "a",
                    mime: "",
                    size: 26,
                },
                ManifestFile {
                    name: "b",
                    mime: "",
                    size: 26,
                },
            ],
        },
    },
    ..
}

Normal (single) files:

MetadataResponse {
    metadata: V3 {
        name: "b",
        mime: "application/octet-stream",
        size: 26,
        manifest: Manifest {
            files: [
                ManifestFile {
                    name: "b",
                    mime: "",
                    size: 26,
                },
            ],
        },
    },
    ..
}

YDX-2147483647 added a commit to YDX-2147483647/ffsend that referenced this issue Sep 29, 2023
@YDX-2147483647 YDX-2147483647 linked a pull request Sep 29, 2023 that will close this issue
YDX-2147483647 added a commit to YDX-2147483647/ffsend that referenced this issue Sep 29, 2023
@YDX-2147483647
Copy link

YDX-2147483647 commented Sep 29, 2023

Hi everyone! I've worked out #169 and timvisee/ffsend-api#82, and you can try it by:

cargo install --git https://github.com/YDX-2147483647/ffsend.git --branch multi-with-api
$ ffsend download https://…
Download & Decrypt …/… [==========] 100.00 % … KB/s
Download complete
Splitting... 👈 This infers that ffsend is downloaing multiple files.

To revert: cargo uninstall ffsend

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants