Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Slow file uploading after meteor upgrade #852

Open
valerymagadeeva opened this issue Sep 25, 2022 · 18 comments
Open

Slow file uploading after meteor upgrade #852

valerymagadeeva opened this issue Sep 25, 2022 · 18 comments
Labels
help wanted Need More Info question

Comments

@valerymagadeeva
Copy link

valerymagadeeva commented Sep 25, 2022 •
edited

Hello! I updated my meteor application to meteor 2.7 (and meteor-files to 2.3.0) and as a result I have a strange issue - the file uploading takes too long time, it uploads extremely slow. (the difference between the old version and the new one is about 30 seconds for 40mb file).
I tried to use both of DDP and HTTP transports and nothing helps. I noticed that when I used HTTP to upload files the meteor server response took at least a second for every chunk. So I guess the problem is somewhere between the client side and the server side.
To upload file from client i use insert() method with chunkSize equals to "dynamic" value. The server side code is the FilesCollection constructor:

const Uploads = new FilesCollection({
    collectionName: 'preupload',
    allowClientCode: false,
    debug: false,
    protected: true,
    onBeforeUpload(file) {},
    onAfterUpload(uploadedFile) {}
})
  • Meteor-Files version - 2.3.0;
  • Meteor version - 2.7;
  • Browser/OS - Google Chrome/MacOS;
  • Seems like that might be server side issue;

Client logs:
[FilesCollection] [insert()]

vue-composition-api.mjs:2246 [FilesCollection] [FileUpload] [constructor]
vue-composition-api.mjs:2246 onBeforeUpload => Mon Sep 26 2022 13:38:11 GMT+0400 (Samara Standard Time)
vue-composition-api.mjs:2246 [FilesCollection] [insert] using WebWorkers
upload.js:368 loadFile ccsetup592.bin (1).zip: 95912.81884765625 ms
vue-composition-api.mjs:2246 [FilesCollection] [UploadInstance] [sendEOF] false
vue-composition-api.mjs:2246 [FilesCollection] [UploadInstance] [end] ccsetup592.bin (1).zip
upload.js:319 insert ccsetup592.bin (1).zip: 96609.42993164062 ms

Server logs(last part) (do not pay attention to warning, i just need console.time for time measurements):

I20220926-09:39:42.742(0)? [FilesCollection] [Upload] [DDP] Got #141/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:42.742(0)? onBeforeUpload => 2022-09-26T09:39:42.742Z
W20220926-09:39:42.743(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:43.262(0)? [FilesCollection] [Upload] [DDP] Got #142/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:43.263(0)? onBeforeUpload => 2022-09-26T09:39:43.262Z
W20220926-09:39:43.263(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:43.888(0)? [FilesCollection] [Upload] [DDP] Got #143/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:43.888(0)? onBeforeUpload => 2022-09-26T09:39:43.888Z
W20220926-09:39:43.888(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:44.825(0)? [FilesCollection] [Upload] [DDP] Got #144/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:44.826(0)? onBeforeUpload => 2022-09-26T09:39:44.825Z
W20220926-09:39:44.826(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:45.649(0)? [FilesCollection] [Upload] [DDP] Got #145/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:45.649(0)? onBeforeUpload => 2022-09-26T09:39:45.649Z
W20220926-09:39:45.650(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:46.173(0)? [FilesCollection] [Upload] [DDP] Got #146/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:46.174(0)? onBeforeUpload => 2022-09-26T09:39:46.173Z
W20220926-09:39:46.174(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:46.954(0)? [FilesCollection] [Upload] [DDP] Got #147/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:46.955(0)? onBeforeUpload => 2022-09-26T09:39:46.954Z
W20220926-09:39:46.955(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:47.503(0)? [FilesCollection] [Upload] [DDP] Got #148/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:47.503(0)? onBeforeUpload => 2022-09-26T09:39:47.503Z
W20220926-09:39:47.504(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:48.121(0)? [FilesCollection] [Upload] [DDP] Got #149/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:48.121(0)? onBeforeUpload => 2022-09-26T09:39:48.121Z
W20220926-09:39:48.121(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:48.192(0)? [FilesCollection] [Upload] [DDP] Got #-1/149 chunks, dst: ccsetup592.bin (1).zip
I20220926-09:39:48.193(0)? onBeforeUpload => 2022-09-26T09:39:48.192Z
W20220926-09:39:48.193(0)? (STDERR) (node:129) Warning: Label 'file uploading1' already exists for console.time()
I20220926-09:39:48.195(0)? [FilesCollection] [Upload] [finish(ing)Upload] -> assets/app/uploads/preupload/sQW9M6qt4WaLbzefJ.zip
I20220926-09:39:48.211(0)? [FilesCollection] [Upload] [finish(ed)Upload] -> assets/app/uploads/preupload/sQW9M6qt4WaLbzefJ.zip
I20220926-09:39:48.212(0)? file uploading1: 1:36.508 (m:ss.mmm)

Thank you in advance!
@dr-dimitru
Copy link
Member

Hello @valerymagadeeva ,

Please update your OP following our ISSUE_TEMPLATE. And post logs with debug enabled

@valerymagadeeva
Copy link
Author

@dr-dimitru updated!

@dr-dimitru
Copy link
Member

@valerymagadeeva thanks, let's start with question:

  1. Why you upload ccsetup592.bin but file saved as ZIP (sQW9M6qt4WaLbzefJ.zip)?

@valerymagadeeva
Copy link
Author

The file's full name is ccsetup592.bin (1).zip - it's zip archive and it should have such extension. Anyway I tried to upload a binary file - the result is the same

@dr-dimitru
Copy link
Member

@valerymagadeeva Do you zip the file during or after upload?

@dr-dimitru
Copy link
Member

@valerymagadeeva to make sure this isn't caused your development environment please download, launch locally, and test uploads via this app (runs on the latest meteor and meteor-files)

@valerymagadeeva
Copy link
Author

@dr-dimitru I unzip the file after uploading in the other part of our system. The problem is not connected to zip file. I tried your demo app and got the same result

@dr-dimitru
Copy link
Member

@valerymagadeeva do you get a good speed when uploading at files.veliov.com ?

@valerymagadeeva
Copy link
Author

@dr-dimitru nope, almost the same result (maybe 10 seconds faster). While in the old version of meteor I have uploaded the file for 45-50 seconds (vs 1m 20 - 1m30 seconds in your demo and my app with the new version of meteor)

@dr-dimitru
Copy link
Member

@valerymagadeeva loads very local, any other device to test on?

I'm getting 10MB/s, which is above my upload speed test
Screen Shot 2022-09-26 at 2 51 30 PM
Screen Shot 2022-09-26 at 2 53 33 PM

@valerymagadeeva
Copy link
Author

Screenshot 2022-09-26 at 16 03 03
I have lower speed when trying to upload via your demo app. But my internet speed is higher
Screenshot 2022-09-26 at 16 04 02

@dr-dimitru
Copy link
Member

dr-dimitru commented Sep 26, 2022 •
edited

  1. But you're saying speed was higher before last update? Including on demo website?
  2. What speed do you have locally?

@valerymagadeeva
Copy link
Author

  1. idk if the speed was higher on your demo website. Now i have two git branches with file uploading - the first one with old Meteor version (1.11 unfortunately and the file is uploading for 40-50 seconds) and the second one with the new version.
  2. What do you mean under local speed? My hard drive write speed?

@dr-dimitru
Copy link
Member

TLTD; we need clear reproduction steps in order to identify an issue

@valerymagadeeva
Copy link
Author

What info can i provide you?

@dr-dimitru
Copy link
Member

@valerymagadeeva can not come up with something specific, as I'm not sure how to localize your issue

@valerymagadeeva
Copy link
Author

@dr-dimitru ok, I'll try to provide something specific to reproduce the issue. But it seems I have a poor upload speed (according to the information from the demo app). Idk why and im sure it's not connected to my internet or something

@dr-dimitru
Copy link
Member

Does anyone else experiencing this? Or have ever experienced?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted Need More Info question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dr-dimitru @valerymagadeeva