Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QUESTION] Is there a way to define timeout for npm install #1151

Closed
sidoruk-sv opened this issue Apr 14, 2020 · 11 comments
Closed

[QUESTION] Is there a way to define timeout for npm install #1151

sidoruk-sv opened this issue Apr 14, 2020 · 11 comments
Assignees
Labels
Awaiting Information further information is requested Bug thing that needs fixing Needs Discussion is pending a discussion Release 6.x work is associated with a specific npm 6 release semver:patch semver patch level for changes

Comments

@sidoruk-sv
Copy link

sidoruk-sv commented Apr 14, 2020

What / Why

For now, we have a problem with the package, which meta info file (where all versions are listed) weights more than 110Mb and it takes 2.5 minutes to download the file on my machine, for our team it causes annoying npm install failed because of

Response timeout while trying to fetch http://regsitry.url/example-meta-file (over 30000ms)

Where

For now, on the local environment, I had to tune npm files, to change the default timeout.
In NPM debug log I found that Response timeout... message comes from
~/.nvm/versions/node/v12.16.1/lib/node_modules/npm/node_modules/node-fetch-npm/src/body.js:189:16
This one sources: https://github.com/npm/node-fetch-npm/blob/latest/src/body.js#L189

I hardcoded that in such a way:

    // allow timeout on slow response body
    if (this.timeout) {
      resTimeout = setTimeout(() => {
        abort = true
        reject(new FetchError(`Response timeout while trying to fetch ${this.url} (over ${this.timeout}ms)`, 'body-timeout'))
-      }, this.timeout)
+      }, 5 * 60* 1000) // 5 minutes timeout to prevent failing on huge meta files downloading
    }

Who

n/a

References

n/a

So the question is:

Is there some parameter/ENV_VARIABLE, that can be passed or be defined before npm install command, which will override this.timeout value in the code that I share.

@sidoruk-sv
Copy link
Author

Also worth mentioning:
couldn't find in code of @npm/cli or node-fetch-npm anything about 30000 which is like the "default timeout" passed to node-fetch-npm somehow

@darrinholst
Copy link

Seems to be used in make-fetch-happen which is used in npm-registry-fetch and pacote. We're running into an issue where the metadata has gotten big enough to push us over 30 seconds in some environments.

There still doesn't seem to be a way to configure the magic timeout option - https://github.com/npm/cli/blob/latest/lib/fetch-package-metadata.js#L56-L58

@murtada-dev
Copy link

murtada-dev commented Apr 14, 2020

Same here I post it on ionic cli github
ionic-team/ionic-cli#4387

@darrinholst
Copy link

downgrading to 6.13.x resolved it for me - ea0ff56#diff-3c78131ed196efbd3ce9fdee2df36e24R79

@sidoruk-sv
Copy link
Author

@darcyclarke for now, we get more problems because of that, cause latest Node.js v12.16.2 was shipped with NPM v6.14.x by default.

Seems like upgrading npm-registry-fetch in NPM v6.14.0 caused our problem with a timeout as @darrinholst mentioned earlier:
ea0ff56

Without the ability to redefine timeout for that npm-registry-fetch in NPM v6.14.x, we had to fixate Node.js version to 12.16.1 as it is the last version with NPM v6.13.x.

And seems like it is a bug, cause before that changes ea0ff56#diff-acf38193ec0e2d9a3b9dc202f239a77aR12, huge meta-files was not a problem at all.

@darcyclarke darcyclarke added Bug thing that needs fixing Release 6.x work is associated with a specific npm 6 release semver:patch semver patch level for changes labels Apr 15, 2020
@darcyclarke darcyclarke added this to the OSS - Sprint 7 milestone Apr 15, 2020
@darcyclarke darcyclarke added Needs Discussion is pending a discussion Awaiting Information further information is requested labels Apr 15, 2020
@darcyclarke
Copy link
Contributor

darcyclarke commented Apr 15, 2020

@sidoruk-sv appreciate you bringing this to our attention. I'm going to take this back internally & see if we can ship a patch to resolve this &/or find/introduce a flag.

@RiZKiT
Copy link

RiZKiT commented Apr 22, 2020

We are unable to deploy our project, most often webpack meta file does timeout, sometimes firebase. It got worse the last weeks, seems the meta file now reached a point were it is consistantly failing. We will try to downgrade to 6.13.7.

Edit: after downgrading we got the first green build after 10+ fails, looks promising.

isaacs added a commit to npm/npm-registry-fetch that referenced this issue Apr 23, 2020
Re: #26
Re: npm/cli#1151

The documented default timeout of 30s was not being set in v4, so we
"fixed the glitch" in v4.0.3, causing problems for people trying to
download large packages.

There's no npm v6 way to specify what timeout to use, so not having a
timeout at all seems like a reasonable default for the v4 line, at
least. Let's roll back that change, and document it. (Arguably, fixing
this bug was a breaking change, and we ought to roll it back.)

This effectively reverts 69c2977, with
documentation of the effective behavior before the change.
isaacs added a commit to npm/npm-registry-fetch that referenced this issue Apr 23, 2020
Re: #26
Re: npm/cli#1151

The previous default of 30s was too small for lots of users, causing
problems when they attempt to download large objects from the npm
registry.

Bump up the default timeout to 5m.

TODO: add a `--fetch-timeout` option on the CLI to explicitly set
`timeout` in the npm.flatOptions object passed to all dependencies.
@isaacs
Copy link
Contributor

isaacs commented Apr 23, 2020

See npm/npm-registry-fetch#26

Let's move the default timeout back to zero for nrf v4 (npm v6), bump it up to 5m in latest (npm v7), and add a --fetch-timeout option in the CLI v7 to set this explicitly.

I think that'd solve the immediate issue and put us in a better spot going forward.

isaacs added a commit to npm/npm-registry-fetch that referenced this issue Apr 28, 2020
Re: #26
Re: npm/cli#1151

The documented default timeout of 30s was not being set in v4, so we
"fixed the glitch" in v4.0.3, causing problems for people trying to
download large packages.

There's no npm v6 way to specify what timeout to use, so not having a
timeout at all seems like a reasonable default for the v4 line, at
least. Let's roll back that change, and document it. (Arguably, fixing
this bug was a breaking change, and we ought to roll it back.)

This effectively reverts 69c2977, with
documentation of the effective behavior before the change.

PR-URL: #27
Credit: @isaacs
Close: #27
Reviewed-by: @isaacs
isaacs added a commit to npm/npm-registry-fetch that referenced this issue Apr 28, 2020
Re: #26
Re: npm/cli#1151

The previous default of 30s was too small for lots of users, causing
problems when they attempt to download large objects from the npm
registry.

Bump up the default timeout to 5m.

TODO: add a `--fetch-timeout` option on the CLI to explicitly set
`timeout` in the npm.flatOptions object passed to all dependencies.

PR-URL: #28
Credit: @isaacs
Close: #28
Reviewed-by: @isaacs
@sidoruk-sv
Copy link
Author

sidoruk-sv commented May 12, 2020

So the current fix is to upgrade to NPM 6.14.5.

It would be perfect when latest Node.js versions (10.x, 12.x, 13.x, 14.x) to be shipped with 6.14.5 instead of 6.14.4:
https://nodejs.org/en/download/releases/

@jdmarshall
Copy link

the npm config documentation has several entries for timeouts. Is this code ignoring those settings, or is there no setting for this specific case and that's the issue?

@darrinholst
Copy link

is there no setting for this specific case and that's the issue?

yes, option should be coming in v7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Awaiting Information further information is requested Bug thing that needs fixing Needs Discussion is pending a discussion Release 6.x work is associated with a specific npm 6 release semver:patch semver patch level for changes
Projects
None yet
Development

No branches or pull requests

7 participants