EINTEGRITY error doesn't say for which module #872

vjpr · 2017-08-14T18:19:38Z

  27393 debug pnpm:progress:
    status: "downloaded_manifest"
    pkgId: "registry.npmjs.org/draft-js/0.10.1"
    pkgVersion: "0.10.1"
  27394 debug pnpm:progress:
    status: "dependencies_installed"
    pkgId: "registry.npmjs.org/draft-js/0.10.1"
  27395 debug pnpm:progress:
    status: "downloaded_manifest"
    pkgId: "registry.npmjs.org/fbjs/0.8.14"
    pkgVersion: "0.8.14"
  27396 debug pnpm:progress:
    status: "dependencies_installed"
    pkgId: "registry.npmjs.org/fbjs/0.8.14"
  27397 debug pnpm:progress:
    status: "downloaded_manifest"
    pkgId: "registry.npmjs.org/object-assign/4.1.1"
    pkgVersion: "4.1.1"
  27398 debug pnpm:progress:
    status: "dependencies_installed"
    pkgId: "registry.npmjs.org/object-assign/4.1.1"
  27399 debug pnpm:progress:
    status: "dependencies_installed"
    pkgId: "registry.npmjs.org/rc-editor-core/0.7.8"
  27400 debug pnpm:fetch: "finish sha1-E7TTyxK++hVIKib+Gy665kAHHks= https://registry.npmjs.org/immutable/-/immutable-3.7.6.tgz"
  27401 debug pnpm:progress:
    status: "fetched"
    pkgId: "registry.npmjs.org/immutable/3.7.6"
  27402 debug pnpm:progress:
    status: "downloaded_manifest"
    pkgId: "registry.npmjs.org/immutable/3.7.6"
    pkgVersion: "3.7.6"
  27403 debug pnpm:progress:
    status: "dependencies_installed"
    pkgId: "registry.npmjs.org/immutable/3.7.6"
  27404 debug pnpm:progress:
    status: "dependencies_installed"
    pkgId: "registry.npmjs.org/draft-js/0.10.1"
  27405 debug pnpm:progress:
    status: "dependencies_installed"
    pkgId: "registry.npmjs.org/rc-editor-mention/0.6.12"
  27406 debug pnpm:progress:
    status: "dependencies_installed"
    pkgId: "registry.npmjs.org/antd/2.12.5"
  27407 error pnpm:
    message:
      code: "EINTEGRITY"
      found: "sha1-dbSmLq6UB++20xCT5eJrkj11178="
      expected:
        - "sha1-mnHEh0chjrylHlGmbaaCA4zct78="
      algorithm: "sha1"
      sri: "sha1-mnHEh0chjrylHlGmbaaCA4zct78="
    err:
      name: "Error"
      message: "sha1-mnHEh0chjrylHlGmbaaCA4zct78= integrity checksum failed when using sha1: wanted sha1-mnHEh0chjrylHlGmbaaCA4zct78= but got sha1-dbSmLq6UB++20xCT5eJrkj11178=. (4292233 bytes)"
      code: "EINTEGRITY"
      stack: "Error: sha1-mnHEh0chjrylHlGmbaaCA4zct78= integrity checksum failed when using sha1: wanted sha1-mnHEh0chjrylHlGmbaaCA4zct78= but got sha1-dbSmLq6UB++20xCT5eJrkj11178=. (4292233 bytes)\n    at Transform.on (/usr/local/lib/node_modules/pnpm/lib/node_modules/ssri/index.js:275:19)\n    at emitNone (events.js:110:20)\n    at Transform.emit (events.js:207:7)\n    at endReadableNT (_stream_readable.js:1047:12)\n    at _combinedTickCallback (internal/process/next_tick.js:102:11)\n    at process._tickCallback (internal/process/next_tick.js:161:9)"

For my particular case it works on macOS but not on Ubuntu.

pnpm version: 1.10.2

Additional information:

node -v prints: 8.1.2
Windows, OS X, or Linux?: Ubuntu 16

BTW: Is there an easier way to check which module it was?

The text was updated successfully, but these errors were encountered:

vjpr · 2017-08-14T18:29:03Z

So I just had to look in the shrinkwrap.yaml.

For me the module was:

  /material-design-icons/3.0.1:
    resolution:
      integrity: sha1-mnHEh0chjrylHlGmbaaCA4zct78=

Could this integrity fail between OS's?

Could it be that it is reusing a package from the store that is broken?

Doesn't seem to be the case. I cleaned everything and it fails every time even w 8000 ithout a shrinkwrap.yaml.

vjpr · 2017-08-14T18:53:46Z

Same issue reported with npm and material-design-icons here: npm/npm#16861 (comment)

zkochan · 2017-08-17T07:44:49Z

The error is thrown from here. The invalid package can be used to write a test

vjpr · 2017-08-17T09:18:02Z

I don't think it is about this package in particular. I am thinking its about a partial tar download occurring in npm-package-client.

jbergstroem · 2017-08-18T22:19:33Z

@vjpr said:
I am thinking its about a partial tar download occurring in npm-package-client.

I suspect this is the root cause for similar issues I've experienced in poor wifi environments.

zkochan · 2017-08-20T17:56:51Z

@jbergstroem do you think we need to redownload the package if this error occurs?

vjpr · 2017-08-20T18:17:17Z

See more detail here: npm/npm#16861 (comment)

I think its an issue with npm-registry-client.

Re-download might be a good workaround for now though. Should definitely have message letting user know what is going on though and have a retry limit.

vjpr · 2017-08-20T19:12:42Z

I was just downloading a module on a patchy connection and got this error:

$ pnpm i -S react-vis
Creating dependency tree
Adding 32 packages to node_modules
ERROR sha1-L8eZlME0N9+o/a8iztjNFGj4GiQ= integrity checksum failed when using sha1: wanted sha1-L8eZlME0N9+o/a8iztjNFGj4Gi
Q= but got sha1-zx2Bd3m2OUeRHPqbJSs5zzDlb+M=. (2195456 bytes)
Resolving: total 77, reused 54, downloaded 22

Running pnpm i react-vis again showed a successful install, but I cannot see it in node_modules. This seems like a bug.

vjpr · 2017-08-21T09:48:34Z

I just hit this again...

$ lsl ~/.pnpm-store/2/registry.npmjs.org/warning/3.0.0
total 24
drwxr-xr-x  6 Vaughan  staff   204B 21 Aug 11:35 .
drwxr-xr-x  4 Vaughan  staff   136B 21 Aug 11:35 ..
-rw-r--r--  1 Vaughan  staff   1.1K 21 Aug 11:35 integrity.json
drwxr-xr-x  3 Vaughan  staff   102B 21 Aug 11:35 node_modules
lrwxr-xr-x  1 Vaughan  staff    20B 21 Aug 11:35 package -> node_modules/warning
-rw-r--r--  1 Vaughan  staff   2.9K 21 Aug 11:35 packed.tgz
$ lsl ~/.pnpm-store/2/registry.npmjs.org/material-design-icons/
total 0
drwxr-xr-x     4 Vaughan  staff   136B 21 Aug 11:35 .
drwxr-xr-x  1834 Vaughan  staff    61K 21 Aug 11:36 ..
drwxr-xr-x     3 Vaughan  staff   102B 21 Aug 11:35 3.0.1
drwxr-xr-x    19 Vaughan  staff   646B 21 Aug 11:37 3.0.1_stage
$ lsl ~/.pnpm-store/2/registry.npmjs.org/material-design-icons/3.0.1
total 26224
drwxr-xr-x  3 Vaughan  staff   102B 21 Aug 11:35 .
drwxr-xr-x  4 Vaughan  staff   136B 21 Aug 11:35 ..
-rw-r--r--  1 Vaughan  staff    13M 21 Aug 11:37 packed.tgz.3627129203
$ lsl ~/.pnpm-store/2/registry.npmjs.org/material-design-icons/3.0.1_stage
total 72
drwxr-xr-x  19 Vaughan  staff   646B 21 Aug 11:37 .
drwxr-xr-x   4 Vaughan  staff   136B 21 Aug 11:35 ..
-rwxr-xr-x   1 Vaughan  staff    13B  2 Sep  2016 .npmignore
-rw-r--r--   1 Vaughan  staff    11K  2 Sep  2016 LICENSE
-rw-r--r--   1 Vaughan  staff   1.9K  2 Sep  2016 README.md
drwxr-xr-x  12 Vaughan  staff   408B 21 Aug 11:35 alert
drwxr-xr-x  12 Vaughan  staff   408B 21 Aug 11:36 av
-rw-r--r--   1 Vaughan  staff   289B  2 Sep  2016 bower.json
drwxr-xr-x  12 Vaughan  staff   408B 21 Aug 11:36 communication
drwxr-xr-x  17 Vaughan  staff   578B 21 Aug 11:36 content
drwxr-xr-x  12 Vaughan  staff   408B 21 Aug 11:36 device
drwxr-xr-x  12 Vaughan  staff   408B 21 Aug 11:37 editor
drwxr-xr-x  12 Vaughan  staff   408B 21 Aug 11:37 file
-rwxr-xr-x   1 Vaughan  staff   3.8K  2 Sep  2016 gulpfile.babel.js
drwxr-xr-x  12 Vaughan  staff   408B 21 Aug 11:37 hardware
drwxr-xr-x  11 Vaughan  staff   374B 21 Aug 11:37 iconfont
drwxr-xr-x   6 Vaughan  staff   204B 21 Aug 11:37 image
-rw-r--r--   1 Vaughan  staff    48B  2 Sep  2016 index.js
-rw-r--r--   1 Vaughan  staff   872B  2 Sep  2016 package.json

So it seems like the 3.0.1_stage folder is present in material-design-icons but not in a package that successfully installed like warning.

packed.tgz.3627129203 is a temp file created by npm-registry-client > https://github.com/npm/fs-write-stream-atomic.

So the issue here first of all is that there should be an error when npm-registry-client fails to download, instead of waiting for it to be caught in pnpm-registry.

~~Also, ~/pnpm-registry was not created after a failed install (when it did not already exist).~~

vjpr · 2017-08-21T11:14:03Z

So I think what is happening is that the npm registry is occasionally prematurely closing its socket for large file downloads, which is not being reported as an error by request http client, although I'm not even sure if it can be.

We need to add re-download logic if the integrity fails.

And is the partially unpacked tar cleaned up on integrity failure?

vjpr · 2017-08-21T12:11:18Z

Ok I feel like I have got to the bottom of this issue now. Details here: https://github.com/npm/registry/issues/202

For pnpm, we need to check the Content-Length of the response, and then retry the download.

zkochan · 2017-08-21T12:27:05Z

There is an old issue about Content-Length: #423. If we solve this, we can close that one

vjpr · 2017-08-21T12:53:35Z

Yep. pnpm/package-store is what needs changing I think.

…

On Mon 21. Aug 2017 at 2:27 PM, Zoltan Kochan ***@***.***> wrote: There is an old issue about Content-Length: #423 <#423>. If we solve this, we can close that one — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#872 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARLRT1dPVdKHa1M3P9dwUZ6oXQQZn7_ks5saXeagaJpZM4O2t5J> .

jbergstroem · 2017-08-28T23:42:41Z

@zkochan said:
@jbergstroem do you think we need to redownload the package if this error occurs?

Apologies, didn't see this until now. I'd either bail early or retry. Seeing how pnpm is taking the Just Works approach (successfully so), I think retrying with a configurable amount of attempts makes sense.

zkochan · 2017-08-29T08:00:44Z

This is the solution npm will apply but they won't add it to npm-registry-client (they are deprecating it slowly), so we'll have to add the retry logic to our package-store package.

The hardest part for me is to write an integration test for it

Ref pnpm/pnpm#872

zkochan · 2017-09-08T19:03:06Z

Published with pnpm@1.13.1

vjpr mentioned this issue Aug 15, 2017

Z_BUF_ERROR - unexpected end of file #600

Closed

zkochan added area: reporting size: S labels Aug 17, 2017

vjpr mentioned this issue Aug 21, 2017

Check the Content-Length of the response, and then retry the download if it doesn't match pnpm/package-store#2

Closed

zkochan added a commit to pnpm/package-store that referenced this issue Sep 8, 2017

fix: re-download when integrity check fails

2233eab

Ref pnpm/pnpm#872

zkochan added the state: in progress label Sep 8, 2017

zkochan added a commit to pnpm/logger that referenced this issue Sep 8, 2017

feat: log attempt number on fetching started

8e88620

Ref pnpm/pnpm#872

zkochan added a commit to pnpm/package-store that referenced this issue Sep 8, 2017

fix: re-download when integrity check fails

33a2db9

Ref pnpm/pnpm#872

zkochan added a commit to pnpm/package-store that referenced this issue Sep 8, 2017

feat: specify for which tarball interity check failed

31bfce0

Ref pnpm/pnpm#872

zkochan added a commit to pnpm/supi that referenced this issue Sep 8, 2017

fix: retry tarball download on bad length and integity failure

f6981ee

Ref pnpm/pnpm#872

zkochan added a commit to pnpm/supi that referenced this issue Sep 8, 2017

fix: retry tarball download on bad length and integity failure

2872b9a

Ref pnpm/pnpm#872

zkochan closed this as completed in eca48f1 Sep 8, 2017

zkochan removed the state: in progress label Sep 8, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

EINTEGRITY error doesn't say for which module #872

EINTEGRITY error doesn't say for which module #872

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

EINTEGRITY error doesn't say for which module #872

EINTEGRITY error doesn't say for which module #872

Comments

Uh oh!

pnpm version: 1.10.2

Additional information:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!