Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macOS codesign fails #446

Open
2 of 3 tasks
frankie567 opened this issue Jan 24, 2018 · 22 comments · May be fixed by #1049
Open
2 of 3 tasks

macOS codesign fails #446

frankie567 opened this issue Jan 24, 2018 · 22 comments · May be fixed by #1049
Labels

Comments

@frankie567
Copy link

This is a

  • Bug Report
  • Feature Request
  • Other

Codesigning the binary produced by nexe on macOS fails. I'm not sure if it's a bug or just that it is not handled by nexe, but I wish to have your point of view about this.

Steps to reproduce:

  1. Create a dummy NodeJS script:
// ~/nexetest/index.js
console.log('hello world')
  1. Compile it using nexe:
nexe index.js
  1. Try to codesign it:
codesign --verbose -s - nexetest 
nexetest: main executable failed strict validation

I also tried to codesign the raw build in ~/.nexe directory. It works but, after building the code, the signature verification fails (obviously). If we try to force a resign, we get the same error as above.

  • Platform(OS/Version): macOS 10.13.2
  • Host Node Version: 8.9.4
  • Target Node Version: 8.9.4
  • Nexe version: 2.0.0-rc.22
  • Python Version: 2.7
@calebboyd
Copy link
Member

Try: #372 (comment)

Nexe relies on mangling the base binary which inherently breaks signing. Using the patch shown in that comment should allow you to embed your application fully. However, resources will not work.

If you really need resources + signing.. I have a plan to implement them, but no bandwidth to work on it at the moment.

@frankie567
Copy link
Author

frankie567 commented Jan 25, 2018

Hi @calebboyd,

Thank you for your feedback. I tried to apply the patch but unfortunately it's not working (still main executable failed strict validation error when signing). Here is my build configuration am I doing something wrong ?

{ 
  build: true,
  make: [ '-j4' ],
  input: './dist/entrypoint.js',
  targets: [ 'macos' ],
  output: 'daemon',
  patches: [
    (x, next) => {
      x.code = () => [x.shims.join(''), x.input].join(';')
      return next()
    },
   async (compiler, next) => {
      await compiler.setFileContentsAsync(
        'lib/_third_party_main.js',
        compiler.code()
      )
      compiler.options.empty = true // <-- ADDED THIS (hack)
      return next()
    }
  ]
}

I would be glad to help on this feature but I fear my knowledge on this subject is very limited.

@calebboyd
Copy link
Member

calebboyd commented Feb 1, 2018

Okay, I think I might know what the problem is. The executable is still mangled with metadata, setting empty after input is picked up but before the output begins should fix it... Pretty weird hack... but should work (edited your example)

@frankie567
Copy link
Author

frankie567 commented Feb 2, 2018

It works! I don't understand what is going on but it works like a charm! Thank you very much for your help 👍

@calebboyd
Copy link
Member

calebboyd commented Feb 2, 2018

Glad it works! Here is the summary:

Normally the binary looks like this:

| Custom (Nexe) Node Binary | Metadata + Your Code | (mangled)

The Patch does this

| Custom (Nexe + Your Code) Node Binary | Metadata | (mangled)

Setting empty: true at that point in time does this

| Custom (Nexe + Your Code) Node Binary | (not mangled!)

@frankie567
Copy link
Author

Thank you for those details, it's clearer now. The only drawback with this hack is that you have to do a clean build (i.e. by deleting the ~/.nexe/ directory) of Node, otherwise changes to source code are not applied.

@calebboyd
Copy link
Member

calebboyd commented Feb 19, 2018

Yes, This is (to my knowledge) an unavoidable side effect of creating a valid Mach-O binary. There is no way to change its contents and maintain validity without reconstructing it each time.

Note: You can also pass --clean and it will delete those contents for you and exit

@Nantris
Copy link

Nantris commented Sep 22, 2018

@calebboyd do you have any ideas about getting this working? I tried the code above with the only modification being the path to my input file, but the executable is still not signable unfortunately.

require('nexe').compile({
  build: true,
  input: './app/host/host.js',
  patches: [
    (x, next) => {
      x.code = () => [x.shims.join(''), x.input].join(';')
      return next()
    },
   (compiler, next) => {
      return compiler.setFileContentsAsync(
        'lib/_third_party_main.js',
        compiler.code()
      ).then(next)
    }
  ]
})

Edit: Trying again with the line compiler.options.empty = true // <-- ADDED THIS (hack) added. Any ideas would still be welcome though, as compilation takes 30 minutes and so my ability to try multiple iterations in a single day is limited.

@Nantris
Copy link

Nantris commented Sep 22, 2018

Uh, wow.... My original code took 30 minutes to build and couldn't be signed.

The code in the comment above took 1.85 seconds and can be signed. I don't know if that's thanks to caching or what, but:


Alt Text

All glory to @calebboyd

@brikendr
Copy link

Could someone please show me how to run the nexe builder with the fix from comment above.

Is this a configuration file you save it to your project root directory and use one of the nexe option to reference it?
Or is it a builder class that you just call like node build.js where build.js contains the configuration from the above comment?

@nonken
Copy link

nonken commented Mar 16, 2020

@brikendr I made this a build file and the build works. codesign --verbose -s - still fails with main executable failed strict validation even though I added the line compiler.options.empty = true to the script. I am using nexe 4.0.0-beta.4 on MacOS 10.15.3.

Does anyone have more information or example on how this can work? @calebboyd you also mentioned that this won't work with resource - is this because files would be included at runtime and invalidate the signature?

I am trying other options in the meantime. Node compilation takes an hour for me, so this is a slow process :)

@kariharju
Copy link

I'm in a similar situation as @brikendr I'm quite new to js and ts world and now have working sets for Windows and linux but the Mac notarization really does not like the executable coming out of nexe (nor any other similar tools). For me nexe calls have to be command-line so I'm trying to decipher how the --patches script.js would need to be formatted and what other arguments need to be given to be able to test this solution.

Are there any .js patch examples anywhere on this? How should I list multiple patches etc.
Simply put this noob has too many variables at the moment ;)

@durran
Copy link

durran commented May 24, 2020

For people coming to this issue now, some of the Nexe API has changed but the core issue has not been documented so well. I thought I would add our config here that got both signing and notarization working on v4.0.0-beta.6 with Node 12.4.0. The configure option was just our build boxes not being up to date, but shouldn't matter.

await compile({
  build: true,
  mangle: false,
  configure: ['--openssl-no-asm'],
  input: this.input,
  output: this.output,
  loglevel: 'verbose',
  targets: [ target ],
  patches: [
    (compiler, next) => {
      compiler.code = () => [ compiler.shims.join(''), compiler.startup ].join(';')
      return next();
    },
    (compiler, next) => {
      return compiler.setFileContentsAsync(
        'lib/_third_party_main.js',
        compiler.code()
      ).then(next);
    }
  ]
});

The { mangle: false, build: true, patches: [] } were the options that set us free, as well as not relying on resources and using Parcel as our bundling tool since it handled a complex Lerna monorepo and all its dependencies. Also note that in recent versions compiler.input from the previous comments became compiler.startup.

@kariharju
Copy link

I just ended up ditching nexe for Mac builds. So just bundling node and my app as .js worked as the node executable seems to conform to Apples desires.
Note that with build=true building will take a long time which will make CI pipeline a bit pointless and if you are paying for the build time (e.g. GitHub Actions) it can also mean costs.

..just to make it clear Apples tools and documentation for signing and notarization are absolutely horrendous. A signing system where everyone is using workarounds, tips and tricks is not exactly reassuring ;)

@maksymilianpiechota
Copy link

maksymilianpiechota commented May 26, 2020

For people coming to this issue now, some of the Nexe API has changed but the core issue has not been documented so well. I thought I would add our config here that got both signing and notarization working on v4.0.0-beta.6 with Node 12.4.0. The configure option was just our build boxes not being up to date, but shouldn't matter.

await compile({
  build: true,
  mangle: false,
  configure: ['--openssl-no-asm'],
  input: this.input,
  output: this.output,
  loglevel: 'verbose',
  targets: [ target ],
  patches: [
    (compiler, next) => {
      compiler.code = () => [ compiler.shims.join(''), compiler.startup ].join(';')
      return next();
    },
    (compiler, next) => {
      return compiler.setFileContentsAsync(
        'lib/_third_party_main.js',
        compiler.code()
      ).then(next);
    }
  ]
});

The { mangle: false, build: true, patches: [] } were the options that set us free, as well as not relying on resources and using Parcel as our bundling tool since it handled a complex Lerna monorepo and all its dependencies. Also note that in recent versions compiler.input from the previous comments became compiler.startup.

@calebboyd
Using version 3.x I got such edits in Node's source code (left is downloaded by hand, right is downloaded by nexe and it seems to be edited in some way by nexe):
image
Of course it can't work so compiler broke. Trying to comment it properly by hand didn't work because nexe did those edits again when fired. Is it some kind of bug?

@durran
So I upgraded to v4.0.0-beta6 and I used following config:

compile({
  build: true,
  make: [ '-j10' ],
  mangle: false,
  configure: ['--openssl-no-asm'],
  input: '../path/to/main/entry.js',
  loglevel: 'verbose',
  targets: [ 'macos-v12.16.3' ],
  patches: [
    (compiler, next) => {
      compiler.code = () => [ compiler.shims.join(''), compiler.startup ].join(';')
      return next();
    },
    (compiler, next) => {
      return compiler.setFileContentsAsync(
        'lib/_third_party_main.js',
        compiler.code()
      ).then(next);
    }
  ]
});

It seems that compilation errors were fixed in some version, so Node compiled and executable with source code is packed successfully. But for above config I get:
internal/bootstrap/pre_execution.js:83
throw 'Invalid Nexe binary';
^
Invalid Nexe binary
(Use node --trace-uncaught ... to show where the exception was thrown)

I debugged that 'mangle: false' option causes it. After commenting it works just fine. But I can't sign code then (I suppose mangling is the most important factor here).

Note that with mangle: false option I can sign the code (but as I said binary does not work)

@selfagency
Copy link

running nexe src/index.js --output=bin/2famsg --targets=macos-12.4.0 --verbose --build true --mangle false --configure=--openssl-no-asm still failed codesigning after compilation, as did trying @durran's approach above, minus the second patch.

@calebboyd
Copy link
Member

calebboyd commented May 26, 2020

Unfortunately node dropped support for _third_party_main.js in Node 12. So unless the version is less than 12 patching _third_party_main as described by the mangle docs, won't work.

nexe/README.md

Lines 113 to 116 in 0a2e8db

- #### `mangle: boolean`
- If set to false, nexe will not include the virtual filesystem (your application and resources) on the output.
- This will cause the output to error as an "Invalid Binary" unless a userland patch alters the contents of lib/_third_party_main.js in the nodejs source.
- default: true

I'll reopen this issue to track this case

@calebboyd calebboyd reopened this May 26, 2020
@selfagency
Copy link

nexe src/index.js --output=bin/2famsg --targets=macos-12.4.0 --verbose --build true --mangle false --configure=--openssl-no-asm; codesign -s "3rd Party Mac Developer Application" bin/2famsg

My binary comes out fine using the above. No compilation errors. But it still won't codesign.

ℹ nexe 4.0.0-beta.6
✔ Downloading Node.jssource from: https://nodejs.org/dist/v12.4.0/node-v12.4.0.tar.gz
✔ Compiling Node...
✔ Node binary compiled
✔ Compiling result
✔ Writing result to file
✔ Entry: 'src/index.js' written to: bin/2famsg
✔ Finished in 2630.117s
bin/2famsg: main executable failed strict validation

@durran
Copy link

durran commented May 27, 2020

I just realised I got a false positive out of that - must have been on a lower version... I can confirm ours is now failing with the same error @maksymilianpiechota mentioned. Going to try again on 11.15.0.

@Vannevelj
Copy link

Is there any setup that is supposed to work? I've got the same issue as the others when using 12.18.2.

When I use 10.15.3, 10.23.0 or 11.15.0 I get this error: #806

Every other symptom the same: if I enable mangle then it signs but doesn't run. If I disable it then it runs but doesn't sign.

@torabian
Copy link

آقا دمتون گرم کار کرد

@MrMathias
Copy link

For people coming to this issue now, some of the Nexe API has changed but the core issue has not been documented so well. I thought I would add our config here that got both signing and notarization working on v4.0.0-beta.6 with Node 12.4.0. The configure option was just our build boxes not being up to date, but shouldn't matter.

await compile({
  build: true,
  mangle: false,
  configure: ['--openssl-no-asm'],
  input: this.input,
  output: this.output,
  loglevel: 'verbose',
  targets: [ target ],
  patches: [
    (compiler, next) => {
      compiler.code = () => [ compiler.shims.join(''), compiler.startup ].join(';')
      return next();
    },
    (compiler, next) => {
      return compiler.setFileContentsAsync(
        'lib/_third_party_main.js',
        compiler.code()
      ).then(next);
    }
  ]
});

The { mangle: false, build: true, patches: [] } were the options that set us free, as well as not relying on resources and using Parcel as our bundling tool since it handled a complex Lerna monorepo and all its dependencies. Also note that in recent versions compiler.input from the previous comments became compiler.startup.

@durran I am afraid that this is the final mention of a successfully signed Nexe-build binary. May I ask you to attempt this again, or maybe go into some more details with what and how you managed to both build, run and sign?

@bruce-one bruce-one linked a pull request Apr 12, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.