Architecture - having our cake and eating it too.

[calebboyd]

Starting a conversation to talk about how we can improve this tool to support and combine a couple of its key features that are currently mutually exclusive (🎂 🍽 )

• Fast Builds (using javascript as the runtime/tooling)
• Signable Builds
• Reliable VFS (and is this the correct approach for bundling)

For catching up to the start of this discussion you can refer to the links in this comment

[cspotcode]

Thanks for starting this. Today I'm working on rebasing #741 to the latest master. In the process, I'm re-learning nexe's architecture and trying to write some documentation comments as I go. I'll pepper questions here.

[calebboyd]

So working on this a little and since its going to be a big breaking change thought it might be good to remove some api surfaces...

So far thinking it will look like this

    Usage
      $ nexe [entry] [options]
    
    Options
      --input,     -i     Alternative to [entry]  
      --target,    -t     Specify a target (can be used more than once)
      --outFile,   -o     Specify the output filename
      --outDir,    -d     Specify a directory to write the output file(s) to.
      --resource,  -r     Specify additional file resources
      
      --cwd,              Base directory from which to conduct operations
      --no-input,         Do not bundle an entry file (produces empty binaries based on target)

    Build Options 
      see https://github.com/nodejs/node/blob/master/BUILDING.md for setup or troubleshooting
      
      --make,             Specify make options
      --vcbuild,          Specify vcbuild options
      --configure,        Specify configure options
      --v8-flag,          Specify a bundled v8-flag for build-time
      --v8-snapshot,      Specify a bundled v8-snapshot for warmup

And targets will be a URL, that supports different schemes.

eg:

• nexe://win32-14.16.0-x86 downloads nexe artifact release
• node://14.16.0-x64 builds node from source
• file://some-dir-somewhere build node from that source
• file|http(s)://some-file-somewhere builds using that asset
• file|http(s)://some-tarball builds from that source

[calebboyd]

cc @aral I'm curious if this surface would work for your usecases

[cspotcode]

Is the idea that --no-input can be used to build the nexe artifact
releases, which can then be uploaded somewhere and referred to in nexe --target? I like that; nice and clean.

There's a bit of ambiguity with how --output changes depending on the
number of targets. It is effectively 2 options: outDir and outFile.
E.g. if you wanted to write a reusable script to emit 1 or more targets
into a directory, it would need special logic to detect if the number of
targets is 1 or >1.

[calebboyd]

yeah thats exactly the idea. And - I agree with your assessment of the --output ambiguity...Might be able to add templating to the outFile option later too..

I should have most of solved problems scaffolded soon. Mostly trying to make the codebase simpler to reason about and contribute to..

[inian]

+1 for --no-input and targets. Will prevent needing to create a temporary file like this and will make caching intermediate builds much easier

[aral]

cc @aral I'm curious if this surface would work for your usecases

Hey @calebboyd, sorry I missed this. Yes, this should be fine and looks neater. Thank you :)

[code-ape]

New to Github Discussions, so apologies in advance if there's a proper format I'm missing here. Reposting suggestion from #878 here as suggested by @calebboyd:

My team is creating a cache of all pre-built nexe objects (os + node versions) and uploaded them all as compressed objects (using gzip). But when we went to use them, we discovered nexe doesn't appear to support compressed object download for remote URL.

Also, we open sourced our build automation today using GitHub Actions if it's of any interest 😄 it's still a bit rough around the edges but hopefully is useful for daily using bleeding edge node versions with Nexe.
https://github.com/urbdyn/nexe_builds