Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

node rollup does not work #20407

Open
7 tasks done
Kreijstal opened this issue Mar 22, 2024 · 13 comments
Open
7 tasks done

node rollup does not work #20407

Kreijstal opened this issue Mar 22, 2024 · 13 comments
Labels

Comments

@Kreijstal
Copy link
Contributor

Description / Steps to reproduce the issue

require node rollup
use rollup -c will give you an esoteric error, but it is clear that it was made with msvc in mind, we can see this, because when we npm install rollup we get: https://www.npmjs.com/package/@rollup/rollup-win32-x64-msvc ,
Not sure if either npm would have to be patched, or rollup would be just patched and installed as a msys2 package.. (If that can be figured out).
Anyway, also reporting this for future viewers. And people that happen to come across this tauri-apps/tauri#6685

Expected behavior

gives a meaningful error or it works

Actual behavior

$ npm run build

> js-interpreter-npm@1.0.8 build
> rollup -c

thread '<unnamed>' panicked at C:\Users\runneradmin\.cargo\registry\src\index.crates.io-6f17d22bba15001f\napi-sys-2.3.0\src\functions.rs:7:3:
Must load N-API bindings
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Verification

Windows Version

MINGW64_NT-10.0-22631 ayylmai 3.4.10.x86_64 2024-02-10 08:39 UTC x86_64 Msys

MINGW environments affected

  • MINGW64
  • MINGW32
  • UCRT64
  • CLANG64
  • CLANG32
  • CLANGARM64

Are you willing to submit a PR?

if I figure out a solution

@Kreijstal
Copy link
Contributor Author

Alternatively, it seems webpack has no trouble with it.

@raedrizqie
Copy link
Contributor

it is not about npm

@Kreijstal
Copy link
Contributor Author

going through the dependency trail now attempting to make it work with napi-rs. napi-rs/napi-rs#2001

Load Node-API [napi_get_last_error_info] from host runtime failed: GetProcAddress failed
Load Node-API [napi_get_uv_event_loop] from host runtime failed: GetProcAddress failed
Load Node-API [napi_fatal_exception] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_threadsafe_function] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_date] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_bigint_int64] from host runtime failed: GetProcAddress failed
Load Node-API [napi_detach_arraybuffer] from host runtime failed: GetProcAddress failed
Load Node-API [napi_add_async_cleanup_hook] from host runtime failed: GetProcAddress failed
Load Node-API [node_api_symbol_for] from host runtime failed: GetProcAddress failed
Load Node-API [node_api_create_external_string_latin1] from host runtime failed: GetProcAddress failed

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src\lib.rs (target\debug\deps\napi-dec594a7a0918046.exe)
Load Node-API [napi_get_last_error_info] from host runtime failed: GetProcAddress failed
Load Node-API [napi_get_uv_event_loop] from host runtime failed: GetProcAddress failed
Load Node-API [napi_fatal_exception] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_threadsafe_function] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_date] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_bigint_int64] from host runtime failed: GetProcAddress failed
Load Node-API [napi_detach_arraybuffer] from host runtime failed: GetProcAddress failed
Load Node-API [napi_add_async_cleanup_hook] from host runtime failed: GetProcAddress failed
Load Node-API [node_api_symbol_for] from host runtime failed: GetProcAddress failed
Load Node-API [node_api_create_external_string_latin1] from host runtime failed: GetProcAddress failed

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src\lib.rs (target\debug\deps\napi_bench-d35302d6888ef6b3.exe)
Load Node-API [napi_get_last_error_info] from host runtime failed: GetProcAddress failed
Load Node-API [napi_get_uv_event_loop] from host runtime failed: GetProcAddress failed
Load Node-API [napi_fatal_exception] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_threadsafe_function] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_date] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_bigint_int64] from host runtime failed: GetProcAddress failed
Load Node-API [napi_detach_arraybuffer] from host runtime failed: GetProcAddress failed
Load Node-API [napi_add_async_cleanup_hook] from host runtime failed: GetProcAddress failed
Load Node-API [node_api_symbol_for] from host runtime failed: GetProcAddress failed
Load Node-API [node_api_create_external_string_latin1] from host runtime failed: GetProcAddress failed

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src\lib.rs (target\debug\deps\napi_build-c42c8f0fb53550c7.exe)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src\lib.rs (target\debug\deps\napi_compat_mode_examples-dee832aaf9cd1cc4.exe)
Load Node-API [napi_get_last_error_info] from host runtime failed: GetProcAddress failed
Load Node-API [napi_get_uv_event_loop] from host runtime failed: GetProcAddress failed
Load Node-API [napi_fatal_exception] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_threadsafe_function] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_date] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_bigint_int64] from host runtime failed: GetProcAddress failed
Load Node-API [napi_detach_arraybuffer] from host runtime failed: GetProcAddress failed
Load Node-API [napi_add_async_cleanup_hook] from host runtime failed: GetProcAddress failed
Load Node-API [node_api_symbol_for] from host runtime failed: GetProcAddress failed
Load Node-API [node_api_create_external_string_latin1] from host runtime failed: GetProcAddress failed

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src\lib.rs (target\debug\deps\napi_derive-44dc2927c7b568c9.exe)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src\lib.rs (target\debug\deps\napi_derive_backend-2cb26fecfc5f6046.exe)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src\lib.rs (target\debug\deps\napi_examples-9d28c32e18bf0c7e.exe)
Load Node-API [napi_get_last_error_info] from host runtime failed: GetProcAddress failed
Load Node-API [napi_get_uv_event_loop] from host runtime failed: GetProcAddress failed
Load Node-API [napi_fatal_exception] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_threadsafe_function] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_date] from host runtime failed: GetProcAddress failed
Load Node-API [napi_create_bigint_int64] from host runtime failed: GetProcAddress failed
Load Node-API [napi_detach_arraybuffer] from host runtime failed: GetProcAddress failed
Load Node-API [napi_add_async_cleanup_hook] from host runtime failed: GetProcAddress failed
Load Node-API [node_api_symbol_for] from host runtime failed: GetProcAddress failed
Load Node-API [node_api_create_external_string_latin1] from host runtime failed: GetProcAddress failed

So, it builds fine, it gives you a hello world, but not sure what is exactly wrong here.

@raedrizqie
Copy link
Contributor

because it donwloads the prebuilt msvc-based rollup modules rollup.win32-x64-msvc.node:

npm http fetch GET 200 https://registry.npmjs.org/@rollup/rollup-win32-x64-msvc/-/rollup-win32-x64-msvc-4.13.0.tgz 1634ms (cache miss)

we need to build rollup.win32-x64-msvc.node from source

@Kreijstal
Copy link
Contributor Author

Kreijstal commented Mar 23, 2024

because it donwloads the prebuilt msvc-based rollup modules rollup.win32-x64-msvc.node:

npm http fetch GET 200 https://registry.npmjs.org/@rollup/rollup-win32-x64-msvc/-/rollup-win32-x64-msvc-4.13.0.tgz 1634ms (cache miss)

we need to build rollup.win32-x64-msvc.node from source

yeah, but for that we need to fix upstream dependencies, like napi-rs

@Kreijstal
Copy link
Contributor Author

just created this small check to check if nodejs can't be loaded but it can..

#include <stdio.h>
#include <windows.h>
#include <string.h>
#include <node/node_api.h>
int main() {
    // Print the MSYSTEM_PREFIX environment variable
    char* msystemPrefix = getenv("MSYSTEM_PREFIX");
    if (msystemPrefix == NULL) {
        printf("MSYSTEM_PREFIX environment variable is not set\n");
        return 1;
    }
    printf("MSYSTEM_PREFIX = %s\n", msystemPrefix);

    // Construct the path to libnode.dll
    char libNodePath[MAX_PATH];
    snprintf(libNodePath, sizeof(libNodePath), "%s\\bin\\libnode.dll", msystemPrefix);

    // Load the DLL
    HMODULE libNode = LoadLibrary(libNodePath);
    if (libNode == NULL) {
        printf("Failed to load %s\n", libNodePath);
        printf("GetLastError() = %lu\n", GetLastError());
        return 1;
    }
    printf("Successfully loaded %s\n", libNodePath);

    // Get the function pointer
    FARPROC napi_get_last_error_info = GetProcAddress(libNode, "napi_get_last_error_info");
    if (napi_get_last_error_info == NULL) {
        printf("Failed to get address of napi_get_last_error_info\n");
        printf("GetLastError() = %lu\n", GetLastError());
        FreeLibrary(libNode);
        return 1;
    }
    printf("Successfully retrieved address of napi_get_last_error_info\n");

    // Write success message to stdout
    printf("Successfully retrieved address of napi_get_last_error_info\n");

    // Free the library
    BOOL freeResult = FreeLibrary(libNode);
    if (!freeResult) {
        printf("Failed to free %s\n", libNodePath);
        printf("GetLastError() = %lu\n", GetLastError());
    } else {
        printf("Successfully freed %s\n", libNodePath);
    }

    return 0;
}

returns

MSYSTEM_PREFIX = C:/Users/topkek/scoop/apps/msys2/2024-01-13/ucrt64
Successfully loaded C:/Users/topkek/scoop/apps/msys2/2024-01-13/ucrt64\bin\libnode.dll
Successfully retrieved address of napi_get_last_error_info
Successfully retrieved address of napi_get_last_error_info
Successfully freed C:/Users/topkek/scoop/apps/msys2/2024-01-13/ucrt64\bin\libnode.dll

@Kreijstal
Copy link
Contributor Author

Kreijstal commented Apr 2, 2024

I managed to reduce the bug surface a bit in case anyone is good with rust

#!/bin/bash

# Clone the napi-rs repository with a depth of 1
git clone --depth 1 https://github.com/napi-rs/napi-rs.git

# Create a sibling directory for the Cargo project
mkdir -p sibling/src

# Change to the sibling directory
cd sibling

# Update the Cargo.toml file to include napi-rs as a dependency
echo '[package]
name = "sibling"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]

[dependencies]
napi = { path = "../napi-rs/crates/napi", default-features = false, features = ["napi4"] }' > Cargo.toml

# Create a minimal src/lib.rs file
echo 'use napi::bindgen_prelude::*;' > src/lib.rs

# Build the project
cargo build

# Copy the generated DLL and rename it with a .node extension
cp target/debug/sibling.dll sibling.node

# Run the Node.js module
node sibling.node

The problem is libloading, it doesn't seem to want to load the functions but I'm not 100% sure either how Libloading is supposed to work, in case anyone is curious about how this magic works, Bindgen prelude exposes a dll function napi_register_module_v1 that node calls while registering the module, however some code is executed before it using the [ctor] attribute from the ctor create, all functions from this crate execute when the dll is loaded alledgedly

@Kreijstal
Copy link
Contributor Author

I am puzzled trying to understand how the rust version works, it attempts to cast GetProcAddress so here I try to recreate it.

#!/bin/bash

# Create the hello.c file
cat > hello.c << EOL
#include <assert.h>
#include <node/node_api.h>

napi_value Method(napi_env env, napi_callback_info info) {
    napi_status status;
    napi_value world;
    status = napi_create_string_utf8(env, "world", NAPI_AUTO_LENGTH, &world);
    assert(status == napi_ok);
    return world;
}

#define DECLARE_NAPI_METHOD(name, func)                                        \\
    { name, 0, func, 0, 0, 0, napi_default, 0 }

napi_value napi_register_module_v1(napi_env env, napi_value exports) {
    napi_status status;
    napi_property_descriptor desc = DECLARE_NAPI_METHOD("hello", Method);
    status = napi_define_properties(env, exports, 1, &desc);
    assert(status == napi_ok);
    return exports;
}
EOL

# Compile the hello.c file
gcc -shared -fPIC hello.c -lnode -o hello.node

# Run the compiled module with Node.js
echo "console.log(require('./hello.node').hello())" | node -

this works
so then I try

gendef $MSYSTEM_PREFIX/bin/libnode.dll
dlltool -d libnode.def -l libnode.lib
gcc -shared -fPIC hello.c libnode.lib -o hello.node

apparently this does runtime linking but not in the way I wanted, this still works, but does not help me understand why they try to call GetProcAddress

@Kreijstal
Copy link
Contributor Author

Kreijstal commented Apr 4, 2024

so I recreated the toy example using GetProcAddress, and in mingw it still works, what I do not see is how they are loading the dll.

#include <assert.h>
#include <node/node_api.h>
#include <windows.h>
#include <stdio.h>

HMODULE nodeModule = NULL;
typedef napi_status (*napi_create_string_utf8_func)(napi_env env,
                                                    const char* str,
                                                    size_t length,
                                                    napi_value* result);
typedef napi_status (*napi_define_properties_func)(napi_env env,
                                                   napi_value object,
                                                   size_t property_count,
                                                   const napi_property_descriptor* properties);
napi_create_string_utf8_func napi_create_string_utf8dyn =NULL;
napi_define_properties_func napi_define_propertiesdyn =NULL;
bool LoadNodeFunctions() {
    nodeModule = LoadLibraryA("libnode.dll");
    if (!nodeModule) {
        printf("Failed to load Node.js DLL\n");
        return false;

   }
     napi_create_string_utf8dyn = (napi_create_string_utf8_func)GetProcAddress(nodeModule, "napi_create_string_utf8");
     napi_define_propertiesdyn = (napi_define_properties_func)GetProcAddress(nodeModule, "napi_define_properties");

    if (!napi_create_string_utf8 || !napi_define_properties) {
        printf("Failed to get function addresses\n");
        FreeLibrary(nodeModule);
        return false;
    }


    return true;
}

napi_value Method(napi_env env, napi_callback_info info) {
    napi_status status;
    napi_value world;
    status = napi_create_string_utf8dyn(env, "world", NAPI_AUTO_LENGTH, &world);
    assert(status == napi_ok);

    return world;
}

#define DECLARE_NAPI_METHOD(name, func)                                        \
    { name, 0, func, 0, 0, 0, napi_default, 0 }

napi_value napi_register_module_v1(napi_env env, napi_value exports) {
    if (!nodeModule) {
        if (!LoadNodeFunctions()) {
            return exports;
        }
    }
    napi_status status;
    napi_property_descriptor desc = DECLARE_NAPI_METHOD("hello", Method);
    status = napi_define_propertiesdyn(env, exports, 1, &desc);
    assert(status == napi_ok);

    return exports;
}

This will compile without linking to libnode.dll explizitly, but it links on runtime, (I imagine they do that so it works on many nodejs versions)

@Kreijstal
Copy link
Contributor Author

someone might have a similar issue here nodejs/node-gyp#2834

@Kreijstal
Copy link
Contributor Author

Kreijstal commented Apr 4, 2024

and I manage to reproduce the mistake of napi-rs

#include <assert.h>
#include <node/node_api.h>
#include <windows.h>
#include <stdio.h>
#include <tlhelp32.h>

#include <psapi.h>

void printMyProcess() {
    // Get the current process ID
    DWORD processId = GetCurrentProcessId();

    // Get a handle to the current process
    HANDLE hProcess = OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_VM_READ, FALSE, processId);
    if (hProcess == NULL) {
        printf("Failed to open process\n");
        return;
    }

    // Get the process name
    char processName[MAX_PATH];
    if (!GetModuleFileNameEx(hProcess, NULL, processName, MAX_PATH)) {
        printf("Failed to get process name\n");
        CloseHandle(hProcess);
        return;
    }

    // Print the process name
    printf("Current process name: %s\n", processName);

    // Close the handle to the process
    CloseHandle(hProcess);
}
typedef enum {
    GET_MODULE_HANDLE_EX_W_UNKNOWN,
    GET_MODULE_HANDLE_EX_W_ERROR
} GetModuleHandleExWError;

GetModuleHandleExWError GetModuleHandleExWErrorFromLastError() {
    DWORD lastError = GetLastError();
    if (lastError != 0) {
        return GET_MODULE_HANDLE_EX_W_ERROR;
    }
    return GET_MODULE_HANDLE_EX_W_UNKNOWN;
}

HMODULE GetCurrentModuleHandle() {
    HMODULE handle = NULL;
    BOOL result = GetModuleHandleExW(0,NULL,
        &handle
    );

    if (result == 0) {
        GetModuleHandleExWError error = GetModuleHandleExWErrorFromLastError();
        if (error == GET_MODULE_HANDLE_EX_W_ERROR) {
            printf("GetModuleHandleExW error: %lu\n", GetLastError());
        } else {
            printf("Unknown GetModuleHandleExW error\n");
        }
        return NULL;
    }

    return handle;
}
HMODULE nodeModule = NULL;
typedef napi_status (*napi_create_string_utf8_func)(napi_env env,
                                                    const char* str,
                                                    size_t length,
                                                    napi_value* result);
typedef napi_status (*napi_define_properties_func)(napi_env env,
                                                   napi_value object,
                                                   size_t property_count,
                                                   const napi_property_descriptor* properties);
napi_create_string_utf8_func napi_create_string_utf8dyn =NULL;
napi_define_properties_func napi_define_propertiesdyn =NULL;
bool LoadNodeFunctions() {
#ifdef LOAD_FROM_PROCESS
    nodeModule = GetCurrentModuleHandle();
#else
    nodeModule = LoadLibraryA("libnode.dll");
#endif
    if (!nodeModule) {
        printf("Failed to load Node.js DLL\n");
        return false;
    }

    napi_create_string_utf8dyn = (napi_create_string_utf8_func)GetProcAddress(nodeModule, "napi_create_string_utf8");
    napi_define_propertiesdyn = (napi_define_properties_func)GetProcAddress(nodeModule, "napi_define_properties");

    if (!napi_create_string_utf8dyn || !napi_define_propertiesdyn) {
        printf("Failed to get function addresses\n");
#ifndef LOAD_FROM_PROCESS
        FreeLibrary(nodeModule);
#endif
        return false;
    }

    return true;
}

napi_value Method(napi_env env, napi_callback_info info) {
    napi_status status;
    napi_value world;
    status = napi_create_string_utf8dyn(env, "world", NAPI_AUTO_LENGTH, &world);
    assert(status == napi_ok);

    return world;
}

#define DECLARE_NAPI_METHOD(name, func)                                        \
    { name, 0, func, 0, 0, 0, napi_default, 0 }

napi_value napi_register_module_v1(napi_env env, napi_value exports) {
    printMyProcess();
    if (!nodeModule) {
        if (!LoadNodeFunctions()) {
            return exports;
        }
    }
    napi_status status;
    napi_property_descriptor desc = DECLARE_NAPI_METHOD("hello", Method);
    status = napi_define_propertiesdyn(env, exports, 1, &desc);
    assert(status == napi_ok);

    return exports;
}

compile as
gcc -DLOAD_FROM_PROCESS -shared -fPIC hello.c -o hello.node
and you get Failed to get function addresses which makes sense

@Kreijstal
Copy link
Contributor Author

Kreijstal commented Apr 4, 2024

Just noticed that other people solve this problem by just statically linking

https://users.rust-lang.org/t/why-is-msvc-the-default-toolchain/44287/7

I guess we can do that, it is much faster as well..

@Kreijstal
Copy link
Contributor Author

patching upstream napi-rs/napi-rs#2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants