Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed to compile on windows with ssl-vendored & cmake-build combination #572

Open
peter-formlogic opened this issue Apr 28, 2023 · 8 comments · May be fixed by #573
Open

Failed to compile on windows with ssl-vendored & cmake-build combination #572

peter-formlogic opened this issue Apr 28, 2023 · 8 comments · May be fixed by #573

Comments

@peter-formlogic
Copy link

Hey, I can't seem to get this to link on windows with a vendored ssl.

I.e, using:

rdkafka = { version = "0.29.0", features = ["ssl-vendored", "cmake-build"] }

I get the following output:

          librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_load_ssl_client_cert referenced in function rd_kafka_ssl_set_certs
          librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_free referenced in function rd_kafka_ssl_ctx_init
          librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_by_id referenced in function rd_kafka_ssl_ctx_init
          librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_ctrl_cmd_string referenced in function rd_kafka_ssl_ctx_init
          librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_init referenced in function rd_kafka_ssl_ctx_init

I am also seeing a lot of these errors:

libopenssl_sys-ba70b1b6d496cd9b.rlib(bio_b64.obj) : warning LNK4099: PDB 'ossl_static.pdb' was not found with 'libopenssl_sys-ba70b1b6d496cd9b.rlib(bio_b64.obj)' or at '..\target\debug\deps\ossl_static.pdb'; linking object as if no debug info

But I can confirm that ossl_static.pdb is in the debug\build\openssl-sys-63196b440406ab23\out\openssl-build\install\lib directory alongside the two lib files:

libcrypto.lib  libssl.lib  ossl_static.pdb

The link.exe statement does include that lib directory:

"/LIBPATH:<snipped>\\target\\debug\\build\\openssl-sys-63196b440406ab23\\out\\openssl-build\\install\\lib"
@WaterKnight1998
Copy link

I am fighting with the same on Linux

@peter-formlogic
Copy link
Author

peter-formlogic commented May 1, 2023 •
edited

Ok, I've investigated this more and it appears to be because openssl-src does not include the engine module by default on windows (& musl) targets: https://github.com/alexcrichton/openssl-src-rs/blob/97babda37a440e13a1492e95af25c7fbd33c20fe/src/lib.rs#L193-L201

So when it compiles the static version of openssl, it does so without the appropriate exports, hence why it's complaining.

If you add the following to Cargo.toml it seems to resolve it (note: it needs to be under build-dependencies), but I think we should include this in the rdkafka library when vendoring and so I'll raise a PR:

[build-dependencies]
openssl-src = { version = "111", features = ["force-engine"] }

@peter-formlogic peter-formlogic linked a pull request May 1, 2023 that will close this issue
Open
@WaterKnight1998
Copy link

@peter-formlogic

Cargo.toml is like follows but I am still getting an issue

rdkafka = { version = "0.29.0", features = ["cmake-build", "ssl-vendored"], default-features = false }

[build-dependencies]
openssl-src = { version = "111.25.3+1.1.1t", features = [ "force-engine" ] }

Error:

 engines/e_afalg.c:24:10: fatal error: linux/version.h: No such file or directory
     24 | #include <linux/version.h>
        |          ^~~~~~~~~~~~~~~~~
  compilation terminated.
  make[1]: *** [Makefile:5752: engines/e_afalg.o] Error 1
  make[1]: *** Waiting for unfinished jobs....
  make: *** [Makefile:177: build_libs] Error 2
  thread 'main' panicked at '

@peter-formlogic
Copy link
Author

Sounds like there are no linux headers installed wherever you're trying to run this.

You could try adding linux-headers-generic as a package on ubuntu and kernel-headers-musl on arch.

Given Cargo.toml:

[package]
name = "rdtest"
version = "0.1.0"
edition = "2021"

[dependencies]
rdkafka = { version = "0.29.0", features = ["cmake-build", "ssl-vendored"]}

[build-dependencies]
openssl-src = { version = "111", features = [ "force-engine" ] }

and main.rs:

use rdkafka::Timestamp;

fn main() {
    let timestamp = Timestamp::now();
    println!("{timestamp:?}");
}

The following docker file compiles & runs just fine:

FROM clux/muslrust

RUN ln -s /bin/g++ /bin/musl-g++

ADD Cargo.toml .
ADD src src

RUN cargo build --release

CMD ["target/x86_64-unknown-linux-musl/release/rdtest"]

@WaterKnight1998
Copy link

Morning, after installing kernel-headers-musl on arch now I get the following error: note: rust-lld: error: unable to find library -lresolv

@WaterKnight1998
Copy link

Removing "gssapi-vendored", "libz-static"], default-features = false did the trick, thanks so much :)

@WaterKnight1998
Copy link

However, in Ubuntu the package you mention doesn't solve the error

@zzl221000 zzl221000 mentioned this issue Jun 1, 2023
Merged
@Orca-bit
Copy link

Orca-bit commented Feb 7, 2024

in openssl-sys, they add this:
https://github.com/sfackler/rust-openssl/blob/745b200425390ab2b0b5aa641a89237b2ff7426e/openssl-sys/Cargo.toml#L28

openssl-src = { version = "300.1.2", optional = true, features = ["legacy"] }

if you add

openssl-src = { version = "300",  features = ["force-engine"] }

these configurations may conflict

so I add this:

[build-dependencies]
openssl-src = { version = "300", features = ["legacy", "force-engine"] }

it works

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Ensure force-engine is set when building vendored openssl peter-formlogic/rust-rdkafka
3 participants
@WaterKnight1998 @Orca-bit @peter-formlogic