Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Static (fully static build) is broken on Linux #2734

Open
myrrc opened this issue Oct 8, 2023 · 2 comments
Open

Static (fully static build) is broken on Linux #2734

myrrc opened this issue Oct 8, 2023 · 2 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@myrrc
Copy link

myrrc commented Oct 8, 2023

First things first, I don't think it's a particular issue of this repository, just the problems of Cmake.
Nevertheless, I need a fully static build of some application using notcurses (notcurses++), which means all libraries that are used (including transitive dependencies) should be packed in a single binary.

Ok, so cmake v1 (notcurses/ repo downloaded as a submodule, built and installed to system)

set(BUILD_EXECUTABLES OFF)
set(BUILD_DOCTEST OFF)
set(USE_PANDOC OFF)
set(BUILD_TESTING OFF)
set(BUILD_FFI_LIBRARY OFF)
add_subdirectory(notcurses)

file(GLOB SRC src/*.cpp)
add_executable(thegg ${SRC})
target_link_libraries(thegg PRIVATE notcurses++-static)

This doesn't work (missing references to notcurses). Ok, patching notcurses so that notcurses++-static links to notcurses-static

This works... and builds a binary with all shared libraries because cmake is cursed and prefers shared libraries to static ones.

Ok, adding explicit magic

set(BUILD_SHARED_LIBS OFF)
set(CMAKE_EXE_LINKER_FLAGS "-static")

This doesn't work (see above) and breaks on linking notcurses (the default, shared one).
Usually library authors have to make a separate option USE_SHARED which turns off building shared libraries. Ok, patching notcurses cmake for this option.

This still doesn't work as cmake can't find paths to dependencies (unistring, tinfo), though .a files are in the same folder as .so ones. Ok, patching dependencies, removing things like ${deflate_STATIC_LIBRARIES} with explicit paths like /usr/lib/linux_x86_64/libdeflate.a... and this works, but only regarding 2 corner cases

  1. We can't use notcurses_accountname() as it relies on getpwid() which in turn can't be linked statically as invoking it on some machine where glibc version is different will break everything (fine, I don't care)
  2. Multimedia support must be turned off

Ok, we want multimedia support. We have either ffmpeg or OpenImageIO (which in turn has dependencies for ffmpeg's libavcodec, anyway).

If we use the defaut option, with -static we get a list of around 70 libraries we need to link with (including rabbitmq). As this is a dead way, going to the second option.

OpenimageIO (apt install libopenimageio2.2 and dev headers) by default comes only in a shared library. Ok, cloning it to a separate way, setting flags for static build
(make -j BUILD_SHARED_LIBS=0 LINKSTATIC=1 USE_PYTHON=0 OIIO_BUILD_TESTS=0 OIIO_BUILD_TOOLS=0), building, and we get two .a files. Patching notcurses cmake with explicit paths for these two...

And getting messages like

notcurses/libnotcurses.a(oiio.cpp.o): in function `oiio_details_destroy(ncvisual_details*)':
oiio.cpp:(.text+0x42): undefined reference to `OpenImageIO_v2_2::ImageBuf::~ImageBuf()'

though there are explicit link references like

target_link_libraries(notcurses-static PRIVATE
  /home/myrrc/thegg/OpenImageIO/build/lib/libOpenImageIO.a
  )

and nm libOpenImageIO.a confirms they're all there.

Actually, there is another way -- download OpenimageIO static build from vcpkg, but I thought adding a package manager to your library would be a bit over the edge.

Maybe there's some simple way I missed.
Unfortunately, having a shared build is impossible as target users surely won't like to download a ton of dependencies including ffmpeg.

Anyway, thanks for the library!

@myrrc myrrc added the bug Something isn't working label Oct 8, 2023
@dankamongmen dankamongmen added this to the 3.1.0 milestone Oct 9, 2023
@dankamongmen dankamongmen self-assigned this Oct 9, 2023
@dankamongmen
Copy link
Owner

well, we can't have that.

i'll try to look into this asap. thanks for the great writeup!

@myrrc
Copy link
Author

myrrc commented Oct 9, 2023

Feel free to ask for clarification.
This is my patch for core Cmakelists

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 5d6c142..de21fc3 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -28,6 +28,7 @@ option(BUILD_FFI_LIBRARY "Build ffi library (containing all symbols which are st
 option(USE_POC "Build small, uninstalled proof-of-concept binaries" ON)
 option(USE_QRCODEGEN "Enable libqrcodegen QR code support" OFF)
 option(USE_STATIC "Build static libraries (in addition to shared)" ON)
+option(USE_SHARED "" ON)
 set(USE_MULTIMEDIA "ffmpeg" CACHE STRING "Multimedia engine, one of 'ffmpeg', 'oiio', or 'none'")
 set_property(CACHE USE_MULTIMEDIA PROPERTY STRINGS ffmpeg oiio none)
 if(NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
@@ -171,7 +172,7 @@ check_include_file("unigbrk.h" HAVE_UNISTRING_H)
 if(NOT "${HAVE_UNISTRING_H}")
   message(FATAL_ERROR "Couldn't find unigbrk.h from GNU libunistring")
 endif()
-find_library(unistring unistring REQUIRED)
+find_library(unistring libunistring.a REQUIRED)
 set_property(GLOBAL APPEND PROPERTY PACKAGES_FOUND libunistring)
 set_package_properties(libunistring PROPERTIES TYPE REQUIRED)
 
@@ -182,7 +183,7 @@ check_include_file("libdeflate.h" HAVE_DEFLATE_H)
 if(NOT "${HAVE_DEFLATE_H}")
   message(FATAL_ERROR "Couldn't find libdeflate.h")
 endif()
-find_library(libdeflate deflate REQUIRED)
+find_library(libdeflate.a deflate REQUIRED)
 set_property(GLOBAL APPEND PROPERTY PACKAGES_FOUND libdeflate)
 set_package_properties(libdeflate PROPERTIES TYPE REQUIRED)
 else()
@@ -217,20 +218,23 @@ file(GLOB COMPATSRC CONFIGURE_DEPENDS src/compat/*.c)
 ############################################################################
 # libnotcurses-core (core shared library, core static library)
 file(GLOB NCCORESRCS CONFIGURE_DEPENDS src/lib/*.c)
+
+if(${USE_SHARED})
 add_library(notcurses-core SHARED ${NCCORESRCS} ${COMPATSRC})
-if(${USE_STATIC})
-add_library(notcurses-core-static STATIC ${NCCORESRCS} ${COMPATSRC})
-else()
-add_library(notcurses-core-static STATIC EXCLUDE_FROM_ALL ${NCCORESRCS} ${COMPATSRC})
 endif()
+
+add_library(notcurses-core-static STATIC ${NCCORESRCS} ${COMPATSRC})
+
 # don't want these on freebsd/dragonfly/osx
 if(${CMAKE_SYSTEM_NAME} STREQUAL "Linux")
+if(${USE_SHARED})
 target_compile_definitions(notcurses-core
   PUBLIC
    _XOPEN_SOURCE=700 # wcwidth(3) requires _XOPEN_SOURCE, and is in our headers
   PRIVATE
    _GNU_SOURCE _DEFAULT_SOURCE
 )
+endif()
 target_compile_definitions(notcurses-core-static
   PUBLIC
    _XOPEN_SOURCE=700 # wcwidth(3) requires _XOPEN_SOURCE, and is in our headers
@@ -238,14 +242,12 @@ target_compile_definitions(notcurses-core-static
    _GNU_SOURCE _DEFAULT_SOURCE
 )
 endif()
+
+if(${USE_SHARED})
 set_target_properties(notcurses-core PROPERTIES
   VERSION ${PROJECT_VERSION}
   SOVERSION ${PROJECT_VERSION_MAJOR}
 )
-set_target_properties(notcurses-core-static PROPERTIES
-  VERSION ${PROJECT_VERSION}
-  OUTPUT_NAME notcurses-core
-)
 target_include_directories(notcurses-core
   BEFORE
   PRIVATE
@@ -257,17 +259,6 @@ target_include_directories(notcurses-core
     "${libdeflate_INCLUDE_DIRS}"
     "${ZLIB_INCLUDE_DIRS}"
 )
-target_include_directories(notcurses-core-static
-  BEFORE
-  PRIVATE
-    include
-    src
-    "${CMAKE_REQUIRED_INCLUDES}"
-    "${PROJECT_BINARY_DIR}/include"
-    "${TERMINFO_STATIC_INCLUDE_DIRS}"
-    "${libdeflate_STATIC_INCLUDE_DIRS}"
-    "${ZLIB_STATIC_INCLUDE_DIRS}"
-)
 target_link_libraries(notcurses-core
   PRIVATE
     "${libdeflate}"
@@ -280,8 +271,34 @@ target_link_libraries(notcurses-core
     Threads::Threads
     "${LIBRT}"
 )
+target_link_directories(notcurses-core
+  PRIVATE
+    "${TERMINFO_LIBRARY_DIRS}"
+    "${libdeflate_LIBRARY_DIRS}"
+    "${ZLIB_LIBRARY_DIRS}"
+)
+
+endif()
+
+set_target_properties(notcurses-core-static PROPERTIES
+  VERSION ${PROJECT_VERSION}
+  OUTPUT_NAME notcurses-core
+)
+target_include_directories(notcurses-core-static
+  BEFORE
+  PRIVATE
+    include
+    src
+    "${CMAKE_REQUIRED_INCLUDES}"
+    "${PROJECT_BINARY_DIR}/include"
+    "${TERMINFO_STATIC_INCLUDE_DIRS}"
+    "${libdeflate_STATIC_INCLUDE_DIRS}"
+    "${ZLIB_STATIC_INCLUDE_DIRS}"
+)
+
 target_link_libraries(notcurses-core-static
   PRIVATE
+    deflate
     "${libdeflate_STATIC_LIBRARIES}"
     "${ZLIB_STATIC_LIBRARIES}"
     "${TERMINFO_STATIC_LIBRARIES}"
@@ -291,12 +308,6 @@ target_link_libraries(notcurses-core-static
     Threads::Threads
     "${LIBRT}"
 )
-target_link_directories(notcurses-core
-  PRIVATE
-    "${TERMINFO_LIBRARY_DIRS}"
-    "${libdeflate_LIBRARY_DIRS}"
-    "${ZLIB_LIBRARY_DIRS}"
-)
 target_link_directories(notcurses-core-static
   PRIVATE
     "${TERMINFO_STATIC_LIBRARY_DIRS}"
@@ -411,7 +422,9 @@ target_link_directories(notcurses-static
 elseif(${USE_OIIO})
 target_include_directories(notcurses PUBLIC "${OIIO_INCLUDE_DIRS}")
 target_include_directories(notcurses-static PUBLIC "${OIIO_STATIC_INCLUDE_DIRS}")
-target_link_libraries(notcurses PRIVATE OpenImageIO)
+target_link_libraries(notcurses-static PRIVATE
+  /home/myrrc/thegg/OpenImageIO/build/lib/libOpenImageIO.a
+  )
 target_link_libraries(notcurses-static PRIVATE ${OIIO_STATIC_LIBRARIES})
 target_link_directories(notcurses PRIVATE ${OIIO_LIBRARY_DIRS})
 target_link_directories(notcurses-static PRIVATE ${OIIO_STATIC_LIBRARY_DIRS})
@@ -527,6 +540,13 @@ target_link_libraries(notcurses++
   PUBLIC
   notcurses)
 
+target_link_libraries(notcurses++-static
+  PRIVATE
+  deflate
+  /home/myrrc/thegg/OpenImageIO/build/lib/libOpenImageIO_Util.a
+  /home/myrrc/thegg/OpenImageIO/build/lib/libOpenImageIO.a
+  notcurses-static)
+
 set(NCPP_COMPILE_OPTIONS
   -Wnull-dereference
   -Wunused
@@ -546,10 +566,13 @@ target_compile_options(notcurses++
   -fPIC
   )
 
+
+if(${USE_SHARED})
 target_compile_options(notcurses-core
   PRIVATE
     -fPIC
 )
+endif()
 
 target_compile_options(notcurses-core-static
   PRIVATE
@@ -1133,7 +1156,10 @@ endif() # BUILD_EXECUTABLES
 if(${BUILD_FFI_LIBRARY})
 LIST(APPEND INSTLIBS notcurses-ffi)
 endif()
+
+if(${USE_SHARED})
 LIST(APPEND INSTLIBS notcurses-core notcurses)
+endif()
 if(${USE_STATIC})
 LIST(APPEND INSTLIBS notcurses-core-static notcurses-static)
 endif()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants