8000 Simplify third-party integration by upsj · Pull Request #786 · ginkgo-project/ginkgo · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Simplify third-party integration #786

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jun 11, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 13 additions & 16 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ include(cmake/hip_path.cmake)
include(cmake/autodetect_executors.cmake)
include(cmake/build_type_helpers.cmake)

# Load other CMake helpers
include(cmake/build_helpers.cmake)
include(cmake/install_helpers.cmake)

if (MSVC)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /bigobj")
endif()
Expand Down Expand Up @@ -178,11 +182,8 @@ if(GINKGO_BUILD_HWLOC AND (MSVC OR WIN32 OR CYGWIN OR APPLE))
message(WARNING "Ginkgo does not support HWLOC on Windows/MacOS, switch GINKGO_BUILD_HWLOC to OFF")
endif()
if(GINKGO_BUILD_HWLOC)
# By default always use external HWLOC
set(GINKGO_USE_EXTERNAL_HWLOC 1)
set(GINKGO_HAVE_HWLOC 1)
else()
set(GINKGO_USE_EXTERNAL_HWLOC 0)
set(GINKGO_HAVE_HWLOC 0)
message(STATUS "HWLOC is being forcibly switched off")
endif()
Expand Down Expand Up @@ -216,22 +217,18 @@ endif()

# Try to find the third party packages before using our subdirectories
include(cmake/package_helpers.cmake)
ginkgo_find_package(GTest "GTest::GTest;GTest::Main" FALSE 1.8.1)
ginkgo_find_package(gflags gflags FALSE 2.2.2)
ginkgo_find_package(RapidJSON rapidjson TRUE 1.1.0)
if(GINKGO_HAVE_HWLOC)
ginkgo_find_package(HWLOC hwloc FALSE 2.1)
if (NOT HWLOC_FOUND)
set(GINKGO_USE_EXTERNAL_HWLOC 0)
endif()
if(GINKGO_BUILD_TESTS)
find_package(GTest 1.10.0) # No need for QUIET as CMake ships FindGTest
endif()
if(GINKGO_BUILD_BENCHMARKS)
find_package(gflags 2.2.2 QUIET)
find_package(RapidJSON 1.1.0 QUIET)
endif()
if(GINKGO_BUILD_HWLOC)
find_package(HWLOC 2.1) # No need for QUIET as we ship FindHWLOC
endif()
# third_party needs to be after flag modification.
add_subdirectory(third_party) # Third-party tools and libraries

# Load CMake helpers
include(cmake/build_helpers.cmake)
include(cmake/install_helpers.cmake)

if(MSVC)
if(BUILD_SHARED_LIBS)
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS TRUE)
Expand Down
30 changes: 6 additions & 24 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,9 +212,6 @@ variables:
Ginkgo relies on third party packages in different cases. These third party
packages can be turned off by disabling the relevant options.

+ GINKGO_BUILD_CUDA=ON:
[CudaArchitectureSelector](https://github.com/ginkgo-project/CudaArchitectureSelector)
(CAS) is a CMake helper to manage CUDA architecture settings;
+ GINKGO_BUILD_TESTS=ON: Our tests are implemented with [Google
Test](https://github.com/google/googletest);
+ GINKGO_BUILD_BENCHMARKS=ON: For argument management we use
Expand All @@ -227,14 +224,10 @@ packages can be turned off by disabling the relevant options.
[hwloc](https://www.open-mpi.org/projects/hwloc) to detect and control cores
and devices.

Usually, Ginkgo uses the internal version of each package. For each of the
packages `GTEST`, `GFLAGS`, `RAPIDJSON` and `CAS`, it is possible to force
Ginkgo to try to use an external version of a package. For this, Ginkgo provides
two ways to find packages. To rely on the CMake `find_package` command, use the
CMake option `-DGINKGO_USE_EXTERNAL_<package>=ON`. `HWLOC` works the opposite
way, Ginkgo always looks for the system's `hwloc` first. In addition, when
installing Ginkgo, `hwloc` will be installed as well into the Ginkgo directory
if there were no system `hwloc` detected.
Ginkgo attempts to use pre-installed versions of these package if they match
version requirements using `find_package`. Otherwise, the configuration step
will download the files for each of the packages `GTest`, `gflags`,
`RapidJSON` and `hwloc` and build them internally.

Note that, if the external packages were not installed to the default location,
the CMake option `-DCMAKE_PREFIX_PATH=<path-list>` needs to be set to the
Expand All @@ -243,19 +236,8 @@ more Information, see the [CMake documentation for
CMAKE_PREFIX_PATH](https://cmake.org/cmake/help/v3.9/variable/CMAKE_PREFIX_PATH.html)
for details.

To manually configure the paths, Ginkgo relies on the [standard xSDK Installation
policies](https://xsdk.info/policies/) for all packages except `CAS` (as it is
neither a library nor a header, it cannot be expressed through the `TPL`
format):
+ `-DTPL_ENABLE_<package>=ON`
+ `-DTPL_<package>_LIBRARIES=/path/to/libraries.{so|a}`
+ `-DTPL_<package>_INCLUDE_DIRS=/path/to/header/directory`

When applicable (e.g. for `GTest` libraries), a `;` separated list can be given
to the `TPL_<package>_{LIBRARIES|INCLUDE_DIRS}` variables.

Note that for convenience, the options `GINKGO_INSTALL_RPATH[_.*]` can be used
to make bind the Ginkgo shared libraries to the path of its dependencies.
For convenience, the options `GINKGO_INSTALL_RPATH[_.*]` can be used
to bind the installed Ginkgo shared libraries to the path of its dependencies.

### Installing Ginkgo

Expand Down
25 changes: 0 additions & 25 deletions cmake/DownloadCMakeLists.txt.in

This file was deleted.

17 changes: 0 additions & 17 deletions cmake/GinkgoConfig.cmake.in
Original file line number Diff line number Diff line change
Expand Up @@ -84,23 +84,6 @@ set(GINKGO_HAVE_PAPI_SDE @GINKGO_HAVE_PAPI_SDE@)
set(GINKGO_HAVE_HWLOC @GINKGO_HAVE_HWLOC@)
set(GINKGO_BUILD_HWLOC @GINKGO_BUILD_HWLOC@)

# Ginkgo external package variables
set(GINKGO_USE_EXTERNAL_CAS "@GINKGO_USE_EXTERNAL_CAS@")
set(GINKGO_USE_EXTERNAL_GTEST "@GINKGO_USE_EXTERNAL_GTEST@")
set(GINKGO_USE_EXTERNAL_GFLAGS "@GINKGO_USE_EXTERNAL_GFLAGS@")
set(GINKGO_USE_EXTERNAL_RAPIDJSON "@GINKGO_USE_EXTERNAL_RAPIDJSON@")
set(GINKGO_USE_EXTERNAL_HWLOC "@GINKGO_USE_EXTERNAL_HWLOC@")
Comment on lines -89 to -92
Copy link
Member
@tcojean tcojean Jun 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having this also helped debugging in case something failed with an external package, knowing it's external would help pinpoint the problem. With the current setup, I'm not sure if we would have a (simple) way to keep something like that. Maybe storing the actual version?


set(TPL_ENABLE_GTEST "@TPL_ENABLE_GTEST@")
set(TPL_GTEST_LIBRARIES "@TPL_GTEST_LIBRARIES@")
set(TPL_GTEST_INCLUDE_DIRS "@TPL_GTEST_INCLUDE_DIRS@")
set(TPL_ENABLE_GFLAGS "@TPL_ENABLE_GFLAGS@")
set(TPL_GFLAGS_LIBRARIES "@TPL_GFLAGS_LIBRARIES@")
set(TPL_GFLAGS_INCLUDE_DIRS "@TPL_GFLAGS_INCLUDE_DIRS@")
set(TPL_ENABLE_RAPIDJSON "@TPL_ENABLE_RAPIDJSON@")
set(TPL_RAPIDJSON_LIBRARIES "@TPL_RAPIDJSON_LIBRARIES@")
set(TPL_RAPIDJSON_INCLUDE_DIRS "@TPL_RAPIDJSON_INCLUDE_DIRS@")

# Ginkgo installation configuration
set(GINKGO_CONFIG_FILE_PATH "${CMAKE_CURRENT_LIST_DIR}")
string(REPLACE "@GINKGO_INSTALL_CONFIG_DIR@" "" GINKGO_INSTALL_PREFIX "${GINKGO_CONFIG_FILE_PATH}")
Expand Down
4 changes: 4 additions & 0 deletions cmake/create_test.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@ function(ginkgo_create_cuda_test test_name)
)
target_compile_options(${TEST_TARGET_NAME}
PRIVATE "$<$<COMPILE_LANGUAGE:CUDA>:${GINKGO_CUDA_ARCH_FLAGS}>")
# we handle CUDA architecture flags for now, disable CMake handling
if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.18)
set_target_properties(${TEST_TARGET_NAME} PROPERTIES CUDA_ARCHITECTURES OFF)
endif()
set_target_properties(${TEST_TARGET_NAME} PROPERTIES
OUTPUT_NAME ${test_name})
if (GINKGO_FAST_TESTS)
Expand Down
4 changes: 2 additions & 2 deletions cmake/install_helpers.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ function(ginkgo_add_install_rpath name)
endif()
if (GINKGO_INSTALL_RPATH_DEPENDENCIES)
set(RPATH_DEPENDENCIES "${ARGN}")
if (GINKGO_HAVE_HWLOC AND GINKGO_USE_EXTERNAL_HWLOC)
if(GINKGO_HAVE_HWLOC AND HWLOC_FOUND)
get_filename_component(HWLOC_LIB_PATH ${HWLOC_LIBRARIES} DIRECTORY)
list(APPEND RPATH_DEPENDENCIES "${HWLOC_LIBRARIES}")
endif()
Expand Down Expand Up @@ -87,7 +87,7 @@ function(ginkgo_install)
)
endif()

if (GINKGO_HAVE_HWLOC AND NOT GINKGO_USE_EXTERNAL_HWLOC)
if (GINKGO_HAVE_HWLOC AND NOT HWLOC_FOUND)
get_filename_component(HWLOC_LIB_PATH ${HWLOC_LIBRARIES} DIRECTORY)
file(GLOB HWLOC_LIBS "${HWLOC_LIB_PATH}/libhwloc*")
install(FILES ${HWLOC_LIBS}
Expand Down
196 changes: 0 additions & 196 deletions cmake/package_helpers.cmake
Original file line number Diff line number Diff line change
@@ -1,62 +1,7 @@
set(PACKAGE_DOWNLOADER_SCRIPT
"${CMAKE_CURRENT_LIST_DIR}/DownloadCMakeLists.txt.in")

set(NON_CMAKE_PACKAGE_DOWNLOADER_SCRIPT
"${CMAKE_CURRENT_LIST_DIR}/DownloadNonCMakeCMakeLists.txt.in")


# Load a package from its repository
#
# \param package_name Name of the package
# \param package_url Url of the package
# \param package_tag Tag or version of the package to be downloaded.
#
function(ginkgo_load_git_package package_name package_url package_tag)
set(GINKGO_THIRD_PARTY_BUILD_TYPE "Debug")
if (CMAKE_BUILD_TYPE MATCHES "[Rr][Ee][Ll][Ee][Aa][Ss][Ee]")
set(GINKGO_THIRD_PARTY_BUILD_TYPE "Release")
endif()
configure_file(${PACKAGE_DOWNLOADER_SCRIPT}
download/CMakeLists.txt)
set(TOOLSET "")
if (NOT "${CMAKE_GENERATOR_TOOLSET}" STREQUAL "")
set(TOOLSET "-T${CMAKE_GENERATOR_TOOLSET}")
endif()
execute_process(COMMAND ${CMAKE_COMMAND} -G "${CMAKE_GENERATOR}" "${TOOLSET}" .
RESULT_VARIABLE result
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/download)
if(result)
message(FATAL_ERROR
"CMake step for ${package_name}/download failed: ${result}")
endif()
if(MSVC)
# MSVC decides the build_type in build step not cmake step, so Ginkgo builds Debug and Release type.
execute_process(COMMAND ${CMAKE_COMMAND} --build . --config Debug
RESULT_VARIABLE result
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/download)
if(result)
message(FATAL_ERROR
"Build Debug step for ${package_name}/download failed: ${result}")
endif()
execute_process(COMMAND ${CMAKE_COMMAND} --build . --config Release
RESULT_VARIABLE result
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/download)
if(result)
message(FATAL_ERROR
"Build Release step for ${package_name}/download failed: ${result}")
endif()
else()
execute_process(COMMAND ${CMAKE_COMMAND} --build .
RESULT_VARIABLE result
WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/download)
if(result)
message(FATAL_ERROR
"Build step for ${package_name}/download failed: ${result}")
endif()
endif()
endfunction()


# Load a package from the url provided and run configure (Non-CMake projects)
#
# \param package_name Name of the package
Expand Down Expand Up @@ -94,147 +39,6 @@ function(ginkgo_load_and_configure_package package_name package_url package_hash
endfunction()


# Add external target to external project.
# Create a new target and declare it as `IMPORTED` for libraries or `INTERFACE`
# for header only projects.
#
# \param new_target New target for the external project
# \param external_name Name of the external project
# \param includedir Path to include directory
# \param libdir Path to library directory
# \param header_only Boolean indicating if this should be a header only target
#
macro(ginkgo_add_tpl_target new_target external_name includedir libdir header_only)
# Declare include directories and library files
set(${external_name}_INCLUDE_DIR "${includedir}")
set(${external_name}_LIBRARY "${libdir}")

# Create an IMPORTED external library available in the GLOBAL scope
if (${header_only})
add_library(${new_target} INTERFACE)
else()
add_library(${new_target} UNKNOWN IMPORTED GLOBAL)
endif()

# Set the target's properties, namely library file and include directory
if (NOT ${header_only})
foreach (lib in LISTS ${${external_name}_LIBRARY})
set_target_properties(${new_target} PROPERTIES IMPORTED_LOCATION ${lib})
endforeach()
endif()
foreach (inc in LISTS ${${external_name}_INCLUDE_DIR})
set_target_properties(${new_target} PROPERTIES INTERFACE_INCLUDE_DIRECTORIES ${inc})
endforeach()
endmacro(ginkgo_add_tpl_target)


# Add external target to external project.
# Create a new target and declare it as `IMPORTED` for libraries or `INTERFACE`
# for header only projects.
#
# \param new_target New target for the external project
# \param external_name Name of the external project
# \param includedir Path to include directory
# \param libdir Path to library directory
# \param build_type Build type {STATIC, SHARED}
# \param debug_postfix The debug postfix to use when building in debug mode
# \param external Name of the external target
# \param header_only Boolean indicating if this should be a header only target
#
macro(ginkgo_add_external_target new_target external_name includedir libdir build_type debug_postfix external header_only)
# Declare include directories and library files
set(${external_name}_BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR}/${libdir})
set(${external_name}_INCLUDE_DIR "${CMAKE_CURRENT_BINARY_DIR}/${includedir}")
if(MSVC)
# Ginkgo only builds Debug and Release, so set the path without CMAKE_CFG_INTDIR.
set(${external_name}_LIBRARY_RELEASE "${${external_name}_BINARY_DIR}/Release/${CMAKE_${build_type}_LIBRARY_PREFIX}${external_name}${CMAKE_${build_type}_LIBRARY_SUFFIX}")
set(${external_name}_LIBRARY_DEBUG "${${external_name}_BINARY_DIR}/Debug/${CMAKE_${build_type}_LIBRARY_PREFIX}${external_name}${debug_postfix}${CMAKE_${build_type}_LIBRARY_SUFFIX}")
else()
set(${external_name}_LIBRARY_RELEASE "${${external_name}_BINARY_DIR}/${CMAKE_CFG_INTDIR}/${CMAKE_${build_type}_LIBRARY_PREFIX}${external_name}${CMAKE_${build_type}_LIBRARY_SUFFIX}")
set(${external_name}_LIBRARY_DEBUG "${${external_name}_BINARY_DIR}/${CMAKE_CFG_INTDIR}/${CMAKE_${build_type}_LIBRARY_PREFIX}${external_name}${debug_postfix}${CMAKE_${build_type}_LIBRARY_SUFFIX}")
endif()
# Create an IMPORTED external library available in the GLOBAL scope
if (${header_only})
add_library(${new_target} INTERFACE)
else()
add_library(${new_target} ${build_type} IMPORTED GLOBAL)
endif()

# Set a dependency to the external target (ExternalProject fetcher and builder)
add_dependencies(${new_target} ${external})

# Set the target's properties, namely library file and include directory
if (NOT ${header_only})
set_target_properties(${new_target} PROPERTIES IMPORTED_LOCATION_RELEASE ${${external_name}_LIBRARY_RELEASE})
set_target_properties(${new_target} PROPERTIES IMPORTED_LOCATION_DEBUG ${${external_name}_LIBRARY_DEBUG})
# Since we do not really manage other build types, let's globally use the DEBUG symbols
if(MSVC)
# Only Debug build uses MDd or MTd, and others use MD or MT.
# MSVC would like to use same runtime library, so we use Debug third-party in Debug and Release third-party in others.
set_target_properties(${new_target} PROPERTIES IMPORTED_LOCATION
${${external_name}_LIBRARY_RELEASE})
else()
if (NOT CMAKE_BUILD_TYPE MATCHES "[Rr][Ee][Ll][Ee][Aa][Ss][Ee]"
AND NOT CMAKE_BUILD_TYPE MATCHES "[Dd][Ee][Bb][Uu][Gg]")
set_target_properties(${new_target} PROPERTIES IMPORTED_LOCATION
${${external_name}_LIBRARY_DEBUG})
endif()
endif()
endif()
set_target_properties(${new_target} PROPERTIES INTERFACE_INCLUDE_DIRECTORIES ${${external_name}_INCLUDE_DIR})
endmacro(ginkgo_add_external_target)


# Ginkgo specific add_subdirectory helper macro.
# If the package was not found or if requested by the user, use the
# internal version of the package.
#
# \param package_name Name of package to be found
# \param dir_name Name of the subdirectory for the package
#
macro(ginkgo_add_subdirectory package_name dir_name)
if (NOT ${package_name}_FOUND)
add_subdirectory(${dir_name})
endif()
endmacro(ginkgo_add_subdirectory)


# Ginkgo specific find_package helper macro. Use this macro for third
# party libraries.
# If the user does not specify otherwise, try to find the package.
#
# \param package_name Name of package to be found
# \param target_list For TPL packages, declare a new target for each library
# \param header_only For TPL packages, declare the tpl package as header only
# \param ARGN Extra specifications for the package finder
#
macro(ginkgo_find_package package_name target_list header_only)
string(TOUPPER ${package_name} _UPACKAGE_NAME)
if (GINKGO_USE_EXTERNAL_${_UPACKAGE_NAME} OR TPL_ENABLE_${_UPACKAGE_NAME})
if (TPL_${_UPACKAGE_NAME}_LIBRARIES AND TPL_${_UPACKAGE_NAME}_INCLUDE_DIRS)
set(${package_name}_FOUND "${TPL_${_UPACKAGE_NAME}_LIBRARIES};${TPL_${_UPACKAGE_NAME}_INCLUDE_DIRS}")
set(_target_list ${target_list}) # CMake weirdness: target_list is not a list anymore
# Count the number of elements in the list. Substract by one to iterate from 0 to the end.
list(LENGTH _target_list _GKO_len1)
math(EXPR _GKO_len2 "${_GKO_len1} - 1")
foreach(val RANGE ${_GKO_len2})
list(GET _target_list ${val} target) # access element number "val" in _target_list
list(GET TPL_${_UPACKAGE_NAME}_LIBRARIES ${val} lib)
ginkgo_add_tpl_target("${target}" "${_UPACKAGE_NAME}" "${TPL_${_UPACKAGE_NAME}_INCLUDE_DIRS}"
"${lib}" ${header_only})
endforeach()
else()
find_package(${package_name} ${ARGN})
if (${package_name}_FOUND)
message(STATUS "Using external version of package ${package_name}. In case of problems, consider setting -DGINKGO_USE_EXTERNAL_${_UPACKAGE_NAME}=OFF.")
else()
message(STATUS "Ginkgo could not find ${package_name}. The internal version will be used. Consider setting `-DCMAKE_PREFIX_PATH` if the package was not system-installed.")
endif()
endif()
endif()
endmacro(ginkgo_find_package)


# Download a file and verify the download
#
# \param url The url of file to be downloaded
Expand Down
Loading
0