8000 Springclean CMake scripts and developer/client build/use interfaces by drbenmorgan · Pull Request #13 · jrmadsen/PTL · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Springclean CMake scripts and developer/client build/use interfaces #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Oct 27, 2021

Conversation

drbenmorgan
Copy link
Contributor

Whilst this may look a significant change from the diffs, it's primarily a clean and tidy of the CMake scripts and use for both developers and clients of PTL.

  • Further "modern" CMake
    • Use of version ranges for CMake itself plus simpler PTL version handling/parsing
    • Canonical form of lowercase commands/macros/function, no repetition of conditional/loop in else()/endif()/end...() commands.
    • Use of target_compile_features to propagate the C++ standard to clients of PTL
  • CMake script simplification
    • Remove functions/macros/variables that are not used or can be replaced with CMake builtins
    • Some split/merge of modules on basis of responsibility and to try and keep options close to configuration required by that option (but this isn't hard and fast)
    • Add "report on change" for all options but reduce final reporting to enabled features only to make things a bit clearer. Things like compiler flags are not reported as these can change on a per-target or per build-mode basis, so only the actual compile commands are truly reliable here.
  • Build simplification
    • Don't force PIC for static libs as this might be unexpected and can be left up to the builder/packager through the CMAKE_POSITION_INDEPENDENT_CODE variable
    • Creation of a PTL/Config.hh file to #define/#undef the PTL_USE_LOCKS and PTL_USE_TBB symbols. Since these determine (AFAICT) the API/ABI of PTL, they've got to be propagated to clients and this saves having to transport -D flags around.
    • The Threads::Threads and TBB::tbb targets are now linked directly to PTL rather than an intermediary INTERFACE target given the simplicity of the link required.
    • Despite modern CMake preferring targets, the flags for sanitisation/coverage have been migrated to append to the CMAKE_..._FLAGS instead of via INTERFACE targets. Given that these flags aren't propagated to the usage requirements of an installed PTL, setting via the FLAGS variables is o.k (and even suggested by CMake and used in VTK among others). This also results in less code and prevents the need to export and setup what become essentially empty interface targets in the install (since the flags only apply to the BUILD_INTERFACE), or to deal with potentially complex genexes (to only link in the BUILD_INTERFACE).
  • Simpler support for using PTL as a CMake subproject via add_subdirectory
    • This part is somewhat specific to the Geant4 experience of vendoring PTL...
    • PTL options are no longer exposed directly to the user (i.e. they are not in the cache/GUI), but may still be set directly or exposed as options by the parent project if required. e.g. In Geant4, we have a GEANT4_USE_TBB option, so we don't want to expose the PTL_USE_TBB equivalent, just set it on the basis of the Geant4 one).
  • Unify the PTLConfig.cmake.in template for the build and install trees
    • Primarily a simplification so that the differences in including/using the targets is handled fully by the PTLTargets.cmake files that's generated for each.
  • Initial configuration/install of a pkg-config file for TBB
    • CMake's great, but good to support the other main tool for finding/using packages!
  • Examples are now built all-in one, so can focus on the code rather than the build scripts

Even with all of that, there are a couple of things missing:

  • I could not get the gpu example building either from the current master or here, so I'm not sure of its state or which CUDA/setup is needed.
  • I'm not 100% sure that all the DLL export symbols will be consistent on Windows
    • There's PTL_ALLOC_EXPORT set as a -D flag globally at build time (so it is set when compiling examples)
    • There's also LIB_BUILD_DLL in source/PTL/Types.hh
    • The PTLConfig.cmake file sets _PTL_ARCHIVE as an interface compile definition of PTL::ptl but this symbol isn't present in the code anywhere.

Anyway, there's way more than enough already in this PR...

@jrmadsen
Copy link
Owner

@drbenmorgan Can you check that #8 (reproducible build) is still valid?

@drbenmorgan
Copy link
Contributor Author

Good spot! I've fixed a mistake that was exporting a build tree path to the installed PTLConfig.cmake file. I've checked that different build/install dirs result in identical installed versions of PTLConfig.cmake per the issue in #8.

@jrmadsen
Copy link
Owner

Don't force PIC for static libs as this might be unexpected and can be left up to the builder/packager through the CMAKE_POSITION_INDEPENDENT_CODE variable

So I generally prefer to distribute these static libraries as PIC since the general idea behind static vs. shared libraries are static libraries copy the symbols by value whereas shared libraries copy the symbols by reference. Thus, its my understanding that if one is building libG4tasking.a as non-PIC then the PIC libptl.a symbols are copied into libG4tasking.a and effectively converted into non-PIC -- eliminating the runtime performance penalty.

@drbenmorgan
Copy link
Contributor Author

So I generally prefer to distribute these static libraries as PIC since the general idea behind static vs. shared libraries are static libraries copy the symbols by value whereas shared libraries copy the symbols by reference. Thus, its my understanding that if one is building libG4tasking.a as non-PIC then the PIC libptl.a symbols are copied into libG4tasking.a and effectively converted into non-PIC -- eliminating the runtime performance penalty.

The argument to have non-PIC the default is that it's just that - generally users/packagers etc (and CMake itself) expect/default to non-PIC statics. CMake defaults, as does Autotools, to build non-PIC statics, but anticipates the use case of building them PIC by offering the CMAKE_POSITION_INDEPENDENT_CODE option (--with-pic in lib tool). If that's forced to ON in the scripts or the target property hard-coded, there's no way to change it.

Looking around different Linux distro packaging guidelines, some require non-PIC statics, others PIC, but the general expectation seems to be that statics are default built without PIC, or at least provide a documented/easy way to change this.

One could provide an additional CMake option in PTL to disable PIC, but given CMake provides a good and clear one already, it just feels simpler/cleaner/less surprising to use that along with the default non-PIC case, i.e.

# default, builds ptl-static non-PIC
cmake <args>

# at user discretion, build ptl-static with PIC
cmake -DCMAKE_POSITION_INDEPENDENT_CODE=ON <args>

@jrmadsen
Copy link
Owner
jrmadsen commented Jul 3, 2021

@drbenmorgan Sorry for the delay, I will get this merged before the end of this next week.

Copy link
Owner
@jrmadsen jrmadsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The examples need to be passing

@drbenmorgan
Copy link
Contributor Author

@jrmadsen The remaining failure on clang-12 looks to be down to the change to not export nominally developer-only flags like sanitisers. I 8000 'm not totally sure why the other builds are fine with similar configurations though. Will try and reproduce locally.

@jrmadsen
Copy link
Owner

Ok @drbenmorgan the only way to fix it is to do:

    if(PTL_USE_SANITIZER AND NOT ("${PTL_SANITIZER_TYPE}" STREQUAL "leak"))
        string(REPLACE " " ";" _sanitize_args "${ptl_sanitize_args}")
        set_target_properties(ptl-shared PROPERTIES
            INTERFACE_LINK_OPTIONS "${_sanitize_args}")
    endif()

This is effectively just ensuring that libtsan or libasan is linked into the final exe.

@jrmadsen
Copy link
Owner

@drbenmorgan LMK if these changes work for you

@drbenmorgan
Copy link
Contributor Author

Thanks for the updates @jmadsen, and apologies this totally slipped my mind. I'll try these out ASAP next week and post back here.

@jrmadsen
Copy link
Owner

@drbenmorgan I am just going to merge this

@jrmadsen jrmadsen force-pushed the modern-cmake branch 2 times, most recently from 0ec3baf to bed35ba Compare October 27, 2021 20:32
drbenmorgan and others added 12 commits October 27, 2021 15:37
- Use version range for minimum CMake version

  Follow recommendation in:

  https://cliutils.gitlab.io/modern-cmake/chapters/basics.html

  to support current 3.8 minimum whilst always setting policy settings
  to those of the running version.

- Don't set policies which will automatically be NEW

- Let CMake fully handle versions without caching

  Caching allows user to set version values, but these are not
  user configurable values.

  Let CMake handle the setting of PTL_... and PROJECT_... version
  variables through the primary call to project() after version
  info has been read from VERSION file.
- Use modern CMake style of lowercase commands and empty else(),
  end...() commands
- Remove all unused custom functions and macros
  - Refactor remaining functionality into dedicated modules
- Remove obsolete/duplicate options and those that only apply to
  examples
- Only report relevant enabled features
  - Not possible to reliably report flags as these can change over
    the build tree
- Do not cache PTL_INSTALL_... variables, but allow override when
  PTL is a subproject, otherwise defaulting to the CMAKE_INSTALL_...
  equivalents
- Only expose ptl_add_option options when PTL is the primary project
  - parent projects can still set these manually
- Use compile features to export CXX Standard
  - Add cxx_std_<VERSION> to all targets built through ptl_build_library
    to ensure this usage requirement is propagated to clients.
  - Remove CMAKE_CXX_EXTENSIONS and CMAKE_CXX_STANDARD_REQUIRED from the
    cache as these should not be set by builders of the project.
  - Remove CMAKE_CXX_STANDARD setting in config files as the targets
    propagate this requirement.
- Do not default CMAKE_INSTALL_RPATH_USE_LINK_PATH setting
  This variable should be left up to the builder/installer as it
  hardcodes RPATH information that may not be wanted.
- Create header to propgate public preproc symbols
  PTL can be built with support for TBB and locks in user queues.
  These are enabled in code via #ifdef blocks on preprocessor symbols
  passed through -D flags. However, the choice is hardcoded at build
  time as it affects the API and ABI of the resultant PTL libraries.

  Promote TBB/Lock symbols to configured values in a new Config.hh file
  generated by CMake. Include this file in PTL code depending on the
  symbols. Install it along side other PTL headers.

  Remove setting of -D flags for TBB and locks from both public and
  private compile definitions.
- Update FindTBB module using version from VTK/Ogre, adding FPHSA support
- Link directly to Threads and TBB targets for simplicity and clarity
- Rationalize CMake/Build settings
  - Prefer use of CMAKE_.._FLAGS for passing sanitizer/coverage flags
    down the build tree. Whilst use of targets/usage requirements is
    preferred in modern CMake, it may not be the simplest solution for
    things like compiler flags that are pure BUILD_INTERFACE. Propagating
    flags through targets may result in these appearing in the install
    interface for the project. This may be confusing, or at worst, lead
    to unexpected behaviour.

    NB: Once CMake >= 3.13 is required, add_{compile,link}_options can
    be used instead.
- Unify build/install CMake config files
  - Handle primary target difference between build/install trees
    transparently via the PTLTargets.cmake file
  - Only use component arguments to find_package to decide on default
    for static/shared linking. Use of BUILD_{STATIC/SHARED}_LIBS may
    not result in the behaviour intended by the client project.
- Generate and install pkg-config file for PTL
Comment out GPU example as it cannot currently build either with
updated CMake scripts or those on master
Set path to custom FindTBB module using a @PACKAGE_...@ variable
to that it will not export build tree information to the install
tree.
@jrmadsen jrmadsen merged commit 50bc10a into jrmadsen:master Oct 27, 2021
@jrmadsen jrmadsen mentioned this pull request Nov 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0