Onion Information

Page Clicks: 0

First Seen: 03/15/2024

Last Indexed: 10/23/2024

Domain Index Total: 397

Preskoči na sadržaj CMake - C ross-supercomputer Make Vedran Miletic , HPC application expert, MPCDF* Max Planck Computing & Data Facility , Meet MPCDF , 2. May 2024 *Parts of the talk borrowed from previous talks by Sebastian Eibl, Markus Rampp, and other colleagues from the MPCDF Applications group Meet MPCDF From the announcement : The series Meet MPCDF offers the opportunity for the users to informally interact with MPCDF staff, in order to discuss relevant kinds of technical topics. Optionally, questions or requests for specific topics to be covered in more depth can be raised in advance via email to training@mpcdf.mpg.de . Users seeking a basic introduction to MPCDF services are referred to our semi-annual online workshop Introduction to MPCDF services (April and October). Software build from Software build on Wikipedia : In software development, a build is the process of converting source code files into standalone software artifact(s) that can be run on a computer, or the result of doing so. usually performed by a build tool (or a combination of tools), such as Make, Ninja, Gradle, and SCons Image source: Wikimedia Commons File:Linker.svg What does a build system do and why is there a need for it? It performs bookkeeping of sources that go into building of software artifacts : sources: files containing code in C, C++, CUDA, Fortran, HIP, Rust, ... building: compilation with GCC, NVIDIA nvcc, Intel ifx; Doxygen, ... artifacts: executables, libraries, API documentation in HTML and PDF, ... bookkeeping: remembering how it all happens so it can be repeated on different systems by different users (incl. sysadmins) Repeatable => build automation possible, e.g. for testing in GitLab CI (cf. Meet MPCDF seminar in March from Cristian C. Lalescu) Ideal build system features easy to use for both software developers and users high level of abstraction error messages should be clear and to the point making sure that every build is up-to-date and consistent avoids unnecessary work (make it fast!) extensible, free and open source, well maintained, ... Make and Ninja have a low level of abstraction, but there are many options with a higher level: CMake , Meson / muon , Bazel , xmake , build2 , Waf , Automake , SCons , ... CMake by Kitware CMake development began in 1999, in response to the need for a cross-platform build environment for the Insight Segmentation and Registration Toolkit (ITK) developer Brad King has stated: > the 'C' in CMake stands for 'cross-platform'" the project was funded by the United States National Library of Medicine as part of the Visible Human Project Image source: Wikimedia Commons File:Cmake.svg Is CMake close to that ideal? cross-platform open-source build system for Linux, macOS, FreeBSD, Windows, and other operating systems built on top of a "native" build system (e.g. GNU Make, BSD Make, MSBuild, Ninja), so users can continue using tools they are familiar with well supported in CLion, QtCreator, and Visual Studio (Code) "out-of-source" builds in parallel to the source tree: you can have many different flavours (with or without debugging, profiling, MPI, OpenMP, GPU support, ...) simple (arguably!) platform-independent and compiler-independent configuration files written in a custom macro language Is CMake close to that ideal? (cont.) automated tracking of dependencies between files (dependency checking can be by-passed with /fast targets) progress indicators (useful for projects that suddenly don't build as quickly as they did previously!) menu-driven configuration to help beginners (configuration options may have very complicated interactions!) ... Qt-based GUI ( cmake-gui ) and curses-based TUI ( ccmake ) ... but also command-line interface available for automating tasks Compared to other tools easier to maintain and more universal than handwritten Makefiles more portable and powerful than autotools easier to do things the right way than in SCons or Waf more popular and better supported by IDEs than Meson, build2, or xmake notably, Meson is increasingly popular in the open-source community smaller and easier to integrate with other tools than Bazel more arguments and comparisons are in Mastering CMake: Why CMake? Basic design and usage CMake just replaces ./configure script; it does not actually build the software (Make or Ninja does that): mkdir build ; cd build [ c ] cmake -DCMAKE_INSTALL_PREFIX = /u/system/test ../source make ; make install CMake's variables (e.g. CMAKE_INSTALL_PREFIX , CMAKE_C_COMPILER , CMAKE_CXX_COMPILER ) are distinct from environment variables (e.g. CC , CXX )! One master configuration file CMakeLists.txt in the top level directory of the source, additional ones in subdirectories. The CMakeLists.txt files contain procedures , not macros: the order is important. (Different from Makefile s!) Two stages of CMake's configure step Create a generic, platform-independent, internal representation. Configuration information is recorded in CMakeCache.txt in each build directory (there can be many with different configs). Load CMakeCache.txt , if it exists from a previous run. Process the commands in the CMakeLists.txt files, updating cmake variables along the way. Rewrite CMakeCache.txt . Iterate until all settings are consistent. Write Makefile s (or build.ninja s) for the required platform. Compiler options and dependencies get hardwired in them. Hello, world! A very simple CMakeLists.txt file: cmake_minimum_required ( VERSION 3.14 ) project ( Hello LANGUAGES Fortran ) add_executable ( hello.f90 ) Invoking cmake : $ cmake . - The Fortran compiler identification is GNU 13 .2.1 -- Detecting Fortran compiler ABI info -- Detecting Fortran compiler ABI info - done -- Check for working Fortran compiler: /usr/bin/f95 - skipped -- Configuring done ( 0 .2s ) -- Generating done ( 0 .0s ) -- Build files have been written to: /u/vedm/hello Hello, world! (cont.) Generates a Makefile : $ cat Makefile # CMAKE generated file: DO NOT EDIT! # Generated by "Unix Makefiles" Generator, CMake Version 3.29 # Default target executed when no arguments are given to make. default_target: all .PHONY : default_target # Allow only one "make -f Makefile2" at a time, but pass parallelism. .NOTPARALLEL: ( ... ) $ make [ 50 % ] Building Fortran object CMakeFiles/hello.dir/hello.f90.o [ 100 % ] Linking Fortran executable hello [ 100 % ] Built target hello Resources mpcdf/training/cmake-recipes on MPCDF GitLab https://gitlab.mpcdf.mpg.de/mpcdf , search for cmake CMake Reference Documentation https://cmake.org/documentation presentation will be shared after the talk Suggested additional readings: Mastering CMake An Introduction to Modern CMake The Architecture of Open Source Applications (Volume 1): CMake recipes for direct use module load git/2.43 git clone https://gitlab.mpcdf.mpg.de/mpcdf/training/cmake-recipes.git cd cmake-recipes module load cmake module load ninja Targets cd 00_targets module load gcc/13 cmake -S . B build -G Ninja ninja -C build cd .. CMake recipes for direct use (cont.) Third-party libraries cd 01_third_party_libraries module load intel/2024.0 module load mkl/2024.0 export MKL_ROOT = $MKL_HOME cmake -S . B build -G Ninja -DCMAKE_CXX_COMPILER = icpx ninja -C build cd .. Much more detail can be found in: Mastering CMake: Finding Packages Modern CMake: OpenMP CMake recipes for direct use (cont.) Skipped in the interest of time, but useful find package : How to link against "unsupported" libraries? glob : How to automatically collect all source files? custom targets : How to create custom targets to build the documentation, etc? configure file : How to pass information from CMake to your code? git configure : How to get the current git hash at configure time? git build : How to get the current git hash at build time? CMake recipes for direct use (cont.) CTest cd 08_ctest cmake -S . B build -G Ninja ninja -C build ctest --test-dir build cd .. Notable testing frameworks for C++: Doctest , Catch2 , GoogleTest CMake recipes for direct use (cont.) Install cd 09_install cd external cmake -S . B build -G Ninja -DCMAKE_INSTALL_PREFIX = /ptmp/vedm/software ninja -C build ninja -C build install cd .. cd internal module load gsl cmake -S . B build -G Ninja -DCMAKE_INSTALL_PREFIX = /ptmp/vedm/software ninja -C build ninja -C build install cd .. cd .. CMake recipes for direct use (cont.) Presets cd 10_presets cmake --preset dev -S . B build-dev -G Ninja ninja -C build-dev cmake --preset production -S . B build-production -G Ninja ninja -C build-production cd .. Could be narrowed further, e.g. developer build (build type Debug with tests enabled) Raven build ( Release with CUDA or HIP compiled for NVIDIA GPUs) Viper build ( Release with SYCL or HIP compiled for AMD GPUs) Add AVX-512 flags for different compilers if ( MSVC ) add_compile_options ( /arch:AVX512 ) elseif ( InteLLVM ) add_compile_options ( -march=skylake-avx512 ) else () # Clang OR GNU add_compile_options ( -march=znver4 ) endif () Official documentation: if , add_compile_options , compiler identification strings Enable compilation of CUDA and HIP sources include ( CheckLanguage ) check_language ( CUDA ) # also works for HIP if ( CMAKE_CUDA_COMPILER ) # HIP would have CMAKE_HIP_COMPILER enable_language ( CUDA ) # Replaces deprecated FindCUDA module else () message ( STATUS "No CUDA support" ) endif () Documentation: Official (CMake): CheckLanguage , CMAKE_LANG_COMPILER , enable_language , CMAKE_HIP_PLATFORM , HIP_ARCHITECTURES Unofficial: Modern CMake: CUDA , ROCm Documentation: Using CMake for HIP CMake in real world scientific software (GROMACS) GROMACS ( repository on GitLab ) has 154 CMakeLists.txt files (total of 14958 lines) and 100 supporting *.cmake files (total of 13402 lines). Examples that follow are simplified from the top-level CMakeLists.txt file (1044 lines). Safeguard: cmake_minimum_required ( VERSION 3.18.4 ) # soon to be 3.25+ C++ standard: set ( CMAKE_CXX_STANDARD 17 ) set ( CMAKE_CXX_STANDARD_REQUIRED ON ) set ( CMAKE_CXX_EXTENSIONS OFF ) Test the compiler behavior, check if it has a standard library: include ( gmxTestIntelLLVM ) # Run through a number of tests for buggy compilers and other issues include ( gmxTestCompilerProblems ) gmx_test_compiler_problems () find_package ( LibStdCpp ) Build as Release by default, but allow other options: set ( valid_build_types "Debug" "Release" "MinSizeRel" "RelWithDebInfo" \ "Reference" "RelWithAssert" "Profile" "TSAN" "ASAN" "MSAN" "UBSAN" ) if ( NOT CMAKE_BUILD_TYPE ) set ( CMAKE_BUILD_TYPE "Release" CACHE STRING "Choose \ the type of build, options are: ${valid_build_types}." FORCE ) # Set the possible values of build type for cmake-gui set_property ( CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS\ ${ valid_build_types } ) endif () Optional features that can be enabled during build: option ( GMX_DOUBLE "Use double precision (much slower)" OFF ) option ( GMX_MPI "Build a parallel (message-passing) version" OFF ) o