Recent site activity

Getting started‎ > ‎

CMake 101


What is cmake

According to cmake homepage

the cross-platform, open-source build system

Build QMCPACK with cmake

To build the package, move to build directory, run cmake and make:

  cd build
  cmake
..
  make

How it works

  • CMakeLists.txt files are recursively included during cmake step.
    • cmake creates a directory tree which mirrors the topdir (e.g., qmcpack) in the build directory
  • CMakeLists.txt are similar to Makefile.am for autoconf
  • CMake directory contains customized "cmake files" by QMCPACK developers to locate libraries and tools that are not fully supported by cmake.
    • Consider them as m4 files for autoconf

topdir/CMakeLists.txt is the main file which

  • defines how to build QMCPACK: dimensionality, precision, real/complex, mpi, openmp, .....
  • selects compilers
  • enables/disables advanced or developing features

The current default build uses

SET(OHMMS_DIM 3 CACHE INTEGER "Select physical dimension"
SET
(OHMMS_INDEXTYPE int)
SET
(OHMMS_PRECISION double)
SET
(APP_PRECISION double)

SET
(PRINT_DEBUG 0 CACHE BOOL "Enable/disable debug printing")
SET
(QMC_COMPLEX 0 CACHE INTEGER "Build for complex binary")
SET
(QMC_MPI 1 CACHE BOOL "Enable/disable MPI")
SET
(QMC_OMP 1 CACHE BOOL "Enable/disable OpenMP")
SET
(QMC_BITS 64 CACHE INTEGER "Select OS bit")
  • OHMMS_xyz s are old macros and will be replaced by APP
    • APP stands for APPlication so that other application can use it without feeling constrained by the name OHMMS.
  • QMC_COMPLEX=1 is for complex wavefunctions and fixed-phase DMC/RMC methods.
  • The "cached" parameters can be modified by users manually during a build by editing CMakeCache.txt.

cmake files (CMakeLists.txt, CMakeCache.txt and those with cmake extension) are text files; you can read them, make sense out of them and modify them.

Our-of-source compilation

Always use out-of-source compilation with cmake. The procedure above, using build directory (empty) and running camke in build, is an example. We can further separate the source (development) and build. Let's assume that the QMCPACK topdir is /home/foo/src/qmcpack. Then, one can build multiple executables in different locations by creating new directories and build QMCPACK in each directory.

/home/foo/build/gcc-real
/home/foo/build/gcc-complex
/home/foo/build/mpi-real

In each directory, e.g., /home/foo/build/gcc-real (after setting the environments properly), execute

$cd /home/foo/build/gcc-real
$cmake
/home/foo/src/qmcpack
$make

There is no need to change sources or cmake files. cmake .. in the main procedure uses .. because the source tree resides in the parent directory.

If something did not work, simply remove the directory (e.g., rm -rf gcc-real) and start again.

Building with environment variables

This method works with GNU, Intel, and IBM XLC compilers. When the libraries are installed in standard locations, e.g., /usr, /usr/local, there is no need to set the XYZ_HOME for XYZ package.

Set the environments (the examples below assume bash, Intel compilers and MKL library)

export CXX=icpc
export CC=icc
export MKL_HOME=/usr/local/intel/mkl/10.0.3.020
export LIBXML2_HOME=/usr/local
export HDF5_HOME=/usr/local
export BOOST_HOME=/usr/local/boost
export EINSPLINE_HOME=/usr/local/einspline
export FFTW_HOME=/usr/local/fftw
How to overwrite the default build

CMakeLists.txt can be edited before cmake step.

  • This is the only way to change OHMMS_PRECISION and OHMMS_INDEXTYPE
  • single-precision has not been debugged (probably compilers will give up).
  • There is NO NEED to use long for OHMMS_INDEXTYPE

Several variables can be overwritten at cmake step by passing arguments to cmake. For example To disable MPI,

cmake -DQMC_MPI=0 ..

variable type default comment
QMC_BUILD_LEVEL int 1 QMC Build Level: 1=bare, 2=developer, 3=experimental
OHMMS_DIM int 3 physical dimension of the build
QMC_MPI bool 1 Eanble/disable MPI
QMC_OMP bool 1 Eanble/disable OpenMP
QMC_COMPLEX bool 0 Eanble/disable complex build
BUILD_QMCTOOLS bool 0 Build tools for QMCPACK
BUILD_SANDBOX bool 0 Build sandbox for testing for the developers
ENABLE_PHDF5 bool 0 Enable use of phdf5
ENABLE_TAU_PROFILE bool 0 Enable tau for profiling

In addition to QMCPACK-defined variables, there are many cmake variables that can be manipulated the same way. Check out cmake wiki.

During cmake step, CMakeCache.txt file is created in the build directory. As the name implies, it contains cached variables that are used during the build stage. This file can be edited to modify the cached variables above.

Selecting compilers

  • Use CXX and CC to set the compilers in your shell, e.g.,
  •  
    export CXX=mpicxx
    export CC=gcc
  • Without setting them, you are using CXX=g++ and CC=gcc and the default properties of the GNU compilers you are using.
  • cmake will use CMake/XYZCompilers.cmake, for XYZ=GNU, Intel, or IBM, if one of them is used.
    • CXX=icpc for Intel compilers
    • CXX=xlC for IBM compilers
  • When you use MPI compilers or scripts, e.g., mpicxx, few problems can arise.
  • On jaguar at ORNL, CMake/CrayXT.cmake is automatically used.
  • You can create your own compiler cmake file for fine tuning (See Writing your own cmake files?)

Development and testing are done mostly on LINUX systems using Intel 10.x or GNU 4.2 and higher compilers. Older compilers will work but supports for OpenMP both at the compiler and run time may not be good. We strongly encourage people to move on to new compilers whenever possible: they are usually getting better with few exceptions, which will be posted on this wiki whenever such cases are encountered.

Building with a toolchain file

Several toolchain files used by the developers are available in config directory. They are used on large-scale parallel machines where setting up all the necessary packages can be tricky.

  • AbeMvapich2.cmake for Abe cluster at NCSA
  • JaguarGNU.cmake for CrayXT systems at ORNL
  • KrakenGNU.cmake for CrayXT5 system at NICS

Check CMakeToolchain for the descriptions of the toolchain file and cmake variables.

Once a suitable toolchain file is found, follow these step (example on abe.ncsa.uiuc.edu):

cd build
cmake -DCMAKE_TOOLCHAIN_FILE=../config/AbeMvapich2.cmake -DCMAKE_VERBOSE_MAKEFILE=TRUE ..
cmake -DCMAKE_TOOLCHAIN_FILE=../config/AbeMvapich2.cmake -DCMAKE_VERBOSE_MAKEFILE=TRUE ..
make

Enabling MPI and/or OpenMP

MPI

MPI is automatically enabled if

  • CXX is set to parallel compilers
  • mpicxx, mpic++, cmpic++ (tungsten at NCSA)
  • mpCC/mpCC_r on AIX
  • mpi.h is found in standard paths, e.g., /usr/include or /usr/local/include
  • SGI Altix

One of these actions will disable MPI

  • Set QMC_MPI environment to 0, e.g., for bash
export QMC_MPI=0
  • Modify topdir/CMakeLists.txt
SET(QMC_MPI 0)

OpenMP

OpenMP is enabled by default in CMakeLists.txt.

  • Intel compilers
  • GNU/OpenMP compilers > 4.2.x on Linux 2.6.x kernels, Mac OS X
  • IBM XL compilers

One of these actions will disable OpenMP

  • Set QMC_OMP environment to 0, e.g., for bash
export QMC_OMP=0
  • Modify topdir/CMakeLists.txt
  • SET(QMC_OMP 0)
    SET
    (ENABLE_OPENMP 0)

If your machine has multiple cores, there is no need to disable OpenMP. However, make sure to set the environment variables which control OpenMP runs. Especially with MKL, set

MKL_NUM_THREADS=1
MKL_SERIAL=YES

so that the blas/lapack calls DO NOT USE threaded version.

Note that the default number of threads on your machine may be set to the number of cores (or CPU units). It is always safe to set the number of threads yourself as

export OMP_NUM_THREADS=1