Compile VASP using Intel oneAPI compilers

Steps:

Prerequisites:

sudo apt update
sudo apt upgrade
sudo apt-get install build-essential cmake cmake-curses-gui libopenmpi-dev openmpi-bin libfftw3-dev

Intel oneAPI Base Toolkit

Download the newest version from https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html?operatingsystem=linux&distributions=onlinehttps://developer.nvidia.com/hpc-sdk-downloads:

wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/1b2baedd-a757-4a79-8abb-a5bf15adae9a/l_HPCKit_p_2024.0.0.49589.sh
sudo sh ./l_HPCKit_p_2024.0.0.49589.sh

Continue with the installation according to the instructions (accept terms, recommended installation, ignore warnings about GUI and continue, skip Enclipse configuration).

Activate the oneAPI with: 

source /opt/intel/oneapi/setvars.sh

Add MKL to the ~/.bashrc:

nano ~/.bashrc

export PATH=/opt/intel/oneapi/mkl/2024.0:$PATH
export LD_LIBRARY_PATH=/opt/intel/oneapi/mkl/2024.0/lib/intel64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/intel/oneapi/compiler/2024.0/lib:$LD_LIBRARY_PATH

Intel® HPC Toolkit

Download the newest version from https://www.intel.com/content/www/us/en/developer/tools/oneapi/hpc-toolkit-download.html?operatingsystem=linux&distributions=online:

wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/1b2baedd-a757-4a79-8abb-a5bf15adae9a/l_HPCKit_p_2024.0.0.49589.sh
sudo sh ./l_HPCKit_p_2024.0.0.49589.sh

Activate the oneAPI with: 

source /opt/intel/oneapi/setvars.sh

OpenMPI compiled with oneAPI

wget https://download.open-mpi.org/release/open-mpi/v5.0/openmpi-5.0.0.tar.gz
mkdir OpenMPI_Intel
tar xvzf openmpi-5.0.0.tar.gz -C OpenMPI_Intel
cd OpenMPI_Intel/openmpi-5.0.0
mkdir build
./configure CC=icx CXX=icpx FC=ifort F77=ifort OMPI_CC=icx OMPI_CXX=icpx OMPI_FC=ifort OMPI_F77=ifort --prefix=/home/lebedmi2/SOFTWARE/OpenMPI/openmpi_intel/openmpi-5.0.0/build
make install -j

export PATH=/home/lebedmi2/SOFTWARE/OpenMPI/OpenMPI_Intel/openmpi-5.0.0/build/bin:$PATH
export LD_LIBRARY_PATH=/home/lebedmi2/SOFTWARE/OpenMPI/OpenMPI_Intel/openmpi-5.0.0/build/lib:$LD_LIBRARY_PATH

HDF5 compiled with oneAPI

mkdir HDF5_Intel
cd HDF5_Intel

Install ZLIB:

sudo apt install zlib1g

Download into HDF5_Intel folder the newest version of HDF5 from https://www.hdfgroup.org/downloads/hdf5/source-code/# , extract it, and create a new folder called ‚build‘ into which the compilation will be made. 

tar xvzf hdf5-1.14.4-2.tar.gz
cd hdf5-1.14.4-2/
mkdir build

 Configurate HDF5 with compilers from oneAPI:

CC=icx CXX=icpx FC=ifort ./configure --prefix=/home/lebedmi2/SOFTWARE/TEST_New_HDF5/HDF5_intel/hdf5-1.14.4-2/build --enable-fortran --enable-cxx --enable-shared 

 Then build and install it:

make -j
make install -j

 Before compiling VASP or running the simulations with this version, make sure to either permanently or temporally export HDF5 directory to system variables:

export PATH=/home/lebedmi2/SOFTWARE/TEST_New_HDF5/HDF5_intel/hdf5-1.14.4-2/build/bin:$PATH
export LD_LIBRARY_PATH=/home/lebedmi2/SOFTWARE/TEST_New_HDF5/HDF5_intel/hdf5-1.14.4-2/build/lib:$LD_LIBRARY_PATH

 OLDER VERSION of HDF5 with Cmake:

Create a new folder where HDF5 will be compiled:

mkdir HDF5_Intel
cd HDF5_Intel

Download into this folder Cmake version of HDF5 from https://www.hdfgroup.org/downloads/hdf5/source-code/#:

tar xvzf CMake-hdf5-1.14.3.tar.gz
cd CMake-hdf5-1.14.3/

Compile LIBAEC and ZLib downloaded with HDF5:

tar xvzf LIBAEC.tar.gz
tar xvzf ZLib.tar.gz
cd libaec-v1.0.6
mkdir build
cd build
cmake ..
sudo make install
cd ..
cd ..
cd zlib-1.3
mkdir build
cd build
cmake ..
sudo make install

Add libsz.so to path:

export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

cd ..
cd ..

Copy hdf5-1.14.3 source to the HDF5_Intel dir:

cp -r hdf5-1.14.3/ ..

In HDF5_Intel dir, make a new directory called “myhdfstuff”. Without this called folder, compilation is problematic:

cd ..
mkdir myhdfstuff

Copy the source files in hdf5-1.14.3 to myhdfstuff:

cp -r hdf5-1.14.3/ myhdfstuff/

Build HDF5:

cd myhdfstuff
mkdir build
cd build
cmake -G "Unix Makefiles" -DBUILD_SHARED_LIBS:BOOL=ON -DHDF5_BUILD_FORTRAN=YES -DHDF5_BUILD_TOOLS:BOOL=ON -DCMAKE_C_COMPILER=icx ../hdf5-1.14.3 
cmake --build . --config Release -j
cpack -C Release CPackConfig.cmake
./HDF5-1.14.3-Linux.sh 

Scroll down with enter, accept license with “y”, and write “n” to have it installed in the build directory.
Before running the simulations with this version, make sure to either permanently or temporally export HDF5 directory to system variables:

export PATH=/home/lebedmi2/SOFTWARE/HDF5_Intel/myhdfstuff/build/HDF_Group/HDF5/1.14.3/bin:$PATH
export LD_LIBRARY_PATH=/home/lebedmi2/SOFTWARE/HDF5_Intel/myhdfstuff/build/HDF_Group/HDF5/1.14.3/lib:$LD_LIBRARY_PATH

VASP compiled with oneAPI and MKL support

tar xvzf vasp.6.4.1.tgz
mv vasp.6.4.1 vasp.6.4.1_Intel_MKL
cd vasp.6.4.1_Intel_MKL
cd vasp.6.4.1

To run commands without sudo (otherwise you could face problems with paths, see below this section), give rights to the currect user for the vasp.6.4.1 folder (change only the username ‚lebedmi2‘ in the following command. You can check name of the user with command ‚whoami‚):

sudo chown -R lebedmi2 .

Create makefile.include file:

nano makefile.include

Into makefile.include, copy here from: makefile.include.intel_ompi_mkl_omp_acc (https://www.vasp.at/wiki/index.php/Makefile.include.intel_ompi_mkl_omp or find it in arch folder of your VASP)..

In makefile.include, set the paths to to MKL (MKLROOT) and HDF5 (HDF5_ROOT). You must also change icc to icx and icpc to icpx. icc and icpc are depreceated and no longer included in newer versions of oneAPI.

MKLROOT    ?= /opt/intel/oneapi/mkl/2024.0
HDF5_ROOT  ?= /home/lebedmi2/SOFTWARE/TEST_New_HDF5/HDF5_intel/hdf5-1.14.4-2/build
CC_LIB = icx
CXX_PARS = icpx

Compile vasp:

make DEPS=1 -j

It will take some time. If no error appeared, check if all libraries are correctly set and none are missing:

cd bin
ldd vasp_std

Export number of threads to bashrc:

export OMP_NUM_THREADS=2

Try if the compiled version is working (write the following command in the same directory where the makefile.include is. It should give computed values and no errors):

make test -j 

If more than one vasp compilation will be installed on the computer, rename the binary:

mv vasp_std vasp_intel

Add path vasp_intel_mkl to bashrc:

export PATH=/home/lebedmi2/SOFTWARE/VASP/vasp.6.4.1_INTEL/vasp.6.4.1/bin:$PATH

Run the simulations with VASP Intel, MKL:

Before running the simulations, all the following should be exported (if they are not set permanently in ~/.bashrc). In my case:

#Intel compilers, libraries
source /opt/intel/oneapi/setvars.sh
#OpenMPI compiled with oneAPI
export PATH=/home/lebedmi2/SOFTWARE/OpenMPI/openmpi_intel/openmpi-5.0.0/build/bin:$PATH
export LD_LIBRARY_PATH=/home/lebedmi2/SOFTWARE/OpenMPI/openmpi_intel/openmpi-5.0.0/build/lib:$LD_LIBRARY_PATH
#HDF5 compiled with oneApi
export PATH=/home/lebedmi2/SOFTWARE/HDF5_Intel/myhdfstuff/build/HDF_Group/HDF5/1.14.3/bin:$PATH
export LD_LIBRARY_PATH=/home/lebedmi2/SOFTWARE/HDF5_Intel/myhdfstuff/build/HDF_Group/HDF5/1.14.3/lib:$LD_LIBRARY_PATH

Then run with, e.g.:

mpirun -np 4 vasp_intel 

To run on all processors and threads:

mpirun --use-hwthread -np N vasp_intel

This may not always be effective, as it depends on the simulation settings and size of the simulated system. For example, with 72 Cr atoms, 10 DAV iterations, 3x3x3 k-sampling, and 350 eV Ecut:

The last command is running as: 20 mpi-ranks, with 2 threads/rank, on 1 nodes
distrk: each k-point on 20 cores, 1 groups