Scribd
Upload
7 views

hitoricae.com-code_aster 146 parallel version with PETSc

This document outlines the steps to build a parallel version of Code_Aster 14.6 using PETSc on a Linux system. It includes detailed instructions for preparing the environment, installing necessary packages, and configuring various libraries such as OpenBLAS, ScaLAPACK, and MUMPS. The final steps involve modifying configuration files and registering the parallel version for use.

Uploaded by

vvgmxdyqlstjagzbzt
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
7 views

hitoricae.com-code_aster 146 parallel version with PETSc

This document outlines the steps to build a parallel version of Code_Aster 14.6 using PETSc on a Linux system. It includes detailed instructions for preparing the environment, installing necessary packages, and configuring various libraries such as OpenBLAS, ScaLAPACK, and MUMPS. The final steps involve modifying configuration files and registering the parallel version for use.

Uploaded by

vvgmxdyqlstjagzbzt
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

code_aster 14.

6 parallel version with PETSc


hitoricae.com/2020/10/31/code_aster-14-6-parallel-version-with-petsc

2020年10月31日

This page is a note of the work to build parallel version.


The linux box is xUbuntu 18.04LTS on VMware.
Python is python 2.7.17 and Python3 is python 3.6.9.
This procedure is almost same as the case of code_aster 14.4.

After installing Linux, we recommend that you start building CodeAster before installing
any other apps.

０．preparation
Preparation is same as the case of CodeAster 14.6.
Some packages are installed as follows.

$sudo apt-get install gfortran g++ python-dev python-numpy liblapack-dev libblas-


dev tcl tk zlib1g-dev bison flex checkinstall openmpi-bin libx11-dev cmake grace
gettext libboost-all-dev swig

And install superlu package.

$sudo apt-get install libsuperlu-dev

And I recommend that you build sequential version at first.

１．OpenBLAS
To get source code from author’s HP. Then execute following procedure.

$ tar xfvz OpenBLAS-0.2.20.tar.gz


$ cd OpenBLAS-0.2.20
$ make NO_AFFINITY=1 USE_OPENMP=1
$ make PREFIX=/opt/OpenBLAS install
$ echo /opt/OpenBLAS/lib | sudo tee -a /etc/ld.so.conf.d/openblas.conf
$ sudo ldconfig

２．Code_Aster eith OpenBLAS


First, the source code of code_aster is unpacked. Next, “setup.py” is modified according to
the reference.

$ cd aster-full-src-14.6.0
$ sed -i "s:PREFER_COMPILER\ =\ 'GNU':PREFER_COMPILER\
=\'GNU_without_MATH'\nMATHLIB=\ '/opt/OpenBLAS/lib/libopenblas.a':g" setup.cfg

And

$ python3 setup.py install

After the build complete, to make host file for parallel calculation.

1/4
$ echo "$HOSTNAME cpu=$(cat /proc/cpuinfo | grep processor | wc -l)" >
/opt/aster/etc/codeaster/mpi_hostfile

３．ScaLAPACK

$ tar xfvz scalapack_installer.tgz


$ cd scalapack_installer
$ ./setup.py --lapacklib=/opt/OpenBLAS/lib/libopenblas.a --mpicc=mpicc --
mpif90=mpif90 --mpiincdir=/usr/lib/x86_64-linux-gnu/openmpi/include --ldflags_c=-
fopenmp --ldflags_fc=-fopenmp --prefix=/opt/scalapack-n

Over.

４．Parmetis
Get source code and unpack it. Integer size is 8 byte in this work.
#define IDXTYPEWIDTH 64″ and “#define REALTYPEWIDTH 32” is written in
~/parmetis-4.0.3/metis/include/metis.h. or add followings to metis.h

#ifdef INTSIZE32
#define IDXTYPEWIDTH 32
#else
#define IDXTYPEWIDTH 64
#endif

then

$ tar xvf parmetis-4.0.3.tar.gz


$ cd parmetis-4.0.3
$ make config prefix=/opt/parmetis-4.0.3
$ make

Next, checking.

$ cd Graphs
$ mpirun -np 8 ptest rotor.graph rotor.graph.xyz

When any error is reported, go to~/parmetis-4.0.3 and execute follows:

$ make install

This part is finished.

５．Scotch and Ptscotch


Unpack scotch-6.0.4-aster5.tar.gz which is included the package of code_aster in /opt.

$ cd /opt/scotch-6.0.4/src

Makefile.inc is here

$ make scotch esmumps ptscotch ptesmumps CCD=mpicc

After building, check it.

$ make check
$ make ptcheck

2/4
This part is finised.

６．MUMPS
Pick mumps-5.1.2-aster6.tar.gz from the package of code_aster out and unpack it. The
name of folder is changed to mumps-5.1.2_mob.

$ cd /opt/mumps-5.1.2_mob

Makefile.inc is.here.

$ make all

Next, to go to ~/examples and check the MUMPS.

７．Petsc
petsc-3.9.4.tar.gz is downloaded from author’s HP. Then unpack it in /opt. First , a few
line (line 43 to 48) is commented out metis.py in /opt/petsc-
3.9.4/config/BuildSystem/config/packages.

def configureLibrary(self):
config.package.Package.configureLibrary(self)
oldFlags = self.compilers.CPPFLAGS
self.compilers.CPPFLAGS += ' '+self.headers.toString(self.include)
# if not self.checkCompile('#include "metis.h"', '#if (IDXTYPEWIDTH != '+
str(self.getDefaultIndexSize())+')\n#error incompatible IDXTYPEWIDTH\n#endif'):
# if self.defaultIndexSize == 64:
# msg= '--with-64-bit-indices option requires a metis build with
IDXTYPEWIDTH=64.\n'
# else:
# msg= 'IDXTYPEWIDTH=64 metis build appears to be specified for a default 32-bit-
indices build of PETSc.\n'
# raise RuntimeError('Metis specified is incompatible!\n'+msg+'Suggest using --
download-metis for a compatible metis')

And when openmpi is used, path to openmpi library is added to LD_LIBRARY_PATH.


Next is a sample.

$ export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/openmpi/lib/:$LD_LIBRARY_PATH

Now the link for HYPRE is not work. Then HYPRE is downloaded
hypre_2.14.0.orig.tar.gz from https://launchpad.net/ubuntu/+source/hypre/2.14.0-
5build1.
In this case, hypre_2.14.0.orig.tar.gz is put under /opt.

Then, “configure” is done.

$./configure --with-debugging=0 COPTFLAGS=-O CXXOPTFLAGS=-O FOPTFLAGS=-O --with-


shared-libraries=0 --with-scalapack-dir=/opt/scalapack-n --PETSC_ARCH=linux-metis-
mumps --with-metis-dir=/opt/aster/public/metis-5.1.0 --with-parmetis-
dir=/opt/parmetis-4.0.3 --with-ptscotch-dir=/opt/scotch-6.0.4 --LIBS="-lgomp" --
with-mumps-dir=/opt/mumps-5.1.2_mob -with-x=0 --with-blas-lapack-lib=
[/opt/OpenBLAS/lib/libopenblas.a] --download-hypre=/opt/hypre_2.14.0.orig.tar.gz -
-download-ml=yes

3/4
and “make”

$ make PETSC_DIR=/opt/petsc-3.9.4 PETSC_ARCH=linux-metis-mumps all

８．Parallel version Code_Aster


Go to the folder stored the source code of code_aster. And unpack aster-14.6.0.

$ cd ~/Install_Files
$ cd aster-full-src-14.6.0/SRC
$ tar xfvz aster-14.6.0.tgz
$ cd aster-14.6.0

Next a part of ~/waftools/mathematics.py is modified to skip the checking of blacs.

# program testing a blacs call, output is 0 and 1


blacs_fragment = r"""
program test_blacs
integer iam, nprocs
# call blacs_pinfo (iam, nprocs)
# print *,iam
# print *,nprocs
end program test_blacs
"""

Then copy Ubuntu_gnu.py and Ubuntu_gnu_mpi.py to the folder. Now preparation of


build is ready.

$ export ASTER_ROOT=/opt/aster
$ export PYTHONPATH=/$ASTER_ROOT/lib/python3.6/site-packages/:$PYTHONPATH
$ ./waf configure --use-config-dir=$ASTER_ROOT/14.6/share/aster --use-
config=Ubuntu_gnu_mpi --prefix=$ASTER_ROOT/PAR14.6MUPT
$ ./waf install -p --jobs=1

When build is finished. add register the parallel version to /opt/aster/etc/codeaster/aster.

# Code_Aster versions
# versions can be absolute paths or relative to ASTER_ROOT
# examples : NEW11, /usr/lib/codeaster/NEW11

# default version (overridden by --vers option)


default_vers : stable

# available versions
# DO NOT EDIT FOLLOWING LINE !
#?vers : VVV?
vers : stable:/opt/aster/14.6/share/aster
vers : 14.6MUPT:/opt/aster/PAR14.6MUPT/share/aster

Whole work is finished.


I got many information from Code_aster forum. I appreciate the form members.

4/4

You might also like