TL;DR - Two releases coming up : Where we at ?
A recent post discussed the roadmap for the CODE-RADE project, and put forward two important dates for releases :
- Infrastructure Foundation Release 2 - 01/03/2016
- Public Release 1 - 29/04/2016
Here’s a quick update of where we’re at 1
Things in
Abinit
Abinit is configured to build for:
- OS : u1404 , sl6
- VERSION : 7.10.5
- GCC Version : 4.9.2 , 5.1.0
All of the dependencies of Abinit are passing :
- fftw3
- gsl
- lapack
- netcdf
This is a good place to be, but abinit itself is failing on all targets due to the FFTW3 dependency. The error is described in #4 .
TODO - check what kind of FFT Abinit wants.
GADGET-2
GADGET is configured to be compiled with
- fftw2
- openmpi
- gsl
- hdf5
The job is not yet complete, since the
check-build.sh
and
deploy.sh
scripts are missing. There seems to be some misconfiguration with the makefiles, which need to be checked.
GROMACS
Gromacs is configured for the targets:
- VERSION : 5.1.1
- GCC Version : 4.9.2 , 5.1.0
- OpenMPI version : 1.8.1 , 1.8.8
Gromacs depends on
- boost
- fftw3
- lapack
as well as the aforementioned GCC and OpenMPI. GROMACS is currently suffering from the wrong FFTW configuration. The build is configured as :
cmake ../
\
-G
"Unix Makefiles"
\
-DCMAKE_C_COMPILER
=
mpicc
\
-DCMAKE_CXX_COMPILER
=
mpicxx
\
-DGMX_X11
=
OFF
\
-DGMX_FFT_LIBRARY
=
fftw3
\
-DGMX_DOUBLE
=
OFF
\
-DGMX_GPU
=
OFF
\
-DGMX_OPENMP
=
ON
\
-DGMX_MPI
=
ON
\
-DGMX_EXTERNAL_BLAS
=
on
\
-DGMX_BUILD_MDRUN_ONLY
=
ON
\
-DCMAKE_PREFIX_PATH
=
${
BOOST_DIR
}
/boost;
${
LAPACK_DIR
}
;
${
FFTW_DIR
}
;
${
OPENMPI_DIR
}
\
-DREGRESSIONTEST_DOWNLOAD
=
ON
\
-DCMAKE_INSTALL_PREFIX
=
${
SOFT_DIR
}
/
${
VERSION
}
-gcc-
${
GCC_VERSION
}
-mpi-
${
OPENMPI_VERSION
}
However, this is failing with the following error message :
– pkg-config could not detect fftw3f, trying generic detectionCould not find fftw3f library named libfftw3f, please specify its location in CMAKE_PREFIX_PATH or FFTWF_LIBRARYby hand (e.g. -DFFTWF_LIBRARY=’/path/to/libfftw3f.so’)CMake Error at cmake/gmxManageFFTLibraries.cmake:81 (MESSAGE):
Cannot find FFTW 3 (with correct precision - libfftw3f for mixed-precisionGROMACS or libfftw3 for double-precision GROMACS). Either choose the rightprecision, choose another FFT(W) library (-DGMX_FFT_LIBRARY), enable theadvanced option to let GROMACS build FFTW 3 for you(-DGMX_BUILD_OWN_FFTW=ON), or use the really slow GROMACS built-in fftpacklibrary (-DGMX_FFT_LIBRARY=fftpack).
The error doesn’t seem to be in
CMAKE_PREFIX_PATH
, so it is likely in one of the FFTW variables.
TODO : Find out how to specify the correct FFTW3 configuration
Update
: 10/02 - fixed in
d8e48ec5
, however GROMACS in the repo only contains
mdp_mpi
and none of the other executables. Need to revisit the configuration.
Python
Python is one of the generic frameworks which is difficult to treat like the other applications in CODE-RADE, because it has it’s own package management tools and internal dependency management. It also doesn’t usually deal with modules, rather using it’s own
virtualenv
to deal with different versions of Python and different modules within it. However, it can still be built from source, and the supporting applications which it uses -
pip
,
easy_install
,
etc
can be added to it. Care should be taken to use the python environment variables properly in order to ensure that the subsequent python modules such as those in the SciPy suite, are installed and delivered correctly.
The current overview of the status of python is :
Python currently depends on
- bzlib
- ncurses
- readline
- sqlite
- tcltk
- zlib-deploy
which are all currently passing.
Python 2.6 and Python 2.7 are failing as follows.
Python 2.6
Python 2.6 needs setuptools. This is downloaded as part of the build script, but fails immediately with :
Traceback
(
most
recent
call
last
):
File
"setup.py"
,
line
21
,
in
<
module
>
exec
(
init_file
.
read
(),
command_ns
)
File
""
,
line
11
,
in
<
module
>
File
"/var/lib/jenkins/workspace/python-deploy/ARCH/x86_64/GCC_VERSION/4.9.2/NAME/python/OS/u1404/SITE/generic/VERSION/2.6.9/Python-2.6.9/setuptools-18.3.2/setuptools/__init__.py"
,
line
12
,
in
<
module
>
from
setuptools.extension
import
Extension
File
"/var/lib/jenkins/workspace/python-deploy/ARCH/x86_64/GCC_VERSION/4.9.2/NAME/python/OS/u1404/SITE/generic/VERSION/2.6.9/Python-2.6.9/setuptools-18.3.2/setuptools/extension.py"
,
line
8
,
in
<
module
>
from
.dist
import
_get_unpatched
File
"/var/lib/jenkins/workspace/python-deploy/ARCH/x86_64/GCC_VERSION/4.9.2/NAME/python/OS/u1404/SITE/generic/VERSION/2.6.9/Python-2.6.9/setuptools-18.3.2/setuptools/dist.py"
,
line
16
,
in
<
module
>
from
setuptools.depends
import
Require
File
"/var/lib/jenkins/workspace/python-deploy/ARCH/x86_64/GCC_VERSION/4.9.2/NAME/python/OS/u1404/SITE/generic/VERSION/2.6.9/Python-2.6.9/setuptools-18.3.2/setuptools/depends.py"
,
line
6
,
in
<
module
>
from
setuptools
import
compat
File
"/var/lib/jenkins/workspace/python-deploy/ARCH/x86_64/GCC_VERSION/4.9.2/NAME/python/OS/u1404/SITE/generic/VERSION/2.6.9/Python-2.6.9/setuptools-18.3.2/setuptools/compat.py"
,
line
28
,
in
<
module
>
import
urllib2
File
"/apprepo/generic/u1404/x86_64/python/2.6.9-gcc-4.9.2/lib/python2.6/urllib2.py"
,
line
93
,
in
<
module
>
import
hashlib
File
"/apprepo/generic/u1404/x86_64/python/2.6.9-gcc-4.9.2/lib/python2.6/hashlib.py"
,
line
138
,
in
<
module
>
sha224
=
__get_builtin_constructor
(
'sha224'
)
File
"/apprepo/generic/u1404/x86_64/python/2.6.9-gcc-4.9.2/lib/python2.6/hashlib.py"
,
line
66
,
in
__get_builtin_constructor
import
_sha256
ImportError
:
No
module
named
_sha256
It seems that Python has not built the sha256 library.
Python 2.7
Python 2.7 seems to have passed
build
and
check-build
, but when cleaning the build directory at the start of
deploy
we get the following error :
+ ./deploy-2.7.11.shrm: cannot remove
`
build/lib.linux-x86_64-2.7
': Directory not empty
This is thrown by the lines
cd
${
WORKSPACE
}
/Python-
${
VERSION
}
/build-
${
BUILD_NUMBER
}
rm -rf
*
This could easily be circumvented by a quick hack, but the origin of the problem is not understoo (yet)
Quantum Espresso
Quantum Espresso is having issues compiling against the LAPACK libraries provided.
WRF
The Weather Research and Forecasting model is very widely used in meteorology and climatology. It depends on
- hdf5 (compiled with MPI)
- jasper
- mpich
- netcdf (compiled with MPI)
all of which are passing.
The project files are not yet ready, so this build is failing to find the scripts. Although the build has been tested locally using the artifacts in
fastrepo
, this has only been done by hand and the scripts have not yet been committed to the repo.
The rest
There are several applications which have been proposed for the Public Release. Apart from those described above, we have the list below. A short summary of the status of each is given.
| Name | Dependencies | Comment |
| GAMESS | HDF5 | Dependencies all fulfilled, but not tested locally |
| Repast Symphony | Java | Dependencies all fulfilled, but not tested locally |
| ABySS | BOOST, Sqlite, SparseHash | SparseHash is not yet in Foundation Release, but is WIP. |
| OpenFOAM | BOOST, OpenMPI, gnuplot, readline, ncurses | gnuplot is not yet in Foundation Release.. The OpenFOAM job and repo are present, but only the build script is prepared. |
| AutoDock | None, apparently | Low hanging fruit ? Need to add the scripts to the repo and create the Jenkins job. |
| Velvet | OpenMP | Low hanging fruit ? Need to add the scripts to the repo and create the Jenkins job. |
| OASES | OpenMP, zlib, Velvet | Low hanging fruit ? Need to add the scripts to the repo and create the Jenkins job. |
| WEKA | Java JDK | JDK has been added with build 113 of fastrepo and WEKA has been tested manually. Compilation is not clear yet. |
| PLink | zlib (uses it's own compiled version), LAPACK | Manual tests have passed. |
| Pythia8 | Several CLHEP and CERN libraries necessary | This one will take a while still. |
| FreeFem++ | GSL, OpenMPI/MPICH, PETSc, LAPACK, BLAS, SuiteSparse, FFTW3, HDF5. | Most of these can be downloaded by the compilation scripts proivded by the package, but that would be a bit couter to the CODE-RADE philosophy. |
| CosmoMC | pmclib, CAMB, GSL, FFTW3, LAPACK, WMAP, fitsio | WMAP, pmclib, CAMB and WMAP are missing. The rest have been integrated. CAMB needs HealPix and Fisher. Work has started on Healpix |
| Gadget 2 | FFTW2, GSL, HDF5 | Work has started, the repo has been created as well as the job. Check and deploy scripts are still to be committed. FFWT2 needs to be compiled with MPICH, it is currently only compiled with OpenMPI |
| HTK | none | Low hanging fruit ? Need to add the scripts to the repo and create the Jenkins job. |
| MITML | none | Low hanging fruit ? Need to add the scripts to the repo and create the Jenkins job. |
| GAMA-Platform | Java JDK | JDK has been added with build 113 of fastrepo and Gama has been tested manually. Compilation is not clear yet. |
| IMA-2 | none | Low hanging fruit ? Need to add the scripts to the repo and create the Jenkins job. |
Next steps.
The applications which have been described as low-hanging fruit are going to be added over the next few days. Significant work needs to be done for a few of the other ones, and lower-lying dependencies need to be added and compiled for different targets. We look forward to having some contribution from those interested, and using some of these as example applications in the upcoming Sci-GaIA winter school .
For now, let’s keep
Footnotes and References
-
We’ll report only on the user applications in the Public Release ↩