The souce of packages that we have installed on Borges are under /scratch/packages. It is not organized and contains many different version of a package. We will gradually put the organized packages under /scratch/cca-chem-pacakges. Currently there are only three packages under /scratch/cca-chem-packages: gamess-component, mpqc and nwchem-5.0.
If you are looking for a package, look under /opt/ifort-pkgs/ first.
1. CCA-tools needs Java, and it needs the right one. One Borges do NOT use the one under /usr/bin. Have "/opt/jdk1.6.0/include:/opt/jdk1.6.0/bin" included in your path before /usr/bin.
2. There are several version of MPI under /opt/ifort-pkgs/. Pick the one you want to use, and have its "/bin/" and "/include/" in your path.
3. I assume many people like to challenge themselves by trying to install cca-tools. Since there are different versions of cca-tools and babel, here is an important tip. Before you install a new version of cca-tools, REMOVE ANY ACCESS to previous installed cca-tools in your path. If you are installing a new version of cca-tools and in your path there is access to old cca-tools, it's very likely that the installation will fail and you will have a lot of fun looking for what went wrong.
[Back to the top]Intel FORTRAN compiler is located in /opt/intel/. The 32-bit version is under /fc sub-directory, and the 64-bit version is under /fce sub-directory. On Borges, I strongly suggest you use the 64-bit version to compile your FORTRAN 90 programs on Borges (The 32-bit version can not be used when compiling mpich2).
When compiling some packages, you may need to set some environment variables to enable use of ifort. For example, when compiling mpich2, you will have to set F90=ifort (also, setting F77=ifort is strongly suggested).
Intel ifort can produce binary code that is not compatible with binary codes generated by g77. If you have FORTRAN 77 codes compiled by g77, you may have to re-compile them with ifort to ensure the compatibility between your FORTRAN 77 and FORTRAN 90 codes.
On Borges, packages compiled with Intel ifort (mpich, cca-tools etc.) are put undet the directory /opt/ifort-pkgs/.
[Back to the top]gfortran is the default FORTRAN 90 compiler on Borges. The version under /usr/bin/gfortran is 4.1.0. While it can be used to compile many FORTRAN 90 programs, we encountered some unexpected errors. For example, if mpich2 is compiled with gfortran and with option "--enable-sharedlib=gcc", and NWChem is also compiled with gfortran, running the testing data will encounter segmentation fault. We did not find this problem when all programs are compiled with Intel ifort.
For some unknown reason, there are gfortran 4.1.1 or 4.1.2 rpm packages for different Linux, but none for RedHar Enterprise 4 (the one on Borges). gfortran-4.2.0 (pre-release) and gfortran-4.3.0 (experimental) are tested, they generated more problems then 4.1.0, and we don't encourage you use them on Borges. (If you insist to use them, they are under (/scratch/packages/gfortran/gcc-4.2/bin for gfortran 4.2.0 and /scratch/packages/gfortran/gcc-trunk/bin for gfortran 4.3.0).
You can directly use gfortran on Borges without adding anything to your shell.
[Back to the top]Which FORTRAN 90 compiler to use is totally up to you. Using Intel ifort you may encounter less troubles (on Borges); however, it is a commercial product and you don't know when Intel will drop their "Non-commercial download" again like they did in 2006. gfortran is free, but you may encountered some troubles that will take a while to figure out that they are caused by compiler. At this monent I suggest using Intel ifort for our short term projects.
The latest update for DDI is: March 2007
The latest update for GAMESS is: March 2007
1. copy the package "gamess.tgz" to the current directory
2. uncompress the package
Note:
Please always use the newest version of gamess-component
Required Packages:
cca-tools-0.6.1_rc2, babel-1.0.2, cca-chem-generic2
Suggested Build Sequences:
1. download the latest version of mpqc that work with babel-1.0 branch
2. create configure file
3. create a build directory for mpqc
4. configuring mpqc
5. compiling mpqc
6. test mpqc
7. test CCA component
Required Package
================
cca-tools version >= 0.6.1
babel version >= 1.0.0
Download
========
If you have an account for cca-forum.org, you can do the following steps:
ssh username@cca-forum.org
cvs co -r release-2-0-branch cca-chem-generic2
If you don't have an account for cca-forum.org, you can copy the package from /scratch/fangp/packages/cca-chem-generic2
Configuration
=============
You have to specify the path to "ccafe-config" script with the option "--with-ccafe-config". An example of configuration is:
../configure --with-ccafe-config=/opt/ifort-pkgs/cca-tools-0.6.1_rc2-babel-1.0.2/bin/ccafe-config --prefix=/clusters/scl/fangp/prefix/cca-chem-generic2
Compilation
===========
make;make install
[Back to the top]Required Package
================
cca-tools version >= 0.6.1
babel version >= 1.0.0
cca-chem-generic2
Download
========
If you have an account for cca-forum.org, you can do the following steps:
ssh username@cca-forum.org
cvs co -r release-2-0-branch cca-chem-apps2
If you don't have an account for cca-forum.org, you can copy the package from /scratch/fangp/packages/cca-chem-apps2Configuration
============
You have to specify the path to "ccafe-config" script with the option "--with-ccafe-config" and the path to "cca-chem-config" script with the option "--with-ccafe-config". An example of configuration is:
./configure --prefix=/clusters/scl/fangp/prefix/cca-chem-apps2 --enable-mpqc --with-cca-chem-config=/clusters/scl/fangp/prefix/cca-chem-generic2/bin/cca-chem-config --with-ccafe-config=/clusters/scl/fangp/prefix/cca-tools-0.6.2_rc3/bin/ccafe-config --with-sc-config=/clusters/scl/fangp/babel-1.0-branch/mpqc/build/bin/sc-config --disable-nwchem --disable-tao
Compilation
===========
make;make install
[Back to the top]MPICH1 on Borges is compiled with the following settings:
./configure --prefix=/opt/ifort-pkgs/mpich-1.2.7p1-sharedlib --enable-sharedlib --enable-f90
Which FORTRAN compile to used must be set through environment variables F77 or F90. Versions to support shared libraries (with extention -sharedlib in directory name) and without supporting shared libraries are built.
[Back to the top]MPICH2 on Borges is compiled with the following settings:
./configure --prefix=/opt/ifort-pkgs/mpich2-1.0.5p4-sharedlib --enable-sharedlibs=gcc --enable-f90 --disable-romio
Which FORTRAN compile to used must be set through environment variables F77 or F90. Versions to support shared libraries (with extension -sharedlib in directory name) and without supporting shared libraries are built. ROMIO for parallel I/O is disabled as it causes linking error when building NWChem 5.0.
[Back to the top]MVAPICH is compiled with the same option as MPICH-1.2.7 which it is built upon.
./configure --prefix=/opt/ifort-pkgs/mpich-1.2.7p1-sharedlib --enable-sharedlib --enable-f90
MVAPICH utilizes Infiniband for message passing. It is installed under /opt/ifort-pkgs/ and currently is under testing.
[Back to the top]Before installing any version of cca-tools, make sure you remove any path to previously installed CCA tools in your $PATH. If you don't do this, you may not be able to install CCA tools properly.
On Borges, CCA tools are installed with the following setting:
./configure --with-F77=/opt/intel/fce/9.1.043/bin/ifort --with-F90=/opt/intel/fce/9.1.043/bin/ifort --prefix=/opt/ifort-pkgs/cca-tools-0.6.1_rc2-babel-1.0.2 --with-F90-vendor=Intel --with-mpi=/opt/ifort-pkgs/mpich2-1.0.5p4-sharedlib/Versions of FORTRAN compilers, mpich2 and installed directory will have to be changed accordingly.
[Back to the top]Intel MKL is installed under /opt/intel/mkl/. According to testing from Dr.Jonathan Bentz, it provides performance a lot better than using the other math libraries on Intel platforms.
To use Intel KML, you need to include the path to Intel KML library into your LD_LIBRARY_PATH. There are several versions, make sure you include the libraries under em64t: /opt/intel/mkl/9.0/lib/em64t
[Back to the top]Intel C++ compiler is installed under /opt/intel/mkl/cce. To use it, put the following line in your login script:
source /opt/intel/cce/9.1.047/bin/iccvars.sh
You can NOT use Intel CPP compiler to compiler cca-tools as cca-tools uses gcc, and there is no option in building cca-tools to let you specify c/c++ compiler you want.
[Back to the top]For the CCA-Tools to support Python, you will need the python built with SHARED LIBRARY support, and the OLD numerical python installed. NOTE!!!! Not the new Numpy, the OLD numerical python. The original version of python built on borges does not support shared library, so I build a new version and have the library installed under /usr/local/lib and /usr/local/lib/python2.5. You will need to include either one of these two in your LD_LIBRARY_PATH.
The old Numerical Python and the new Numpy are both installed on borges. You do not need to do anything w to use them except knowing the module names. If you want to install them on your own machine, the source code is under /scratch/packages. Just follow the instruction in the README file, it is very straightforward. You need root password if you want to install them under /usr/local/
[Back to the top]To build cca-tools, you need java and javac. On borges, you need the version that support x86_64. There is only java under /usr/bin, no javac. The version that supports x86_64 is installed under /opt/jdk1.6.0/. Please include the following in your path before building cca-tools.
/opt/jdk1.6.0/include:/opt/jdk1.6.0/bin
[Back to the top]GotoBLAS is another math library optimized for different platforms. I am still testing it. I hope I can find a time to do a comparison between GAMESS-GotoBLAS and GAMESS-IntelMKL and put the results here.
Some people may want to build cca-chem-generic2 and test the compatiability by themselves. Here are the procedures for those with patience.
First, all the packages required for the testing are under /scratch/cca-chem-packages/ directory on borges t1 node.
Now, the interesting part, building and testing procedures.
(1). Have a cup of good tea besides you. This is very important. Don't drink something with too much caffine; that makes you cranky and when things are not working well, you might smash your poor computer.
(2). Start building the base packages; in this case, it's cca-tools-0.6.1_rc2.tar.gz, cca-chem-generic2-old.tgz, and gamess-component-0.1.4-v2.tgz. I have tested this combination on several platforms and they are happy with each other, no one complains. You can use the instructions in this page to build them. For gamess-component, there is a configuration example (plain text file) in it. Building sequence is (1) cca-tools-0.6.1_rc3, (2) cca-chem-generic2-old and (3) gamess-component.
(3). If everything is built properly, you can type 'make test' under gamess-component directory and you should be able to see the result of a sample computation, followed by an error message. Ignore the error message, it will be solve in about another hundered years. If you are using mpich2 on your own machine, don't forget to type mpdboot before running the test.
(4). Repeat the building procedures with cca-tools-0.6.2_rc3, and cca-tools-0.6.4_rc8. You will find that cca-tools-0.6.2_rc3 is workingwith the other two packages, but the combination of cca-tools-0.6.4_rc8, cca-chem-generic2-old and gamess-component-0.1.4-v2 is not.
(5). Replace cca-chem-generic2-old with cca-chem-generic2-0925-2007.tgz, and gamess-component-0.1.4-v2 with gamess-component-0.1.4-v6.tgz. The two new packages are happy to see cca-tools-0.6.2_rc3 and 0.6.4_rc8, but not 0.6.1_rc2.
(6). Remember the new base package combination: cca-tools-0.6.4_rc8, cca-chem-generic2-0925-2007 and gamess-component-0.1.4-v6. If some packages are updated, you have to build cca-tools first, then cca-chem-generic2, and then gamess-component and run the testing in GAMESS. If the newly built packages don't like each other, let me know. Or you can go through possible combinations to find out which package is the bully one and need some special attention.
(7). I have those built on Borges (Redhat Enterprise), my own computer (Debian) and the virtual machine(Fedora Core 6) on my laptop (Mac-Intel). They are all behaving well so far. If you are using Debian and can not build cca-tools properly, you might want to check your python installation. CCA-tools need a configuration file in your python installation. It's not a problem on Redhat Enterprise or Fedora Core. On Debian, you need to install a python development package to get this, and it only works for python 2.3, not 2.4.
(8). For those who are using Intel-base Mac, here is a tip to save money and be able to compile FORTRAN programs. NO FORTRAN77 COMPILER availabe on Intel-base Mac. If you want one, spend a couple hundred dollars to buy it from Intel. Well, thanks a lot.
To walk around it, what you can do is to have the program 'Parallels Desktop' installed (less than a hundred bucks) and create a virtual machine, then run the cca-project on this virtual machine. Use Fedora Core 6, not 7. There is a bug in version 7 that prevents you from installind the OS properly on a virtual machine. Using version 6 is just fine for the cca-project.
(9). To understand the inside of the cca-chem-generic2 package, you might want to build its documentation from scratch since there is no pdf or word file available. Here is the procedure I 'found' a while ago:
According to Joe " As long as you have doxygen in your path, you should be able to build the html from your top object." This is, unfortunately, not the case on many platforms.
In this version of cca-chem-generic2, you can not just build the document. You have to configure cca-chem-generic2 first then you can build the document (The Makefile to build the document will be generated from Makefile.in under /doc after you configure cca-chem-generic2).
Once you have Makefile for /doc generated, and have doxygen in your path, you can type "make" to build the document, and you will FAIL. You will get the error message like:
"sh: dot: command not found".
If you type "man dot" under whatever linux system you are using, you will get a man page that is totally useless to you. This is because the "dot" command required to build cca-chem-generic2 is totally differernt from the "dot" command in linux.
The dot command required to build cca-chem-generic2 document is part of a package called "graphviz". You have to manually build the package to be able to use this "dot" command. You can find a webpage with a lot of rpm packages for graphviz, but none for el4.x86_64 (Borges).
With all those set, then the document can be built.
(10) Now, have fun building these packages and drinking your tea.
[Back to the top]