Issues - GAMESS CCA Components Manual 1.0

Next: Documentation, Previous: Code Example, Up: Top

8 Issues

The shared library could not be loaded correctly when running with ”sockets” mode on Borges cluster. When running the gamess-component package with ”DDI/sockets” mode on multiple nodes, the shared libraries could not be loaded correctly on some nodes. For example, the error message shows the MKL shared libraries could not be found on t2 when running on t1 and t2. We solved this problem by encoding ”rpath” in the application. By inserting the ”-R /path-to-libs” flag to the linking command in the file ”shared/Makefile”, a C/C++ compiler could encode the search path in the shared birary. This can be done multiple times to add all the search paths to the linking list. For example, we use
```
          -L/opt/perf/tau-2.16.4/x86_64/lib -R /opt/perf/tau-2.16.4/x86_64/lib -lTAU  
     
```
to specify the search path for TAU shared libraries. When using the Intel Fortran compiler. Try "-Xlinker -rpath -Xlinker path-to-lib" flag instead.
Use Performance::Measurement component for performance evaluation for gamess-component package on Borges. When using a timer provided by Performance::Measurement component in a driver drivers::GAMESSTwoeiDriver, the following code was in "go" method:
```
          static Performance::Measurement measurement = babel_cast<Performance::Measurement>(services_.getPort("MeasurementPort"));
          if (measurement._is_nil()) {
              std::cout << "didn't get measurement\n";
              abort();
          }
          
          static Performance::Timer tm =
                 measurement.createTimerWithNameTypeGroup("driver_twoei_loop","TIMER","Chemistry_driver");
          
          tm.start();
          // some computation here
          tm.stop();
          
          services_.releasePort("MeasurementPort");
     
```
We use MPI program to start ccafe-batch in parallel. However, the profiles got are only profile.0.0.0 even when running on multiple nodes. One possible reason for this problem is that TAU may not be correctly configured with MPI program. To test if TAU is configured with MPI program, goto $tau-dir/examples/taucompiler/c and type "make". For the gamess-component package, we choose not to use Performance::Measurement components currently, because we just need timers for measuring the performance of both components and the wrapper functons, which can be provided by TAU in a very simple way.

The finalization of DDI and CCAFFEINE framework. This issue is not the priority now, so it may not be solved in the near future. The error messages when running GAMESS CCA components with GAMESS/DDI/MPI mode with 4 processes are showed as the follows:

          (14466) /scratch/fangp/cca-tools-0.6.2_rc3/ccaffeine-0.8.2/cxx/dc/user_iface/CmdLineClientMain.cxx: MPI_Finalize being called.
          (14463) /scratch/fangp/cca-tools-0.6.2_rc3/ccaffeine-0.8.2/cxx/dc/user_iface/CmdLineClientMain.cxx: MPI_Finalize being called.
          rank 2 in job 27  t1.tools.scl.ameslab.gov_45684   caused collective abort of all ranks
            exit status of rank 2: killed by signal 9
          make[1]: *** [test] Error 137
          make[1]: Leaving directory `/clusters/scl/fangp/babel-1.0-branch/gamess-component-0.1.3-2/components/examples'
          make: *** [test] Error 2

Integer type mis-match between c/c++ and Fortran 77 for gamess-component package on borges.

When GAMESS source codes compile on LINUX x86_64 architecture, it needs to use some compiler options, such as ”-fdefault-integer-8” for the gfortran compiler and ”-i8” for ifort and pgf77 compilers, for converting integers to 64 bits. Those options are hard-coded in the script and it will run into errors if removing them. However, the gcc or g++ compilers that used to compile components have no this kind of options to turn default integer type to 64 bits. The machine dependent option ”-m64” does not explicitly turn default integer to 64 bits. Therefore, when GAMESS CCA component tries pass an integer reference to GAMESS Fortran 77 wrapper functions, it gives ”index error”. A concrete example is helpful for understanding the problem. For example, in the implementation file ”GAMESS_GaussianBasis_Atomic_Impl.cxx”, the following the FORTRAN 77 wrapper function gamess_katom is called:
```
          int index=1;
          int katom;
          
          gamess_katom( &index, &katom);
     
```
The function header for games_katom is:
```
          void gamess_katom_(int* index, int* iatom);
     
```
The Fortran 77 implementation is:
```
          *-----------------------------------------------------------------
          * Deck nshel ** subroutine gamess_katom
          * Get the atom index for a shell
          *------------------------------------------------------------------
          
                subroutine gamess_katom(index,iatom)
           
          #include "shell.fh"
                if ((index .lt. 0) .and. (index .gt. mxsh)) then
                   write(iw,*) "index = ", index, " is invalid."
                   return
                end if
          
                write(iw,*) "index = ", index
          
                iatom = KATOM(index+1)
                return
                end
     
```
When the variable index was passed as a parameter from CCA interfaces to GAMESS wrapper functions, it printed out a very large number instead of the original value 1 (due to the INDEX error). Solve: The above problem occurs because the gcc and g++ compilers do not set the length of an integer to 64 bits, while Fortran 77 do that implicitly when the corresponding options are specified during the compilation. We need to set the integer type in c/c++ to 64 bits explicitly. Thus, for each variable with integer type that passed to the Fortran 77 wrapper functions, we need to explicitly declare it as ”int64_t” data type. For example,in ”GAMESS_GaussianBasis_Atomic_Impl.cxx”, we had
```
          int64_t index=1;
          int64_t katom;
          gamess_katom( &index, &katom);
     
```
The function header for games_katom is:
```
          void gamess_katom_(int64_t* index, int64_t* iatom);
     
```
However, to make GAMESS CCA component works on borges, the portability has been sacrificed. We need to find a way to solve the integer type mis-match between c/c++ and Fortran 77 without explicitly declare the integer type to ”int64_t” in the component implementations. The strategy we used to solve this issue is to add a flag "-DLINUX64" to compile component implementation files when the architecture is "LINUX64". We also added a macro into the header files under the directory ”legacy/gamess/include”.
```
          #ifdef LINUX64
              typedef int64_t INT_TYPE;
          #else
              typedef int INT_TYPE;
          #endif
     
```
Instead of define an integer as ”int” or ”int64_t” in the component implementation files, we use ”INT_TYPE” so that the interoperability of the integer type between C/C++ and FORTRAN 77 has been taken cared in the header files.