The IBM clusters have run serveral operating systems over its lifetime. The dual-processor nodes started out with AIX v4.3.3 while the quad processor nodes started out with AIX v5.1L. These have now been completely replaced with Linux on all nodes.
The current core OS is Linux. The current setup is based on the Debian PPC Linux distribution. To this we have added a 64 bit kernel (currently 2.6.6 from the Ames bitkeeper repository) and a 64 bit toolchain in parellel with the Debian 32 bit code ( in /opt). The 64 bit toolchain include 64 bit versions of gcc/g77 (v3.2.3) along with supporting libraries. In addition the IBM compilers (xlc/vac/xlf) are capable of producing either 32bit or 64bit (add -q64) executables. All compute nodes are booted via the network and the root file system is served out over NFS.