====== Mac OSX CLuster ======
\\
This page explains how I made to install an old Xserve 1.1 cluster, using the Mac Os X 10.4.8 Tiger provided system. Note that I couldn't find any ways to install a Linux on it, because bootcamp is not available on Mac Os X Tiger, and I did not have the Leopard licences. I also tried Refi and Refind boot managers, but the locked EFI used on Xserve 1.1 is in 32 bits when the OS has to be 64 bits. The only way would be to build a grub efi in 32 bits, with a 64 bist system behind. I choose to take my chance with the Mac OS X Tiger, and it was a good choice after all if you keep it simple.

First step is getting a decent GCC suite. The one provided with Xcode 2.5 is unusable.

\\
=====  GCC on Mac Os X =====
\\

How to build / compile a GCC for Mac Os X (on a Xserve 1.1). These instructions are made for Mac Os X Tiger (10.4.x), but should be also working with Leopard and Snow Leopard.

First you need xcode 2.5. The compilers provided by xcode are buggy and not usable, but sufficient to compile another GCC. xcode 2.5 provide gcc 3.3 and gcc 4.0.1 (default).

You can download xcode 2.5 on mac support website (you need to free register first). I used xcode25_8m2558_developerdvd.dmg . Once downloaded, put it on your server, and mount the archive :

<code bash>
hdiutil mount xcode25_8m2558_developerdvd.dmg
</code>

First, clean every other Xcode SDK that could be on your OS :

<code bash>
sudo /Developer/Library/uninstall-devtools --mode=all
</code>

Then, go in :

<code bash>
cd /Volumes/Xcode\ Tools/Packages
</code bash>

And install the package :

<code bash>
sudo installer -verbose -pkg "XcodeTools.mpkg/" -target /
</code bash>

Important : the gcc provided here by xcode do not generate code until ou specify the architecture aimed. gcc myprog.c will ouput only preprocessed code, you need to specify gcc -arch x86_64 myprog.c to get something)
Now, get the infrastructure versions of GMP, MPFR, MPC and the GCC 4.6.4 (ftp://gcc.gnu.org/pub/gcc/infrastructure/), and extract them.

The triky part : (Do not use make -j ? to build faster, it will generate bad builds, no idea why).

<code bash>
./configure --prefix=/soft/system/gmp/4.3.2 CFLAGS='-arch x86_64' CPPFLAGS='-arch x86_64' CXXFLAGS='-arch x86_64' LDFLAGS='-arch x86_64' ABI=64
make
make check
sudo make install

./configure --prefix=/soft/system/mpfr/2.4.2 --with-gmp=/soft/system/gmp/4.3.2 CFLAGS='-arch x86_64' CPPFLAGS='-arch x86_64' CXXFLAGS='-arch x86_64' LDFLAGS='-arch x86_64' ABI=64
make
make check
sudo make install

./configure --prefix=/soft/system/mpc/0.8.1 --with-gmp=/soft/system/gmp/4.3.2 --with-mpfr=/soft/system/mpfr/2.4.2 CFLAGS='-arch x86_64' CPPFLAGS='-arch x86_64' CXXFLAGS='-arch x86_64' LDFLAGS='-arch x86_64' ABI=64
make
make check
sudo make install
</code>

Ok, you get x86_64 libraries. (if not, the system will tell you gcc/i686-apple-darwin8/4.0.1/ld: Undefined symbols: ..., which means your libs are not in x86_64).\\
To build GCC, you need to change few things because xgcc generated do not like flags like -arch needed by the Xcode GCC (in x86_64-apple-darwin8.8.4/libgcc, it will say configure: error: cannot compute suffix of object files: cannot compile, see here http://gcc.gnu.org/wiki/FAQ#configure_suffix).

Export libs :

<code bash>
export LD_LIBRARY_PATH=/soft/system/gmp/4.3.2/lib/:/soft/system/mpfr/2.4.2/lib/:/soft/system/mpc/0.8.1/lib/$LD_LIBRARY_PATH
../gcc-4.6.4/./configure --prefix=/soft/compiler/gcc-4.6.4 --enable-languages=c,c++,objc,obj-c++,fortran --disable-multilib --with-gmp=/soft/system/gmp/4.3.2 --with-mpfr=/soft/system/mpfr/2.4.2 --with-mpc=/soft/system/mpc/0.8.1/ CPPFLAGS='-arch x86_64' LDFLAGS='-arch x86_64' CXX="g++ -arch x86_64" CC="gcc -m64 -arch x86_64" ABI=64
</code>

Ugly, that is right, but it works.

<code bash>
make
make -k check (will fail because of missing system tools).
sudo make install

export LD_LIBRARY_PATH=/soft/compiler/gcc-4.6.4/lib:$LD_LIBRARY_PATH
export PATH=/soft/compiler/gcc-4.6.4/bin/:$PATH
</code>

You made it, GCC x86_64 is now working.

\\
===== OpenMPI =====
\\

Next step is to compile OpenMPI. The only version working on Tiger is the 1.5.5 (1.4.x gives SegFault, and 1.6.x do not support Mac Os X under 10.5).

<code bash>
./configure --prefix=/soft/gcc-4.6.x-soft/system/openmpi/1.5.5 CC=gcc CXX=g++ F77=gfortran FC=gfortran

export PATH=/soft/gcc-4.6.x-soft/system/openmpi/1.4.5/bin/:$PATH
export LD_LIBRARY_PATH=/soft/gcc-4.6.x-soft/system/openmpi/1.4.5/lib/:$LD_LIBRARY_PATH
</code>

And you are good to go.

Last step, to avoid : "malloc.h no such file or directory", go in /usr/include and create a link :

<code bash>
cd /usr/include
sudo ln -s malloc/malloc.h malloc.h
</code>

From now, using hostfiles with OpenMPI, you can use all the cluster in parallel if you copy gcc and openmp bin/libs/etc we built on all nodes.

\\
===== Few more libs =====
\\

==== OpenSSL ====
\\

Version : 0.9.8g

<code bash>
./configure darwin64-x86_64-cc --prefix=/soft/system/openssl/0.9.8g shared
export LD_LIBRARY_PATH=/soft/system/openssl/0.9.8g/lib/:$LD_LIBRARY_PATH
</code>

Note that the compiler use (cc) is the old one from apple. We do this because ot the -arch flag in OpenSSL install not compatible with our GCC. This is not a problem because we use darwin64-x86_64-cc, compatible with our toolchain.

\\
==== libgpg-error ====
\\

Version libgpg-error : 1.9

<code bash>
./configure --enable-static CC=gcc --prefix=/soft/gcc-4.6.x-soft/system/libgpg-error/1.9
vim Makefile
:%s/i386/x86_64/gc
make
</code>

\\
==== libgcrypt ====
\\

Version : 1.5.2

<code bash>
./configure --with-gpg-error-prefix=/soft/gcc-4.6.x-soft/system/libgpg-error/1.9/ --prefix=/soft/gcc-4.6.x-soft/system/libgcrypt/1.5.2
</code>

\\
==== munge ====
\\

Version : 0.5.10

<code bash>
./configure LDFLAGS="-L/soft/system/openssl/0.9.8g/lib/" --prefix=/soft/gcc-4.6.x-soft/system/munge/0.5.10 CC=gcc
export LD_LIBRARY_PATH=/soft/gcc-4.6.x-soft/system/munge/0.5.10/lib/:$LD_LIBRARY_PATH
export PATH=/soft/gcc-4.6.x-soft/system/munge/0.5.10/bin/:$PATH
</code>

<code>
Warning : linking with openssh will result in "invalid credential". Due to openssl, works with libgcrypt.
</code>

Use :

<code bash>
./configure --with-libgcrypt-prefix=/soft/gcc-4.6.x-soft/system/libgcrypt/1.5.2/ --prefix=/soft/gcc-4.6.x-soft/system/munge/0.5.10-gcrypt CC=gcc
</code>

See documentation on munge's web site to use it.

<code bash>
sudo chown -R toto:toto /soft/gcc-4.6.x-soft/system/munge/0.5.10-gcrypt/etc
sudo chown -R toto:toto /soft/gcc-4.6.x-soft/system/munge/0.5.10-gcrypt/var/
sudo chmod 0400 /soft/gcc-4.6.x-soft/system/munge/0.5.10-gcrypt/etc/munge/munge.key
</code>

\\
==== slurm ====
\\

version : 2.2.7 (the only version compatible with our Mac Os X Tiger).
Note : slurm can compile this way, but I got a segfault when launching the daemon. I didn't succeed to make it work. You are warned.

**With OpenSSL :**

<code bash>
./configure LDFLAGS="-L/soft/system/openssl/0.9.8g/lib/ -L/soft/gcc-4.6.x-soft/system/munge/0.5.10/lib" --with-ssl=/soft/system/openssl/0.9.8g --with-munge=/soft/gcc-4.6.x-soft/system/munge/0.5.10 --prefix=/soft/gcc-4.6.x-soft/system/slurm/2.2.7/
</code>

**With LibgCrypt :**

<code bash>
./configure --with-munge=/soft/gcc-4.6.x-soft/system/munge/0.5.10-gcrypt --prefix=/soft/gcc-4.6.x-soft/system/slurm/2.2.7/
</code>