Now that master server is installed, it is time to configure the basic io servers, using NFS.
To do this, you will need to install the io server with Centos 7.2 also, and configure repository. You can install this server manually (USB stick, DVD, etc), or start like a compute node: install it using PXE.
Have a look at managing cluster chapter on how to install a server using PXE.
First thing to do on the nfs node is to configure network on it. We will use a static ip for io nodes.
Go to /etc/sysconfig/network-script, and edit ifcfg-enp0s3 to the following:
DEVICE="enp0s3" NAME="enp0s3" TYPE="Ethernet" NM_CONTROLLED=no ONBOOT="yes" BOOTPROTO="static" IPADDR="10.0.1.1" NETMASK=255.255.0.0
Now restart network to activate the interface:
systemctl restart network
Lets disable NetworkManager, and Firewall:
systemctl stop NetworkManager systemctl stop firewalld systemctl disable NetworkManager systemctl disable firewalld
We need to setup repository client, to be able to install packages.
From master (batman), just upload repository file to the nfs node:
scp /etc/yum.repos.d/os_base.local.repo email@example.com:/etc/yum.repos.d/os_base.local.repo
For record, file was:
[os_base.local] name=os_base.local.repo baseurl=http://10.0.0.1/repo/os_base.local.repo gpgcheck=0 enabled=1
Login again on nfs node,
Also remove CentOs original repository files, but for safety, backup them, and update yum :
mkdir /etc/yum.repos.d.old/ mv /etc/yum.repos.d/CentOS-* /etc/yum.repos.d.old yum clean all yum update
This server will host the /home and /hpc-software of the whole cluster. /home is the main users directory, where they will be able to upload and compile their code, and also set their configuration. It will be accessible on each login and compute nodes. /hpc-software contains all shared tools and libraries to be available on each compute and login nodes.
Install needed packages:
yum install nfs-utils
/home should already be there, make /hpc-software dir:
Then, edit nano /etc/exports to tell nfs server which directories to export:
/hpc-softwares 10.0.0.0/16(rw,no_root_squash,sync) /home 10.0.0.0/16(rw,no_root_squash,sync)
And start nfs server:
systemctl enable rpcbind systemctl enable nfs-server systemctl start rpcbind systemctl start nfs-server
That's all for basic NFS server. You can tune it if needed, or proceed to compute nodes installation.
Edit /etc/default/nfs-kernel-server and add more workers (to handle more connections) :
# Number of servers to start up RPCNFSDCOUNT=8
Then restart nfs.
Increase buffer sizes on client side, especially for large files manipulations networks. In fstab (on compute/login nodes), use rsize=524288,wsize=524288 for mount options.
Use nfsV3 instead of v4. In fstab, add vers=3 for mount options.
Increase MTU (see network). This parameter is very important as it has a major impact on performances (9000 is a good start for Ethernet, and 65536 for IPoverIB Infiniband).