CIS(theta)Computing Independent Study2009-2011: CIS(theta) Meeting XII (2010-2011)

We had a great meeting this week! We finally broke down and decided to install our own software stack for MPI on our existing 32bit Ubuntu 10.04 Desktop! We reviewed what works on our LAN (openSSH) and what doesn't work (PXE). Maybe we can add the functionality from pelicanHPC such as Octave and MPITB? We could even write our own C++ code for fractals. What about mpi4py or parallel python?

We can learn from our experience this year. I think we need to get openMPI working over openSSH using public-key authentication (as detailed below). Then we can run povray or blender or even our own C++ applications on a more permanent cluster installation! So, for example, its MPIPOV over openMPI over openSSH with public keys over Ubuntu over 64bit dual-core AMD Athlons with 750MB RAM over gigE, right? Here's what we discussed.

InstantCluster Step 1: Infrastructure

Make sure your cores have enough ventilation. The room has to have powerful air conditioning too. These two factors may seem trivial but will become crucial when running the entire cluster for extended periods of time! Also, you need to have enough electrical power, preferably with cabling out of the way, to run all cores simultaneously. Don't forget to do the same with all your Ethernet cabling. We have CAT6E cables to support our gigE Ethernet cards and switches. We are lucky that this step was taken care of for us already!

InstantCluster Step 2: Hardware

You need some up to date fast Ethernet switches plus Ethernet cards and cores as well as plenty of RAM in each Linux box. As stated above, our gigE LAN was setup for us. Also, we have 64bit dual-core AMD Athlons and our HP boxes have 750 MB of RAM. I'd rather 1 or 2 GB of RAM, but that will have to wait for an upgrade!

InstantCluster Step 3: Firmware

We wasted way too much time last year trying out all kinds of Linux distros looking for a good 64bit base for our cluster. This year we spent way too much time testing out different liveCD distros. Recently, we downgraded from 64bit Ubuntu 10.04 Desktop edition to the 32bit version on our Linux partitions. 64bit gives us access to more RAM and larger maxint, but was proving to be a pain to maintain. Just to name one problem, jre and flash were hard to install and update on FireFox. Last year we tried Fedora, Rocks, Oscar, CentOS, Scientific Linux and, finally, Ubuntu. 32bit Ubuntu has proven very easy to use and maintain, so I think we'll stick with it for the cluster!

InstantCluster Step 4: Software Stack

On top of Ubuntu we need to add openSSH, public-key authentication and openMPI. In step 5 we can discuss an application to scatter/gather over the cluster whether it be graphical (fractals, povray, blender, openGL) or number crunching (C++ or python app for Mersenne Primes or Beal's Conjecture). So, what follows is a summary of what we did to get up to plublic-key authentication. We didn't have time to do more than that, but we made some good progress getting this much working so far. This summary is based on the http://cs.calvin.edu/curriculum/cs/374/MPI/ link listed above. I added some notes in [...]. First, we installed openSSH-server from http://packages.ubuntu.com using our proxy server, then:

If you have no .ssh directory in your home directory, ssh to some other machine in the lab; then Ctrl-d to close the connection, creating .ssh and some related files.
From your home directory, make .ssh secure by entering:
```
chmod 700 .ssh
```
Next, make .ssh your working directory by entering:
```
cd .ssh
```
To list/view the contents of the directory, enter:
```
ls -a [we used ls -l]
```
To generate your public and private keys, enter:
```
ssh-keygen -t rsa
```
The first prompt is for the name of the file in which your private key will be stored; press Enter to accept the default name (id_rsa).The next two prompts are for the password you want, and since we are trying to avoid entering passwords, just press Enter at both prompts, returning you to the system prompt.
To compare the previous output of ls and see what new files have been created, enter:
```
ls -a [we used ls -l]
```
You should see id_rsa containing your private key, and id_rsa.pub containing your public key.

To make your public key the only thing needed for you to ssh to a different machine, enter:

cat id_rsa.pub >> authorized_keys

[The Linux boxes on our LAN, soon to be cluster, have IPs ranging from 10.5.129.1 to 10.5.129.24 So, we copied each id_rsa.pub file to temp01-temp24 and uploaded these files to our ftp server. On the ftp server we just ran cat tempnn >> authorized_keys for each temp file to generate one master authorized_keys file for all nodes that we could just download to each node's .ssh dir.]

[optional] To make it so that only you can read or write the file containing your private key, enter:
```
chmod 600 id_rsa
```
[optional] To make it so that only you can read or write the file containing your authorized keys, enter:
```
chmod 600 authorized_keys
```