CIS(theta) 2010-2011 - Eureka, we did it! - Meeting XIV

Aim: 

Step 4 is completed!

Attending: 

CIS(theta) 2010-2011: DavidG, RyanH

Absent: 

CIS(theta) 2010-2011: HerbertK, JoshG

Reading:

NA

Research: 

openSSH and openMPI

http://packages.ubuntu.com

public keys
http://cs.calvin.edu/curriculum/cs/374/MPI/ssh.html

MPI intro, nice!
http://www.ualberta.ca/CNS/RESEARCH/Courses/2001/PPandV/Intro_to_MPI.pdf

MPI on Ubuntu
http://ubuntuforums.org/showthread.php?t=1327253

Sample MPI code
http://www.cc.gatech.edu/projects/ihpcl/mpi.html

Thomas Jefferson High courses
http://academics.tjhsst.edu/compsci/parallel/
Thomas Jefferson High paper
http://www.tjhsst.edu/~rlatimer/techlab07/BWardPaperQ3-07.pdf
Thomas Jefferson High ftp
http://www.tjhsst.edu/~rlatimer/techlab07/
Thomas Jefferson High teacher
http://www.tjhsst.edu/~rlatimer/
MPI4py
http://blog.mikael.johanssons.org/archive/2008/05/parallell-and-cluster-mpi4py/

Parallel Python

http://www.parallelpython.com/

IPython

http://ipython.scipy.org/moin/

Large Integer number crunching Mersenne Primes
http://www.hoise.com/primeur/03/articles/weekly/AE-PR-01-04-37.html
Large Integer number crunching Beal Conjecture
http://www.bealconjecture.com/

MPIPOV
http://comp.uark.edu/~ewe/misc/povray.html
POVRAY
http://himiko.dnsalias.net/wordpress/2010/03/29/persistence-of-vision-ray-tracer-povray/
MPI and blender
http://www.blender.org/forum/viewtopic.php?t=10244&view=next&sid=3003d90233a27b81c5093a374c2b0e31
More MPI and blender
http://wiki.vislab.usyd.edu.au/moinwiki/BlenderMods/

InstantCluster Step 4: Software Stack (Completed)

By Jove, I think we've done it! OK, we had a really short meeting today as everyone had to come late due to xtra help and AP exams.... Anyway, we actually got openMPI working last time and didn't even know it! All we had to do was add the IPs to a file called "machines" to get nearly 2700 MFLOPs using 2 cores per node on 3 nodes!

Now, we can scale the cluster up from 3 nodes to all 24 to get over 20 GFLOPs. The trick was to use the 10.10.*.* IPs and not the 10.5.*.* ones. Maybe we could get the 3 servers in on this too for an additional 3 or more GFLOPs? 

BTW, we figured out how to setup the "machines" file by running ClusterByNight first, thanx Kevin Lynagh! That's also where we got the idea to run openMPI over openSSH. Sorry, but PXE boot just didn't work for us! We should also be able to add the octave/mpitb functionality from pelicanHPC as well, thanx Michael Creel!

We only installed gfortran on one box so we could compile flops.f on that node (lets call it the master node):

mpif77 -o flops flops.f

and copied "flops" to all the worker boxes (and used chmod 755 to make it executable) getting about 2650 MFLOPs on 6 cores over 3 nodes:

mpirun -np 6 --hostfile machines flops

What we messed up last time was making the "machines" file based on the 10.5.*.* static IPs we set up on eth1. When we listed all the 10.10.*.* DHCP IPs used on eth0, all worked fine! So, openMPI just defaults to eth0?

Well, it was almost fine. When the remote processes terminated, the master node hung. I think that's because the 10.5.*.* IPs are listed in the authorized_keys file. Maybe we can just edit that and make them 10.10.*.* too?

Well, that's all for now, enjoy!

Happy Clustering,

calcpage@aol.com

http://shop.ebay.com/items/_ti_calculator_active

http://mathforum.org/kb/profile.jspa?userID=32552