Friday, August 7, 2009

Poor Man's Cluster - Step 4

Poor Man's Cluster - Step 4
Topic: LinuxClusters

My students wrote a shell script called chopsplit.sh to scatter/gather the jobs.  We rendered several ray tracings using povray with this script (see zazzle site listed in my sig).  This file, chopsplit.sh, has to be in /home/cluster on the master node with permissions 755 (use chmod as root to change this).  You can write the following script in any text editor:
#!/bin/bash
#Arguments should be:
#pov file
#width
#height
#output file
#machines
if [[ $# -lt 5 ]]; then
echo "Usage: $0 scene width height output machines"
exit
fi
prefix=/home/cluster/povray-3.6/
imgdir=/home/cluster/distimg/
myip=`ip addr show | grep inet\ 10.5.129 | cut -f6- -d\ | cut -f-1 -d/`
if [[ ${1:0:1} == / ]]; then
scene=$1
else
scene=${1#scenes/}
fi
width=$2
height=$3
output=${4%.ppm}".ppm"
shift 4
tprocs=$(( $#*2 ))
procs=0
rm -f ${imgdir}/*
for a in $@; do
addr="10.5.129."${a#10.5.129.}
addr=${addr/10.5.129.self/127.0.0.1}
addr=${addr/10.5.129.localhost/127.0.0.1}
addr=${addr/10.5.129.127.0.0.1/127.0.0.1}
echo Starting $addr-1
ssh -x cluster@$addr "$prefix/povray +i$prefix/scenes/$scene -v -d -p +fp +w$width +h$height +sr$((procs*height/tprocs+1)) +er$(((procs+1)*height/tprocs)) +a0.3 +L$prefix/include +L$prefix/scenes/${scene%/*} +o- 2>/dev/null | chop - - $((9+${#width}+${#height})) && echo $addr-3 done 1>&2" >$imgdir/3$a${output} &
sizes[procs]=$((((procs+1)*height/tprocs-procs*height/tprocs)*3*width))
procs=$((procs+1))
echo Starting $addr-2
ssh -x cluster@$addr "$prefix/povray +i$prefix/scenes/$scene -v -d -p +fp +w$width +h$height +sr$((procs*height/tprocs+1)) +er$(((procs+1)*height/tprocs)) +a0.3 +L$prefix/include +L$prefix/scenes/${scene%/*} +o- 2>/dev/null | chop - - $((9+${#width}+${#height})) && echo $addr-4 done 1>&2" >$imgdir/4$a${output} &
sizes[procs]=$((((procs+1)*height/tprocs-procs*height/tprocs)*3*width))
procs=$((procs+1))
# echo $addr
done
echo ${sizes[@]}
procs=0
echo -e "P6\n$width $height\n255" > $output
for a in $@; do
until [[ `du -b $imgdir/3$a${output} 2>/dev/null | cut -f-1` -eq ${sizes[procs]} ]]; do
sleep 1
done
echo $a$output-3
cat $imgdir/3$a${output} >> $output
procs=$((procs+1))
until [[ `du -b $imgdir/4$a${output} 2>/dev/null | cut -f-1` -eq ${sizes[procs]} ]]; do
sleep 1
done
echo $a$output-4
cat $imgdir/4$a${output} >> $output
procs=$((procs+1))
done
echo $output built successfully!
Good Luck,

Poor Man's Cluster - Step 4

Poor Man's Cluster - Step 4
Topic: LinuxClusters

My students wrote a shell script called chopsplit.sh to scatter/gather the jobs.  We rendered several ray tracings using povray with this script (see zazzle site listed in my sig).  This file, chopsplit.sh, has to be in /home/cluster on the master node with permissions 755 (use chmod as root to change this).  You can write the following script in any text editor:
#!/bin/bash
#Arguments should be:
#pov file
#width
#height
#output file
#machines
if [[ $# -lt 5 ]]; then
echo "Usage: $0 scene width height output machines"
exit
fi
prefix=/home/cluster/povray-3.6/
imgdir=/home/cluster/distimg/
myip=`ip addr show | grep inet\ 10.5.129 | cut -f6- -d\ | cut -f-1 -d/`
if [[ ${1:0:1} == / ]]; then
scene=$1
else
scene=${1#scenes/}
fi
width=$2
height=$3
output=${4%.ppm}".ppm"
shift 4
tprocs=$(( $#*2 ))
procs=0
rm -f ${imgdir}/*
for a in $@; do
addr="10.5.129."${a#10.5.129.}
addr=${addr/10.5.129.self/127.0.0.1}
addr=${addr/10.5.129.localhost/127.0.0.1}
addr=${addr/10.5.129.127.0.0.1/127.0.0.1}
echo Starting $addr-1
ssh -x cluster@$addr "$prefix/povray +i$prefix/scenes/$scene -v -d -p +fp +w$width +h$height +sr$((procs*height/tprocs+1)) +er$(((procs+1)*height/tprocs)) +a0.3 +L$prefix/include +L$prefix/scenes/${scene%/*} +o- 2>/dev/null | chop - - $((9+${#width}+${#height})) && echo $addr-3 done 1>&2" >$imgdir/3$a${output} &
sizes[procs]=$((((procs+1)*height/tprocs-procs*height/tprocs)*3*width))
procs=$((procs+1))
echo Starting $addr-2
ssh -x cluster@$addr "$prefix/povray +i$prefix/scenes/$scene -v -d -p +fp +w$width +h$height +sr$((procs*height/tprocs+1)) +er$(((procs+1)*height/tprocs)) +a0.3 +L$prefix/include +L$prefix/scenes/${scene%/*} +o- 2>/dev/null | chop - - $((9+${#width}+${#height})) && echo $addr-4 done 1>&2" >$imgdir/4$a${output} &
sizes[procs]=$((((procs+1)*height/tprocs-procs*height/tprocs)*3*width))
procs=$((procs+1))
# echo $addr
done
echo ${sizes[@]}
procs=0
echo -e "P6\n$width $height\n255" > $output
for a in $@; do
until [[ `du -b $imgdir/3$a${output} 2>/dev/null | cut -f-1` -eq ${sizes[procs]} ]]; do
sleep 1
done
echo $a$output-3
cat $imgdir/3$a${output} >> $output
procs=$((procs+1))
until [[ `du -b $imgdir/4$a${output} 2>/dev/null | cut -f-1` -eq ${sizes[procs]} ]]; do
sleep 1
done
echo $a$output-4
cat $imgdir/4$a${output} >> $output
procs=$((procs+1))
done
echo $output built successfully!
Good Luck,

Thursday, August 6, 2009

Poor Man's Cluster - Step 3

Poor Man's Cluster - Step 3
Topic: LinuxClusters

OK, at this point what we did was make a new user on each and every node just for us clustering types!  I login to each node (PC) as root and use adduser/userdel command lines.  This user has the same userid, say "cluster," and passwd on every node.
Then we set up ssh public key authentication for each of these new users as detailed here:
With this setup, you need only login as "cluster" on one linuxbox and you can ssh into any other linuxbox with the same userid/passwd and you don't have to specify either the userid or the passwd!  Its like you are logged into all 25 PCs at once!  This makes it easy to send jobs from the PC you are logged into (the master node) and recieve results from all the other nodes (the worker nodes) whenever we need to scatter/gather jobs.
Until next time,

Poor Man's Cluster - Step 3

Poor Man's Cluster - Step 3
Topic: LinuxClusters

OK, at this point what we did was make a new user on each and every node just for us clustering types!  I login to each node (PC) as root and use adduser/userdel command lines.  This user has the same userid, say "cluster," and passwd on every node.
Then we set up ssh public key authentication for each of these new users as detailed here:
With this setup, you need only login as "cluster" on one linuxbox and you can ssh into any other linuxbox with the same userid/passwd and you don't have to specify either the userid or the passwd!  Its like you are logged into all 25 PCs at once!  This makes it easy to send jobs from the PC you are logged into (the master node) and recieve results from all the other nodes (the worker nodes) whenever we need to scatter/gather jobs.
Until next time,

Thursday, July 30, 2009

Poor Man's Cluster - Step 2

Poor Man's Cluster - Step 2
Topic: LinuxClusters

OK, so we chose the KNOPPIX DVD last year (2008-2009 school year).  We downloaded the latest version from http://www.knoppix.net/  This is a very nice Linux distribution based on Debian (http://www.debian.org/distrib/packages) making for a very good bootable or live Linux DVD with great hardware autodetection.  Here are the steps involved in simply installing the DVD (no cluster apps yet) to a Linux partition on the hard drive of each PC you want in the cluster:
(0) boot dvd, open root shell, run "qtparted" to adjust partitions as desired.
(1) now run "knoppix-installer" from the same root shell and follow the instructions given in the script.
(2) reboot PC and put dvd away
(3) login as root - single by following the next 3 steps at boot-up:
in grub select top then press
select kernel then press
type "single" at end of line then press

(3a) to edit bootloader run: "joe /boot/grub/menu.1st"
(3b) then run "netcardconfig" with no dhcp, selecting ethernet card for Linux network (we had one for Linux off the internet and one for Windows on the internet) - here are the settings I used (edit according to your local network, or intranet,  setup):
address = 10.5.129.1 (this varies by station, I went up to 10.5.129.25)
netmask = 255.255.0.0
network = 10.5.129.0
broadcast = 10.5.255.255
gateway = 10.5.0.254
dns = 10.1.1.9, 10.1.1.3, 10.1.1.5
(3c) now run "joe /etc/network/interfaces" and edit accordingly making sure ONLY lo and the ethernet card you selected are listed - NO hotplug - NO eth0 (if you selected eth1 for example)
(3d) we also need to run "update-rc.d ssh defaults" to set up a ssh server (sshd) on each PC or cluster node (we need this for cluster apps to talk to eachother).
(4) this step is optional - we needed to install ati drivers for graphics acceleration - we may not need this next year with redhat - run the following commands as root:
(4a) ftp 10.5.254.245/ATI/*.run (anonymously)
(4b) chmod 755 *.run
(4c) Xorg -configure
(4d) test line given and exit X with
(4e) cp /xorg.conf.new /etc/X11/xorg.conf
(4f) ./ati-driver-installer-8-11-x86.x86_64.run
(4g) aticonfig -initial (twice)
(4h) reboot
(5) login as regular user (we used "apcs" in knoppix-installer) and set-up local printers using KWrite and test graphics accelorator with glxinfo, glxgears, tuxracer, bzflag....
Wow, that's all folks!
HTH,

Poor Man's Cluster - Step 2

Poor Man's Cluster - Step 2
Topic: LinuxClusters

OK, so we chose the KNOPPIX DVD last year (2008-2009 school year).  We downloaded the latest version from http://www.knoppix.net/  This is a very nice Linux distribution based on Debian (http://www.debian.org/distrib/packages) making for a very good bootable or live Linux DVD with great hardware autodetection.  Here are the steps involved in simply installing the DVD (no cluster apps yet) to a Linux partition on the hard drive of each PC you want in the cluster:
(0) boot dvd, open root shell, run "qtparted" to adjust partitions as desired.
(1) now run "knoppix-installer" from the same root shell and follow the instructions given in the script.
(2) reboot PC and put dvd away
(3) login as root - single by following the next 3 steps at boot-up:
in grub select top then press
select kernel then press
type "single" at end of line then press

(3a) to edit bootloader run: "joe /boot/grub/menu.1st"
(3b) then run "netcardconfig" with no dhcp, selecting ethernet card for Linux network (we had one for Linux off the internet and one for Windows on the internet) - here are the settings I used (edit according to your local network, or intranet,  setup):
address = 10.5.129.1 (this varies by station, I went up to 10.5.129.25)
netmask = 255.255.0.0
network = 10.5.129.0
broadcast = 10.5.255.255
gateway = 10.5.0.254
dns = 10.1.1.9, 10.1.1.3, 10.1.1.5
(3c) now run "joe /etc/network/interfaces" and edit accordingly making sure ONLY lo and the ethernet card you selected are listed - NO hotplug - NO eth0 (if you selected eth1 for example)
(3d) we also need to run "update-rc.d ssh defaults" to set up a ssh server (sshd) on each PC or cluster node (we need this for cluster apps to talk to eachother).
(4) this step is optional - we needed to install ati drivers for graphics acceleration - we may not need this next year with redhat - run the following commands as root:
(4a) ftp 10.5.254.245/ATI/*.run (anonymously)
(4b) chmod 755 *.run
(4c) Xorg -configure
(4d) test line given and exit X with
(4e) cp /xorg.conf.new /etc/X11/xorg.conf
(4f) ./ati-driver-installer-8-11-x86.x86_64.run
(4g) aticonfig -initial (twice)
(4h) reboot
(5) login as regular user (we used "apcs" in knoppix-installer) and set-up local printers using KWrite and test graphics accelorator with glxinfo, glxgears, tuxracer, bzflag....
Wow, that's all folks!
HTH,

Saturday, June 20, 2009