Showing posts with label hpc. Show all posts
Showing posts with label hpc. Show all posts

Friday, December 20, 2013

CIS(theta) 2013-2014 December Meeting Firing on all cores!


CIS(theta) 2013-2014 

December Meeting
Firing on all cores!
Today we booted up the entire LAN using the pelicanHPC 64bit 2.9 with mpi4py DVD! We finally got up to 24 nodes with 96 cores and nearly 50 GFLOP/s!! Great job guys!!! Now we have to decide if we are going to install a native MPI stack. We also need to research sample MPI code online to learn how to program the cluster in FORTRAN77, C, C++ and python. We used the code above, which estimates pi using Reimann Sums and arctan(x), to stress the cluster. BTW, we don't have 100 cores as my PC is on another logical subnet and its only dual core anyway.
To compile:
mpif77 -o flops flops.f
To execute
mpirun -np 96 --hostfile ~/tmp/bhosts flops


November Meeting II
Firing up the cluster for the first time!
We are trying to figure out if we can use a bootable Linux CD to boot up the cluster or if we want to add an MPI stack with public key authenticated sshd to our Ubuntu Desktop. So, we downloaded the pelicanHPC ISO, burned a DVD and got 17 compute nodes or 68 cores running at about 35 GFLOP/s! 

Hey, that's not too shabby, just 13 years ago, top500.org reports that all you needed were 50 cores running at 55 MFLOP/s to make the list of one of the 500 fastest clusters in the world! We may even get up to 50 MFLOP/s if we get all 100 cores up and running. 

So, we're looking at pelicanHPCBCCD and ClusterByNight for MPI. We used to run clusterKNOPPIXBCCD and Quantian with openMOSIX. Too bad openMOSIX isn't supported anymore. It was so easy to code for MOSIX. In C or C++ all you had to do was use the fork() function to start a new process. openMOSIX ran in the Linux Kernel. Whenever your processes came close to using 100% of you CPU, MOSIX would automagically send one of your forked jobs to another processor. When we started using MOSIX, we only had one core per PC, so that meant send the job to another PC on the LAN. In those days (late 1990s) we managed to run 100 cores anyway by taking over 4 PC Labs on my floor at school....



November Meeting I
Last Linux Install Fest!
We finally got the new 64bit Ubuntu 13.10 Saucy Salamader installed correctly on all student PCs. I'll upgrade my station at a later date. We got this down to a science: 
(1a) download latest ISO and burn 8 DVDs
(1b) install DVD with plain vanilla defaults
(2) edit eth0 connection
(3a) edit system settings/network/proxy
(3b) edit system settings/appearance
(3c) edit system settings/brightness
(3d) edit system/settings/printers
(4) edit sources to main in USC
(5a) sudo apt-get update
(5b) sudo apt-get upgade
(6a) USC install WINE and TIEMU
(6b) add TI89 and TI92 ROMs to TIEMU
(7a) download, extract and install TilEm
(7b) add TI83 and TI84 ROMs to TilEm
(8a) downlaod and extract SAGE.lzma
(8b) echo "alias sage=$HOME/Desktop/SAGE/sage" >> ~./bashrc
(8c) source ~/.bashrc
(9) all we left on the desktop was the SAGE directory, a short cut to TilEm and our handy dandy Trig Table (see below)
(10a) to use TilEm, the students just click on the short cut
(10b) to use TIEMU and SAGE open a terminal and type "tiemu" or "sage" on a command line.
(11) I'll add TilEm and TIEMU to the start-up menu. I was going to add "sage -n) too, but I don't want FireFox hogging all my Desktops at bootup!

October Meeting II
Linux Install Fest Again!
I downloaded 8 copies the latest version of Ubuntu Desktop 32bit ISO: 13.10 Saucy Salamander. Then I burned 8 DVDs and reinstalled the Linux partition on the second row of 8 PCs in our classroom. The geek squad did all the tweaks this time! 

We are having a massive problem with VTI running under WINE. It's so slow as to be unusable. We have to figure out a solution. Maybe 13.04 is better? Or maybe I can get WINE's direct ppa and use their latest version?

Another problem is that we have a SAGE directory on the Desktop extracted from the latest compiled version of SAGE. In there we have a SAGE bash script. I usually make it executable and students can just click it to run SAGE. In 13.10, the script will not execute, the script just opens in gedit even though I made the file executable. I even ran "which bash" to see if bash moved (it did) so I could update the #! line but still no joy!

How about an alias like:
echo "alias sage=$HOME/Desktop/SAGE/sage" >> ~/.bashrc

then:
source ~/.bashrc

October Meeting I
Linux Install Fest!
We downloaded 8 copies the latest version of Ubuntu Desktop 64bit ISO: 13.10 Saucy Salamander. Then we burned 8 DVDs and reinstalled the Linux partition on the first row of 8 PCs in our classroom. I did all the tweaks the following day.

Next week I'll reinstall the 2nd row and the Geek Squad will do all the tweaks. I like to keep the Student Stations very simple. So, we will only tweak as follows:
(1) configure Network (eth0, proxy)
(2) configure Appearence and Brightness 
(3) configure Printers
(4) sudo apt-get update (after setting sources to main).
(5) sudo apt-get upgrade
(6) install WINE
(7) copy VTI to the desktop
(8) extract SAGE to the desktop

September Meeting I
Administrativa!
(1) Wreath of the Unknown Server: We visited our first ssh server, Colossus, which is still in the switch room though dormant. I set it up for the first time in 1995 running Slackware Linux. Colossus ran for 12 years straight, 24x7 never having to shut down, reboot or even have anything re-installed!

(2) Display Case Unveiled: We took down a ton of fractal prints and ray tracings from Room 429 to the 2 cases on the 1st floor near the art wing. We decorated both cases as best we could and left before anyone saw us. Must have been gremlins.

(3) Recruiting: We decided that we have more than enough qualified CIS(theta) members for this year's Geek Squad, so we tabled that topic.

(4) Planning: We have to wait another 2 weeks 10/4 at which point Ubuntu 13.10 Desktop Edition should be available for a mini install fest. After that, we may use bootable cluster Linux CD distros such as BCCD and pelicanHPC to learn MPI using C++ or Python. We also talked about installing an MPI stack on each hdd along with public key authenticated ssh. We would like to make a fractal zoom animation.

(5) Summary: This year's CIS(theta) team is off to a good start. Shadowfax, our 100 core cluster, is in good hands!
==================================
What we are researching IV (Jan):MPI
==================================
What we are researching IV (Dec):MOSIX
==================================
What we are researching III (Nov)
==================================
What we are researching II (Oct)
==================================
What we are researching I (Sept)
Thomas Jefferson High courses
Thomas Jefferson High paper
Thomas Jefferson High ftp
Thomas Jefferson High teacher
==================================
Daily Attendance:
CIS(theta) 2013-2014: Richard, Tyler absent

BiWeekly Topic:
CIS(theta) 2013-2014 - Firing up all cores! 

Monthly Reading:
Chapter 5: Building Parallel Programs (JAN)
Chapter 4: Building Parallel Programs (DEC)
Chapter 3: Building Parallel Programs (NOV)
Chapter 2: Building Parallel Programs (OCT)
Chapter 1: Building Parallel Programs (SEP)
==================================
Membership (alphabetic by first name):

CIS(theta) 2013-2014: BryanS, CheyenneC, DanielG, HarineeN, RichardH, RyanW, TatianaR, TylerK

CIS(theta) 2012-2013: Kyle Seipp

CIS(theta) 2011-2012: Graham Smith, George Abreu, Kenny Krug, LucasEager-Leavitt

CIS(theta) 2010-2011: David Gonzalez, Herbert Kwok, Jay Wong, Josh Granoff, Ryan Hothan

CIS(theta) 2009-2010: Arthur Dysart, Devin Bramble, Jeremy Agostino, Steve Beller

CIS(theta) 2008-2009: Marc Aldorasi, Mitchel Wong

CIS(theta) 2007-2008: Chris Rai, Frank Kotarski, Nathaniel Roman

CIS(theta) 1988-2007: A. Jorge Garcia, Gabriel Garcia, James McLurkin, Joe Bernstein, ... too many to mention here!
==================================
CIS(theta) DropBox!
==================================
Well, that's all folks!

Monday, January 14, 2013

CIS(theta) 2012-2013: January Update #1!


Computing Independent Study 2012-2013



JANUARY UPDATE #1
We tried to isolate our LAN from other DHCP servers but could not. So we still only have individual 3-core SMP boxes running at about 1.2 GFLOPs. We also tried our hand at mpi4py, but we have a way to go in that department too!
UPDATE: mpi4py was left out of the current pelicanHPC by mistake! No wonder mpi4py didn't work! Look here:
http://pelicanhpc.788819.n4.nabble.com/pelicanhpc-2-9-amd64-and-mpi4py-td4650446.html



DECEMBER UPDATE
We finally had a chance to meet this month in our new PC Lab! We got pelicanHPC to run 3 cores at about 1.2 GFLOPS. We have AMD Phenom IIs. These CPUs are supposed to be quad-cores. It seems one core is dead. Even so, that's not a bad start. However, we could only run pelican in SMP mode. We could not PXE boot any other nodes. We are also looking into Flame Fractals.
********************

NOVEMBER UPDATE
Sorry to say that we had no meetings this month. I was hoping for 3 meetings, but Hurricane Sandy changed everything! We are in a new room where we may be able to try out liveLinux CD based MPI clusters next month. Stay tuned!
********************

OCTOBER UPDATE #2
That was a great meeting today! We burned 8 CDs of Precise Pangolin and reinstalled the whole back row of our PC LAB/Classroom. Many thanx go to Jeremy for coming to vist today and lending a hand! 
RELATED POST: http://shadowfaxrant.blogspot.com/2012/11/pc-lab-period-of-reconstruction-at.html

Here's the steps we followed for a minimal install of the Student Stations (64bit Athlons):
Step1: 
Reboot each Linux box with the current  CD. Answer some basic questions about time zone, userid, passwd, no login on bootup, etc.
Step2:
Reboot each Linux box without the CD. Make sure to configure the gigE cards and proxy server:
IP: 10.5.129.x
NetMask:255.255.0.0
GateWay: 10.5.0.254
DNS: 10.5.0.254
Proxy: 10.0.0.125
Step3:
Configure System Settings as desired (unit circle trig calculator background, no screensaver, etc).
Step4: 
We had to switch Software Sources in the Ubuntu Software Center (edit/source) to Main before this would work:
sudo apt-get update
sudo apt-get upgrade
Step5:
Now, we could use the Ubuntu Software Center to install WINE.
Step6:
I copied my VTI83 and VTI89 directories from my memory stick to the Desktop. Then, after editing preferences to have VTI open with WINE, I configured each calculator.
Step7:
I will edit my /etc/crontabs tomorrow....
Step8:
We haven't decided what else we may have to install (local SAGE server, JRE, openSSH, openMPI,etc). We'll have to think about that! Here's some info on install fests from prior years:
http://shadowfaxrant.blogspot.com/2011/05/2-so-many-hard-drives-so-little-time.html
http://shadowfaxrant.blogspot.com/2011/01/then-god-mage-midterm-week-and-saw-that.html
http://shadowfaxrant.blogspot.com/2010/06/so-many-linux-distros-so-little-time.html
********************


OCTOBER UPDATE
We decided to try out the new Ubuntu Linux 64bit Desktop 12.04 nicknamed Precise Pangolin. So we surfed on over to http://www.ubuntu.com and downloaded the latest ISO. We burned the CD, rebooted a guinea pig box and reinstalled it. This should be a simple procedure as we no longer use dualboot or dualnic boxes. However, we ran into a SNAFU right away! Intranet gigE works fine, but we can't get on the Internet? OOPs, we forgot the network proxy. If at first you don't succeed, try, try again!

Ubuntu Release History
4.10 Warty Warthog (mammal)
5.04 Hoary Hedgehog (mammal)
5.10 Breezy Badger (mammal)
6.06 Dapper Drake (bird)
6.10 Edgy Eft (amphibian)
7.04 Feisty Faun (mammal)
7.10 Gutsy Gibbon (mammal)
8.04 Hardy Heron (bird)
8.10 Intrepid Ibex (mammal)
9.04 Jaunty Jackalope (mythical beast)
9.10 Karmic Koala (mammal)
10.04 Lucid Lynx (mammal) 
10.10 Maverick Meerkat (mammal) 
11.04 Natty Narwahl (mammal) 
11.10 Oneiric Ocelot (mammal) 
12.04 Precise Pangolin (mammal) 
12.10 Quantal Quetzal (bird) release: 10/18

Guardian, our ssh server, is running 10.04 32bit. Guardian has a dualcore 32bit intel Xeon processor with 2GB RAM and a 512GB RAID drive.

Caprica, our ftp server, is running 10.04 32bit. Caprica has a dualcore 32bit intel Xeon processor with 2GB RAM and a 512GB RAID drive.

Shadowfax, our teacher station, is running 11.10 32bit. Shadowfax has a dualcore 64bit amd Athlon processor with 2GB RAM and a 256GB hdd. We use a 32bit OS here as SmartNotebook doesn't run on 64bit....

Alpha-Omega, our student stations, are running 11.04 64bit. These Linux boxes, like Shadowfax, have dualcore 64bit amd Athlon processors with 2GB RAM and a 256GB hdd.

We are only upgrading Alpha-Omega to 12.04 (or 12.10 if it's available when we upgrade in a couple of weeks). We are also waiting for a hardware upgrade for Alpha-Omega to amd quadcore Phenoms!
********************

SEPTEMBER UPDATE
We had our traditional first organizational meeting:

(1) Wreath of the Unknown Server: 
We visited our first ssh server, Colossus, which is still in the switch room though dormant. I set it up for the first time in 1995 running Slackware Linux. Colossus ran for 12 years straight, 24x7 never having to shut down, reboot or even have anything re-installed! Colossus would not die. We finally just replaced Colossus with a dual-core Intel Xeon box complete with a RAID drive running 1TB. Old Linux boxes never die, they just fade away...


(2) Display Case Unveiled: 
We took down a ton of fractal prints and ray tracings from Room 429 to the 2 cases on the 1st floor near the art wing. We decorated both cases as best we could and left before anyone saw us. Must have been gremlins.

(3) Recruiting 2012: 
We decided that we did not have a good pool of candidates to recruit more CIS(theta) members for this year's Geek Squad, so we tabled that topic.

(4) Planing 2012: 
Next meeting would have been 9/28 but that's Yum Kipur. So, we have to wait another 2 weeks after that for 10/10 at which point Ubuntu Precise Pangolin 64bit release 12.10 Desktop Edition should be available for a mini install fest. After that, we may use bootable cluster Linux CD distros to learn MPI.
==================================
What we are researching I (Sept)
(look what this school did in the 80s): 
Thomas Jefferson High courses
http://academics.tjhsst.edu/compsci/parallel/
Thomas Jefferson High paper
http://www.tjhsst.edu/~rlatimer/techlab07/BWardPaperQ3-07.pdf
Thomas Jefferson High ftp
http://www.tjhsst.edu/~rlatimer/techlab07/
Thomas Jefferson High teacher
http://www.tjhsst.edu/~rlatimer/


What we are researching II (Oct)
(clustering environments): 
Parallel Virtual Machine
http://www.csm.ornl.gov/pvm/
openMOSIX
http://openmosix.sourceforge.net/instant_openmosix_clusters.html
Message Passing Interface
http://www.open-mpi.org/


What we are researching III (Dec)
(instant MPI clusters via liveCDs): 
Cluster By Night
BCCD
pelicanHPC
Flame Fractals
http://en.wikipedia.org/wiki/Fractal_flame


What we are researching IV (Jan)
==================================
Today's Topic:
CIS(theta) 2012-2013 - mpi4py


Today's Attendance:
CIS(theta) 2012-2013: Kyle Seipp


Today's Reading:
Chapter 4: Building Parallel Programs (BPP) using clusters and parallelJava
==================================
Membership (alphabetic by first name):
CIS(theta) 2012-2013: 
Kyle Seipp

CIS(theta) 2011-2012: 
Graham Smith, George Abreu, Kenny Krug, LucasEager-Leavitt

CIS(theta) 2010-2011: 
David Gonzalez, Herbert Kwok, Jay Wong, Josh Granoff, Ryan Hothan

CIS(theta) 2009-2010: 
Arthur Dysart*, Devin Bramble, Jeremy Agostino, Steve Beller

CIS(theta) 2008-2009: 
Marc Aldorasi, Mitchel Wong*

CIS(theta) 2007-2008: 
Chris Rai, Frank Kotarski, Nathaniel Roman

CIS(theta) 1988-2007: 
A. Jorge Garcia, Gabriel Garcia, James McLurkin, Joe Bernstein, ... too many to mention here!

*nonFB
==================================
Well, that's all folks, enjoy!
Happy Clustering,
mailto:calcpage@aol.com


Saturday, December 1, 2012

CIS(theta) 2012-2013 - November Update!




Computing Independent Study
2012-2013 - Ubuntu Install Fest
Meeting III


NOVEMBER UPDATE
Sorry to say that we had no meetings this month. I was hoping for 3 meetings, but Hurricane Sandy changed everything! We are in a new room where we may be able to try out liveLinux CD based MPI clusters next month. Stay tuned!
********************

That was a great meeting today! We burned 8 CDs of Precise Pangolin and reinstalled the whole back row of our PC LAB/Classroom. Many thanx go to Jeremy for coming to vist today and lending a hand! Here's the steps we followed for a minimal install of the Student Stations (64bit Athlons):
Step1: 
Reboot each Linux box with the current  CD. Answer some basic questions about time zone, userid, passwd, no login on bootup, etc.
Step2:
Reboot each Linux box without the CD. Make sure to configure the gigE cards and proxy server:
IP: 10.5.129.x
NetMask:255.255.0.0
GateWay: 10.5.0.254
DNS: 10.5.0.254
Proxy: 10.0.0.125
Step3:
Configure System Settings as desired (unit circle trig calculator background, no screensaver, etc).
Step4: 
We had to switch Software Sources in the Ubuntu Software Center (edit/source) to Main before this would work:
sudo apt-get update
sudo apt-get upgrade
Step5:
Now, we could use the Ubuntu Software Center to install WINE.
Step6:
I copied my VTI83 and VTI89 directories from my memory stick to the Desktop. Then, after editing preferences to have VTI open with WINE, I configured each calculator.
Step7:
I will edit my /etc/crontabs tomorrow....
Step8:
We haven't decided what else we may have to install (local SAGE server, JRE, openSSH, openMPI,etc). We'll have to think about that! Here's some info on install fests from prior years:
http://shadowfaxrant.blogspot.com/2011/05/2-so-many-hard-drives-so-little-time.html
http://shadowfaxrant.blogspot.com/2011/01/then-god-mage-midterm-week-and-saw-that.html
http://shadowfaxrant.blogspot.com/2010/06/so-many-linux-distros-so-little-time.html


Computing Independent Study
2012-2013 - Ubuntu Install Fest
Meeting II

We decided to try out the new Ubuntu Linux 64bit Desktop 12.04 nicknamed Precise Pangolin. So we surfed on over to http://www.ubuntu.com and downloaded the latest ISO. We burned the CD, rebooted a guinea pig box and reinstalled it. This should be a simple procedure as we no longer use dualboot or dualnic boxes. However, we ran into a SNAFU right away! Intranet gigE works fine, but we can't get on the Internet? OOPs, we forgot the network proxy. If at first you don't succeed, try, try again!

Ubuntu Release History
4.10 Warty Warthog (mammal)
5.04 Hoary Hedgehog (mammal)
5.10 Breezy Badger (mammal)
6.06 Dapper Drake (bird)
6.10 Edgy Eft (amphibian)
7.04 Feisty Faun (mammal)
7.10 Gutsy Gibbon (mammal)
8.04 Hardy Heron (bird)
8.10 Intrepid Ibex (mammal)
9.04 Jaunty Jackalope (mythical beast)
9.10 Karmic Koala (mammal)
10.04 Lucid Lynx (mammal) 
10.10 Maverick Meerkat (mammal) 
11.04 Natty Narwahl (mammal) 
11.10 Oneiric Ocelot (mammal) 
12.04 Precise Pangolin (mammal) 
12.10 Quantal Quetzal (bird) release: 10/18

Guardian, our ssh server, is running 10.04 32bit. Guardian has a dualcore 32bit intel Xeon processor with 2GB RAM and a 512GB RAID drive.

Caprica, our ftp server, is running 10.04 32bit. Caprica has a dualcore 32bit intel Xeon processor with 2GB RAM and a 512GB RAID drive.

Shadowfax, our teacher station, is running 11.10 32bit. Shadowfax has a dualcore 64bit amd Athlon processor with 2GB RAM and a 256GB hdd. We use a 32bit OS here as SmartNotebook doesn't run on 64bit....

Alpha-Omega, our student stations, are running 11.04 64bit. These Linux boxes, like Shadowfax, have dualcore 64bit amd Athlon processors with 2GB RAM and a 256GB hdd.

We are only upgrading Alpha-Omega to 12.04 (or 12.10 if it's available when we upgrade in a couple of weeks). We are also waiting for a hardware upgrade for Alpha-Omega to amd quadcore Phenoms!

Computing Independent Study
2012-2013 - Administrativa
Meeting I (updated)

We had our traditional first organizational meeting:

(1) Wreath of the Unknown Server: 
We visited our first ssh server, Colossus, which is still in the switch room though dormant. I set it up for the first time in 1995 running Slackware Linux. Colossus ran for 12 years straight, 24x7 never having to shut down, reboot or even have anything re-installed! Colossus would not die. We finally just replaced Colossus with a dual-core Intel Xeon box complete with a RAID drive running 1TB. Old Linux boxes never die, they just fade away...


(2) Display Case Unveiled: 
We took down a ton of fractal prints and ray tracings from Room 429 to the 2 cases on the 1st floor near the art wing. We decorated both cases as best we could and left before anyone saw us. Must have been gremlins.

(3) Recruiting 2012: 
We decided that we did not have a good pool of candidates to recruit more CIS(theta) members for this year's Geek Squad, so we tabled that topic.

(4) Planing 2012: 
Next meeting would have been 9/28 but that's Yum Kipur. So, we have to wait another 2 weeks after that for 10/10 at which point Ubuntu Precise Pangolin 64bit release 12.10 Desktop Edition should be available for a mini install fest. After that, we may use bootable cluster Linux CD distros to learn MPI.
==================================
What we are researching I (Sept)
(look what this school did in the 80s): 

Thomas Jefferson High courses
http://academics.tjhsst.edu/compsci/parallel/
Thomas Jefferson High paper
http://www.tjhsst.edu/~rlatimer/techlab07/BWardPaperQ3-07.pdf
Thomas Jefferson High ftp
http://www.tjhsst.edu/~rlatimer/techlab07/
Thomas Jefferson High teacher
http://www.tjhsst.edu/~rlatimer/

What we are researching II (Oct)
(clustering environments): 

Parallel Virtual Machine
http://www.csm.ornl.gov/pvm/
openMOSIX
http://openmosix.sourceforge.net/instant_openmosix_clusters.html
Message Passing Interface
http://www.open-mpi.org/


==================================
Today's Topic:
CIS(theta) 2012-2013 - Ubuntu Install Fest! - Meeting 3


Today's Attendance:
CIS(theta) 2012-2013: Kyle Seipp
CIS(theta) 2009-2010: Jeremy Agostino


Today's Reading:
Chapter 2: Building Parallel Programs (BPP) using clusters and parallelJava
==================================
Membership (alphabetic by first name):
CIS(theta) 2012-2013: 
Kyle Seipp

CIS(theta) 2011-2012: 
Graham Smith, George Abreu, Kenny Krug, LucasEager-Leavitt

CIS(theta) 2010-2011: 
David Gonzalez, Herbert Kwok, Jay Wong, Josh Granoff, Ryan Hothan

CIS(theta) 2009-2010: 
Arthur Dysart*, Devin Bramble, Jeremy Agostino, Steve Beller

CIS(theta) 2008-2009: 
Marc Aldorasi, Mitchel Wong*

CIS(theta) 2007-2008: 
Chris Rai, Frank Kotarski, Nathaniel Roman

CIS(theta) 1988-2007: 
A. Jorge Garcia, Gabriel Garcia, James McLurkin, Joe Bernstein, ... too many to mention here!

*nonFB
==================================
Well, that's all folks, enjoy!
Happy Clustering,
mailto:calcpage@aol.com