Systems  
Status displays
System status
Retired systems «
 
 
 
 
 
 
 
 
 
 
 
 

Monolith

Monolith

The cluster Monolith (previously called "Linux Cluster 2" and "New Cluster") consists of over 200 PC computers. Each computer has two Intel Xeon processors at 2.2 GHz, 2 GBytes primary memory (ECC DDR) and a disk memory with 80 GBytes. In addition there is a common main disk storage of 7 TBytes.

Usage

To open your account and for all of your support questions, please send an email to support@nsc.liu.se.

Please read the instructions for Monolith in the Monolith User Guide. Do you have any problems or questions, please mail us at the address support@nsc.liu.se.

Both processors are now available on each compute node. There are 198 nodes available for users, meaning that there are 396 processors in total available for computational tasks. As an example, if you need 32 processors in your job, you allocate:

	#PBS -l nodes=16:ppn=2
Nodes on Monolith are allocated on a node basis. This means that you will be accounted for two processors on a node, even if you use only one. If you e.g. run parameter variations on a problem as one-processor jobs, it would be a good idea to run two variations in each job, thus using both processors.

The maximum wall clock run time for a job is 144 hours, i.e. 6 days.

Both /home/ and /disk/global are available. Store sources, and other crucial data on /home/. Backup of /home/ is made every night. Generated data, results of your computations belongs on the large /disk/global/ (no backup).

  • Monolith will always be requested and allocated on a node basis. Example:
      With one processor/node:     you need 32 processors ->
    	#PBS -l nodes=32:ppn=1
      With two processors/node:    you need 32 processors ->
    	#PBS -l nodes=16:ppn=2
    
  • How to write a run script to start two jobs (in separate directories) at the same time.

    Is the following script correct?

     #PBS -l nodes=1:ppn=2
     cd /directory_one/
     ./aa
     cd /directory_two/
     ./aa
    
    That is almost correct, but if you do as shown above, the first "./aa" computation will go to completion before you start with the next, i.e. they will not compute in parallel. You have to "fork away/put in parallel/put in the background" at least the first of them by ending the command line with a '&' sign. This also means that you have to let the script "wait" for the completion of those computations, that you have forked away.

    Summary:

    #PBS -l nodes=1:ppn=2
    cd /directory_one/
    ./aa &
    cd /directory_two/
    ./aa &
    wait
    

Academic users reach the computer as monolith.nsc.liu.se. SMHI users reach the computer as login-2.monolith.nsc.liu.se.

You can obtain info on how much time you have used the present month with the command projinfo.

You run it on your service node on Monolith, normally the computer monolith.nsc.liu.se and it tells you how many CPU hours you have been accounted for during the current month. An example:

	[y_user@login-1 y_user]$ projinfo

	Project        Used[h]
	  User
	---------------------

	p2003999       1590.1
	  x_user        100.0
	  y_user       1490.1

	[y_user@login-1 y_user]$
This output means that the batch system until now has accounted your project 1590 CPU hours this month, one hundred of these hours for the jobs of x_user and 1490 CPU hours for the jobs of y_user.

The accounting is accumulated at the end of each batch job, according to how many computing nodes you have asked for (please remember that each node has two CPUs, meaning that you are accounted for two CPUs for each node) and how long time the job actually run.

An example on this, assuming the following definitions in your job script:

#PBS -l nodes=4:ppn=2
#PBS -l walltime=10:00:00
If we also assume that the job completes after three of the requested ten hours, you are accounted for 24 cpu hours (4 * 2 * 3).

If would be the same, if you requested nodes=4:ppn=1, because the system makes reservations only down to the granularity of nodes, not CPU:s. (This may change later.) And it does not matter if you really run your job on all those nodes and all those processors, you are still accounted for them all.

As you see from the example, you are not accounted for making long reservations, only for how long time your job really runs. The down side with long reservations is that longer jobs get a lower priority in job scheduling, so you might wait longer for the job to start.

Link to the Monolith status page

Software

Fortran compilers

Intel (ifc) Fortran 90/95 as well as PGI 4.0 (pgf77 and pgf90) and g77 compilers are available on the system.

MPI

The cluster is equipped with a fast interconnect (SCI) with its own mpi implementation (ScaMPI). ScaMPI performance between two nodes is about 4.5 us latency for small packages and 260 MB/s bandwidth for large ones.

For compatibility reasons two other MPI implementations (LAM and MPICH) are also available. These two run over a comparatively slow ethernet (100 Mb/s) and will not be suitable for communication intensive applications.

Libraries

The final list of provided mathematics libraries is yet to be determined but SCALAPACK will soon be available:

  • BLAS (optimized and reference) on /usr/local/lib/
  • LAPACK on /usr/local/lib/
  • SCALAPACK

If you need other libraries you can either build them yourself or ask for our assistance building and/or installing them globally on the system. Vampir is now installed.

Applications

Several application programs have now been installed on Monolith, see [here].

Performance

The theoretical peak performance of Monolith is 1.8 Tflops, and so far we have obtained 1.132 Tflops with LINPACK.

The TOP 500 listing for November 2004 is now available. Monolith is at place 315 on the TOP 500 Supercomputer Sites listing.

Whats in a Name?

Monolith
  • is a single great stone often in the form of an obelisk or column.
  • is a massive structure.
  • is an organized whole that acts as a single unified powerful and/or influential force.
  • is an important item in the Stanley Kubrick and Arthur C. Clarke's science fiction classic movie 2001: A Space Odessey.
  • has the last letters identical to the official abbreviation LiTH for the technical faculty in Linköping.






Page last modified: 2008-06-05 13:40
For more information contact us at info@nsc.liu.se.