Iceberg and Polar Differences

General

• iceberg
  • 128 nodes with 8 cores per node
  • 3.0GHz quad-core Xeon Core processor
  • 32GB memory per node totalling 4TB for the system
• polar
  • 256 nodes with 8 cores per node
  • 2.8GHz quad-core Xeon Nehalem processor
  • 48GB memory per node totalling 12TB for the system

Software versions

• SLES version
Look in /etc/SuSE-release
• SGI ProPack
Look in /etc/sgi-propack-release
• MPT, ifort, MKL, etc
To list loaded modules
module list
To list all available modules
module avail

Compiler Options

• iceberg
  -xSSSE3
• polar
  -xSSE4.2

MPI jobs - binding tasks to processors

• iceberg
  dplace is not available
• polar
  dplace is available and usage does not appear to impact performance
  
  mpiexec dplace -s1 <program>

OpenMP jobs - binding tasks to processors

• iceberg
  Use KMP_AFFINITY or omplace. Dplace is not available.

  setenv KMP_AFFINITY verbose, granularity=fine,compact

  For more information on KMP_AFFINITY see the document on Intel's web site: Thread Affinity Interface

  For more information on omplace, see omplace man page.

• polar
  Use omplace. For more information on omplace, see omplace man page.
  Do not use KMP_AFFINITY. There is a bug in the Intel OpenMP library that causes KMP_AFFINITY to interfere with dplace, resulting in poor performace.

Hyperthreading - running more than one task/thread per core

• iceberg
  Hyperthreading is not supported.
• polar
  Hyperthreading is supported.
  Can specify up to 16 ncpus and mpiprocs in job script even though only 8 physical cores per node.

  #PBS -l select=1:ncpus=16:mpiprocs=16

  For more information see the document on Intel's web site: Hyper-Threading Technology