SGI Altix ICE Frequently Asked Questions

General

Batch Queue

File Management

Compilers

Writing Parallel Code

General

Batch Queue

  • Where can I learn about PBS?

    See PBS

  • Why is there another job running on the same node as my job?

    Specify the exclusive directive in your PBS script.

    See PBS exclusive use of cluster nodes

  • Can I run jobs that require a graphical interface?

    Yes, if the binary supports X11 and your workstation has a working X11 server.

    When you ssh to polar, you will need to tell the ssh client to enable X11 forwarding. With the command line version, you will need to add "-X".

    Once you've logged in, run the following: 

    module load x11_lustre

    If you get an error while loading the module, the problem is most likely the X11 server on your workstation. Ensure that it's running and test it if possible.

    Jobs with X11 input/ouput must be interactive. When you submit your job, you must have the following flags: "-V -X -I". Keep in mind that when you submit a job with the -I flag, any job script you specify will not be executed.

    When the job executes, you will get a shell on one of the compute nodes. Your display should already be set and any binary you execute that has a graphical display should work properly.

File Management

  • What's the best way to manage my files in scratch space?

    See the Large File FAQ

Compilers

Writing Parallel Code

  • Is automatic parallelization supported?

    Yes. You can have the ifort compiler automatically parallelize your program by adding the -parallel switch to your compilations. By also adding the switch -par_report3 you can also get a detailed report of how the compiler parallelized your program.

  • Does the system support OpenMP?

    Yes. If your program uses OpenMP directives then the switch -openmp must be used on both the compile and link stages of your program build. In addition the environmental variable OMP_NUM_THREADS should be set to the number of threads that you want. Note there are 8 cores/node, so OpenMP jobs can span only 8 cores.

    To execute OpenMPI programs use the following format: 

    omplace -c 0-7 executable
  • Can I use OpenMP and automatic parallelization in the same program?

    Yes. You can mix both types within the same program. However, if a file contains OpenMP directives and you specify both the -openmp and the -parallel switches, then the compiler will ignore the -parallel switch for that file.

  • Is Cluster OpenMP supported?

    No.

  • Is MPI supported?

    Yes. SGI's Message Passing Toolkit includes MPI libraries. Use ifort and icc to compile and link your program and include -lmpi in your list of link libraries. To run your program include the lines: 

    set MPI_DSM_DISTRIBUTE
set MPI_USE_IB
set MPI_MEMMAP_OFF
mpiexec -n mmm executable

    where mmm is the number of MPI processes. The use of -n mmm is optional and if not specifed the number of processors will be determined by the #PBS -l settings.

  • Is MPI/OpenMP mixed mode programs supported?

    Yes. OpenMP threads will be divided among, at most, 8 cores. Remember to set the environment variable OMP_NUM_THREADS to the number of threads you want.

    To execute a hybrid OpenMP/MPI program use the following format: 

    mpiexec -n number_of_nodes omplace -nt number_of_threads_per_node executable
  • Is MPI-2 supported?

    Yes, MPT supports the MPI-2 standard.

This Is An Official U.S. Navy Website.

U.S. Naval Research Lab 4555 Overlook Ave., SW Washington, DC 20375