homecloud computingcloud data Processingcloud of thingsclound of networks
cloud data processing

Introduction to Parallel Programming with OpenMP and MPI

Duration: 5 Days

Course Background

MPI is a language-independent communications protocol used to program parallel computers. It supports both point-to-point and collective communication protocols. MPI "is a message-passing application programmer interface, whose underlying design goals are high performance, scalability, and portability. MPI is widley used in high-performance computing applications. MPI is commonly used for parallelism between nodes and is often combined with OpenMP for parallelism in shared memory multi-processor systems running on a single node. The aim of this course is to provide an intensive overview of using both MPI and OpenMP together and a discussion of how these kinds of techniques can be deployed effectively in the cloud.

Course Prerequisites and Target Audience

Attendees should be experienced C/C++ programmers with a sound knowledge of

  • Pointers and pointer operations
  • Computing using multidimensional arrays
  • File I/O and file and directory manipulation
  • Data structures and classes
  • Collection classes such as linked lists and vectors
Attendeeds are also expected to have some familiarity with
  • Multithreading
  • Memory allocation and memory management
  • Computer architectures and instruction sets
  • Basic techniques for code profiling and code optimisation
  • Code debugging
  • Sockets programming and distributed computing

Course Outline

  • Introduction to Parallel Architectures and Programming Models
  • Introduction to MPI
    • MPI Process model
    • LLNL MPI Implementations and Compilers
    • Point to Point Communication Routines
      • General Concepts
      • MPI Message Passing Routine Arguments
      • Blocking Message Passing Routines
      • Non-Blocking Message Passing Routines
  • Collective Communication Routines
  • Derived Data Types
  • Group and Communicator Management Routines
  • Virtual Topologies
  • Domain decomposition of structured and unstructured grids
  • Parallel debugging
  • Shared Memory Parallelization with OpenMP
    • Introduction and execution model
    • Work sharing directives
    • Data environments
    • Verifying OpenMP parallelizations
    • OpenMP Performance tuning
  • Combined MPI and OpenMP programming
    • Models for Parallel Programming on Hybrid Systems
    • Running OpenMP on clusters or clusters of SMP Nodes
    • MPI-2 Parallel file I/O
  • Implementing a Scientific Computing Environment Using OpenStack
    • Overview of Cloud Computing and OpenStack
    • Building an HPC Cluster using OpenStack and MPI