72778 - High-Performance Computing

Learning outcomes

At the end of the course, students will be acquainted with the fundamental programming techniques for high performance computer architectures. Students will be able to design, implement and benchmark parallel programs on shared-memory and distributed-memory systems.

Course contents

Syllabus

• Parallel algorithms: synchronous and asynchronous algorithms; parallel programming patterns
• Introduction to HPC architectures: Flynn taxonomy, shared memory and distributed memory architectures; GPGPU
• Parallel programming patterns (embarassingly parallel; stencil; work farm; scan; reduce)
• SIMD programming using compiler intrinsics or auto-vectorization (GCC compiler)
• Programming shared-memory architectures with C/OpenMP
• Programming distributed-memory architectures with C/MPI
• GPU programming with CUDA
• Performance evaluation of parallel programs: measuring and understanding speedup and efficiency

Prerequisites

The course requires good programming skills with the C language in a Unix/Linux environment, and basic knowledge of computer architectures.

Readings/Bibliography

Selected parts from the following textbooks:

• Peter Pacheco, An Introduction to Parallel Programming, Morgan Kaufmann, 2011, ISBN 978-0123742605
• I. Foster. Designing and Building Parallel Programs. Addison-Wesley, 1995, ISBN 978-0201575941; Available online.
• Jason Sanders, Edward Kandrot, CUDA by Example: An Introduction to General-Purpose GPU Programming, Addison-Wesley, 2010, ISBN 978-0131387683
• Georg Hager, Gerhard Wellein, Introduction to High Performance Computing for Scientists and Engineers, CRC Press; 1 edition (July 2, 2010), ISBN 978-1439811924

Teaching methods

The foundations of parallel programming and several parallel programming costructs are introduced during the lectures. Then, the same concepts are applied in the lab through a set of small programming assignments under the supervision of the instructor. The solution of every programming exercise is provided after the lab sessions, in order to support self-study.

Assessment methods

Written exam and programming project. The written exam consists of a set of questions (normally, four) on the topics addressed during the lectures. The written exam is closed-book and consists of a set of open questions and/or multiple-choice quizzes. The maximum number of points granted by each correct answer is indicated in to the question. The programming project requires the implementation of a parallel program according to specifications provided by the instructor, together with a written report. The project evaluation will consider the clarity, correctness and efficiency of the program submitted for evaluation, and the quality of the accompanying report. Both the written exam and the programming project are graded on a scale from 0 to 32; each one is passed with a grade equal to or greater than 18. The final grade is computed as the weighted average of the grades of the written part (weight 4) and the programming assignment (weight 6).

Teaching tools

Lectures: slides and practical programming demos will often be used, integrated with exercises and Q&A sessions at the blackboard.

Lab: all programming activities will use the gcc compiler with OpenMP support, MPI and the CUDA/C compiler. All tools are freely available, in order to support self-study activities on own hardware.

Links to further information

http://www.moreno.marzolla.name/teaching/HPC/

Office hours

See the website of Moreno Marzolla