High Performance Computing

This graduate-level course focuses on developing, analyzing, and implementing parallel and locality efficient algorithms. It covers theoretical underpinnings and includes practical exercises on modern HPC systems like Cilk Plus, OpenMP, CUDA, MPI, and PGAS models. The course emphasizes scalability in terms of problem size and system size (number of cores or compute nodes). To achieve scalability, students are taught to reduce asymptotic complexity and minimize communication and data movement. The course covers algorithm design and analysis for multicore/shared memory machines, distributed memory machines, and machines with deep memory hierarchies. Students will apply these techniques to fundamental problems such as sorting, search on trees and graphs, and linear algebra. The practical aspect of the course involves implementing algorithms on real parallel and distributed systems using programming models like Cilk Plus, OpenMP, and MPI. The goal is to bridge the gap between theory and practice by testing the effectiveness of algorithms in real-world scenarios.