Course title

Parallel and Distributed Systems

Course code

ΜΚ34

Course type

Compulsory

Course level

Undergraduate (first cycle)

Year of studies

3nd

Semester

6th

ECTS Credits

5

URL

http://eclass.uowm.gr/courses/ICTE161/ & http://arch.icte.uowm.gr/courses/parallel/

Hours per week

4 (2 hours theory & 2 hours laboratory)

Instructor(s)

Minas Dasygenis (Lecturer)

Course content

Introduction to Parallel Systems. History of Parallel and Distributed Systems. Von Neumann organization. Flynn taxonomy. Pipeline. Multi computer systems and multi-core. Distributed and shared memory. Uniform and non-uniform Memory Architectures. Performance estimation. Scaling. Interconnection networks. Laws of Grosch, Amdahl, Gustafson-Barsis. Designing and programming parallel applications. MPI. Synchronization. Dependencies Graph. Scheduling. Shared Memory coherence. MESI. Parallel Processing at GPU. Models and communication mechanisms of processes. Vector Processing. Cluster & Grid Computing. Parallelizing application examples. Synchronization issues.

Laboratory assignments of parallel programming of OpenMPI, OpenMP, threads and CUDA.

Expected
learning outcomes and competences to be acquired

Upon successful completion of this course, students will be able to demonstrate knowledge and understanding of:

  • the reasons that the sequential computing has been abandoned,

  • the similarities and differences of parallel architectures,

  • the CPU cores interconnection networks,

  • the memory coherency problems and the possible solutions,

  • the significance of the clock synchronization of distributed systems,

  • the multi-core CPU and GPU strengths and weaknesses,

  • the granularity of the parallel processing.


From the laboratory assignments, students will gain the abilities to:

  • scale an application,

  • transform an application to exploit the available parallelism,

  • develop and debug parallel programs,

  • utilize the openmpi framework for distributed parallel systems,

  • utilize the openmp framework for shared memory parallel systems,

  • utilize the cuda framework for GPU parallel systems,

  • utilize the POSIX threads for shared memory parallel systems,

  • utilize a batch submission system for the grid,

  • utilize a batch submission system for a computer cluster,

  • analyze and locate application hotspots,

  • measure the performance of parallel and distributed systems,

  • identify the best architectures and system for solving a given computational problem.

Prerequisites

Operating Systems, C Programming (not compulsory)

Teaching methods

Lectures, Powerpoint slides, Lecture Notes, in class quizzes, e-class, automated examination system i-exams, opencourses video lectures, laboratory exercises, semester group project.

Assessment methods

Written final theory exam 50%, final lab exam 10%, three mini exams 15%, 12 weekly laboratory exercises 10%, 1 semester team project 15% .

Language of instruction

Greek

Recommended bibliography

  • ANDREW S. TANENBAUM, MAARTEN VAN STEEN, ΚΑΤΑΝΕΜΗΜΕΝΑ ΣΥΣΤΗΜΑΤΑ: ΑΡΧΕΣ ΚΑΙ ΥΠΟΔΕΙΓΜΑΤΑ, ΕΚΔΟΣΕΙΣ ΚΛΕΙΔΑΡΙΘΜΟΣ, 2006. [13777] https://service.eudoxus.gr/search/#a/id:13777/0

  • DAVID B. KIRK, WEN-MEI W. HWU, ΠΡΟΓΡΑΜΜΑΤΙΣΜΟΣ ΜΑΖΙΚΑ ΠΑΡΑΛΛΗΛΩΝ ΕΠΕΞΕΡΓΑΣΤΩΝ, ΕΚΔΟΣΕΙΣ ΚΛΕΙΔΑΡΙΘΜΟΣ, 2010. [12279261] https://service.eudoxus.gr/search/#a/id:12279261/0

  • ΣΤΕΛΙΟΣ ΠΑΠΑΔΑΚΗΣ, ΚΩΣΤΑΣ ΔΙΑΜΑΝΤΑΡΑΣ, ΠΡΟΓΡΑΜΜΑΤΙΣΜΟΣ ΚΑΙ ΑΡΧΙΤΕΚΤΟΝΙΚΗ ΣΥΣΤΗΜΑΤΩΝ ΠΑΡΑΛΛΗΛΗΣ ΕΠΕΞΕΡΓΑΣΙΑΣ, ΕΚΔΟΣΕΙΣ ΚΛΕΙΔΑΡΙΘΜΟΣ, 2012. [12532275] https://service.eudoxus.gr/search/#a/id:12532275/0




Lectures


Laboratory Lectures
Laboratories Exercises