LARGE-SCALE COMPUTING

LARGE-SCALE COMPUTING

_
iten
Codice
101799
ANNO ACCADEMICO
2020/2021
CFU
9 cfu al 1° anno di 10852 COMPUTER SCIENCE (LM-18) GENOVA
SETTORE SCIENTIFICO DISCIPLINARE
INF/01
LINGUA
Inglese
SEDE
GENOVA (COMPUTER SCIENCE )
periodo
1° Semestre
materiale didattico

PRESENTAZIONE

We will study principles and algorithms, architectures and technologies, programming models and frameworks needed to support data-intensive applications

OBIETTIVI E CONTENUTI

OBIETTIVI FORMATIVI

Learning the theoretical, methodological, and technological fundamentals of advanced data processing architectures, large-scale distributed environments, and data intensive programming including Docker, HDFS, Hadoop, Spark, and Cloud/IoT platforms.

OBIETTIVI FORMATIVI (DETTAGLIO) E RISULTATI DI APPRENDIMENTO

 

Labs will be focused on map-reduce architectures and libraries
Technology: HDFS, Hadoop, Spark using Python and Java/Scala
Intermediate lab activities (5/6 labs)
Final project

PREREQUISITI

Sequential, Concurrent and Distributed Programming
Database Theory and Practice
Basic notions of Data Analysis
 

Modalità didattiche

Frontal lectures
Lab sessions

PROGRAMMA/CONTENUTO

Distributed Systems and Distributed Programming
Virtualization and containers 
Parallel Python
Distributed data systems and shared nothing architectures
Partitioning
Replication
Fault Tolerance
CAP Theorem
Map/Filter/Reduce and Generators in Python
Map in Multiprocessing
Introduction to Hadoop and HDFS
Map Reduce
Map Reduce: Simple Design Patterns and Relational Algebra Operators
Hadoop Runtime System
Apache Spark
Apache Spark Internals
PySpark, Java/Scala Spark
Streaming Data 
Streaming Spark

DOCENTI E COMMISSIONI

Ricevimento: Su appuntamento in presenza o Teams

Ricevimento: Su appuntamento, via email o Microsoft Teams Stanza: Valle Puggia – 327

Commissione d'esame

GIORGIO DELZANNO (Presidente)

GIOVANNA GUERRINI

BARBARA CATANIA

LEZIONI

Modalità didattiche

Frontal lectures
Lab sessions

ESAMI

Modalità d'esame

Final online test with open and closed questions
Project presentation and discussion 
Bonusf for at least 70% attendance rate (if in presence) and lab assignments successfully delivered

Modalità di accertamento

The proposed exercises, project and final test cover both conceptual and practical aspects presented in the course