PhD course (CIS):

Programming Frameworks for Deep Learning (3hp)

Programming Models and Accelerated Learning on Parallel and Heterogeneous Systems

HT/2018

Recommended for
Graduate (CUGS, CIS, ...) students interested in the use and the structure of domain-specific programming frameworks for artificial neural networks and deep learning.

Organization and Schedule

• Introductory lecture (ca. 3h) on friday 7 dec 2018 13:15, room Donald Knuth
• Miniproject
• Demo/presentation session (ca. 6h) on friday 15 february 2019, 09:15-12:15 in D. Knuth/J. von Neumann, 13:15-16:00 in J. von Neumann

The course was last given
This is a new course.

Goals
The course studies the programming model, structure, optimization and acceleration opportunities of some popular domain-specific programming frameworks for deep learning, such as TensorFlow.

Prerequisites
Programming in C++ and/or Python.
Some background in parallel and accelerator computing, such as TDDD56 or TDDC78.
Linear algebra.
Discrete mathematics.
Data structures and algorithms.
Some introductory course on machine learning including neural networks.

Contents

• Short recapitulation of fundamental issues in machine learning, feed-forward neural networks, backpropagation learning algorithm.
• Study of several popular domain-specific programming frameworks for neural networks and deep learning applications, such as TensorFlow, Caffe, Dlib.
  Programming constructs, Framework structure, Examples. Classification.
• Opportunities and techniques for accelerated learning and execution on parallel systems (multicore, GPGPU).
• Individual miniproject by participants.
  Presentation of miniproject results.

Slides

• Presentation slides (restricted access, for course participants only)

Literature

No mandatory course book, but some references are given below.

Background on deep learning, e.g.:

• Heikki Huttunen: Deep Neural Networks: A Signal Processing Perspective.
  Chapter in S. S. Bhattacharyya et al. (eds.), Handbook of Signal Processing Systems, 3rd edition, 2019, pages 133-163.
  DOI: 10.1007/978-3-319-91734-4_4

• Matthew Scarpino: TensorFlow for Dummies. Wiley, 2018.
  Available in the Campus Valla Library as electronic resource.

Teachers
Christoph Kessler, IDA, Linköpings universitet (course leader, lecturer, examiner)

Examination

• Active participation in the introductory lecture and presentation sessions
• Successful individual miniproject and presentation

Credit
3hp if both examination moments are fulfilled.

Comments
New course 2018.

Related Courses
This course could complement the course Neural Networks and Deep Learning, which is also announced for HT2018.