Deep Learning for Computer Graphics
Fall 2018
Instructor: Nima Kalantari

Course information

Time and location: TR 11:10 - 12:25 in 126 HRBB
Office hours: T 10:00 - 11:00
Office location: 527B HRBB
Email: nimak@tamu.edu

Overview

In this course, we first cover the basic principles of deep learning, including different network architectures and training strategies. We then study the recent use of deep learning in a variety of computer graphics applications such as view synthesis, image inpainting, colorization, and Monte Carlo denoising. Specifically, we discuss different solutions for this challenging problem, including novel system designs and dataset preparation strategies.

Prerequisites

There are no prerequisites. However, basic understanding of machine learning and computer graphics is recommended. Basic knowledge of Python is also recommended, but not necessary.

Course format

This is a graduate level course and it mainly consists of students presenting recent research papers on the applications of deep learning in computer graphics. For the first few classes, the instructor covers the basics of deep learning. Afterwards, each class consists of two presentations by students. Each presentation should be 20 minutes with an extra 15 minutes for class discussion.

We will have two assignments, which need to be implemented in one of the deep learning packages (PyTorch, TensorFlow, etc.). We will also have a final project, which could be anything related to the use of deep learning in computer graphics.

Textbook

There is no textbook for this course, since it is based on recent research papers. The papers are all available online.

Grading

Schedule*

Date Topic Papers Presenter(s) Assignments
Aug 28 Introduction and overview Nima
Aug 30 Basics of deep learning I Nima
Sep 4 Basics of deep learning II Nima
Sep 6 Style transfer Gatys 16, Ruder 16 Tushar, Duc
Sep 11 Image synthesis I Radford 15, Isola 17 Prateek, Nitin
Sep 13 Image synthesis II Chen 17, Qi 18 Saurabh, Qinbo
Sep 18 View Synthesis I Zhou 16, Flynn 16 Nishant, Avinash Hw 1 out
Sep 20 View Synthesis II Kalantari 16, Wu 17 Avinash, Yaohua
Sep 25 Colorization I Larsson 16, Lizuka 17 Buvaneish, Buvaniesh
Sep 27 Colorization II Zhang 17, He 18 Stuti, Peng
Oct 2 High dynamic range imaging I Kalantari 17, Wu 18 Duc, Yaohua Hw 1 due
Oct 4 High dynamic range imaging II Eilertsen 17, Endo 17 Stuti, Saurabh
Oct 9 Image inpainting Pathak 16, Lizuka 17 Yerania, Yashwanth Hw 2 out
Oct 11 Image super-resolution Dong 14, Ledig 17 Prateek, Samia
Oct 16 Sketches I Wang 15, Sangkloy 17 Yerania, Fieza
Oct 18 Sketches II Sangkloy 17, Simo-Serra 16 Fieza, Peng
Oct 23 Final project proposals I
Oct 25 Final project proposals I Hw 2 due
Oct 30 Intrinsic decomposition I Narihira 15, Shi 17 Cassandra, Cassandra
Nov 1 Intrinsic decomposition II Liu 17, Bi 18 Yashwanth, Peeyush
Nov 6 Monte Carlo denoising I Kalantari 15, Bako 17 Logan, Brennen
Nov 8 Monte Carlo denoising II Chaitanya 17, Vogels 18 Nitin, Brennen
Nov 13 Shading Nalbach 17, Thomas 17 Kevin, Logan
Nov 15 Relighting Ren 13, Ren 15 Xiaojing, Tushar
Nov 20 Image denoising Gharbi 16, Mildenhall 18 Xiaojing, Peeyush
Nov 22 Thanksgiving - no class
Nov 27 Other topics I Zhou 18, Xu 18 Kevin, Samia
Nov 29 Other topics II Ephrat 18, Prashnani 18 Qinbo, Nishant
Dec 4 Final project presentation I
Dec 6 Final project presentation II

*Schedule is subject to small changes.