CSCI 8000 | 60609: ADVANCED SPECIAL TOPICS IN COMPUTER SCIENCE

- EFFICIENT DEEP LEARNINNG

SYLLABUS Fall-2024

Instructor
Dr. Geng Yuan, School of Computing,
Office: Boyd Research and Education Center, Room 807
Email: [email protected]

Class Location and Times

Tue and Thur, 15:55 pm – 17:10 pm Boyd Grad Rsch Ctr 0302 Aug 14, 2024 - Dec 03, 2024
Wed, 16:10 pm – 17:00 pm Boyd Grad Rsch Ctr 0302 Aug 14, 2024 - Dec 03, 2024
Information on modality: In-person
Office Hours: Wed 3:00 pm – 4:00 pm, Boyd 807
Teaching Assistant: N/A
TA Office Hours: N/A

Textbook (not mandatory)

Deep Learning (by Ian Goodfellow, Yoshua Bengio, Aaron Courville)
Amazon Link

Prerequisites
Knowledge of programming in Python
Preferably have a basic knowledge of deep learning

Description
This class is a graduate-level course that welcomes both Masters and PhD students to enroll. It will focus
on the recent advances in Efficient Deep Learning applications and research fields. Students will have the
opportunity to get familiar with the methods and tools to facilitate the energy efficiency of deep learning
and learn the basics of how to conduct research.

Learning Objectives / Outcomes

The objective is to make students competitive in the growing job markets and research activities of artificial
intelligence and big data areas. There are two major components in this course.

1 | Page
The first is to review the basics of deep learning models, e.g., feedforward neural networks, deep
convolutional neural networks (VGG, ResNet, MobileNet, Yolo), transformer-based neural networks (e.g.,
BERT, ViT, GPT), and their inference and training algorithms. And introducing deep learning
tasks/applications, e.g., image classification, object detection, speech recognition, robotics, etc.

The second part is introducing different methods and tools for enhancing energy efficiency of deep learning
training and inference. At this part, students will review the classic and most advanced literature of efficient
deep learning methods and will give presentations in group.

This class requires programming in Python, and students will learn effective deep learning tools such as
PyTorch and TensorFlow.

Topics covered
1. Basics of deep learning
2. Model pruning
3. Quantization
4. Efficient network design and Neural Architecture Search (NAS)
5. Knowledge Distillation
6. Sparse Training
7. Data-efficient learning

Grading policy
Homework (1~2 small projects): 25%
Attendance: 25%
Presentations: 50%
* Bonus: up to 5%

Grading System:
A : 100~93 C : 76.65~73.33
A- : 92.99~90 C- : 73.32~70
B+ : 89.99~86.66 D+ : 69.99~66.66
B : 86.65~83.33 D : 66.65~63.33
B- : 83.32~80 D- : 63.32~60
C+ : 79.99~76.66 F : 59.99~0

2 | Page
Grading Policy
• Homework assignments are to be submitted online or handed to the instructor on the assignment
due date. Assignments submitted after the due date will be deducted 10 points for each day late.

• Presentations MUST strictly follow the schedule. Any rescheduling will cause a score deduction
(up to 50% of that presentation), unless the group can find another group to swap time slots with.

• The final version of the presentation slides must be submitted before the class start time of the
presentation day. Any delay may cause a score deduction.

Absences
You are expected to attend each class punctually and remain for the entire class period.
You need to inform the instructor in advance (before the class start time) if you expect to miss a class or
leave the course early.
If you miss class, your absence will be excused by the instructor only if a doctor’s certificate or other
evidence is submitted (e.g., COVID). But you still need to inform the instructor before the class start time
of that day.
You remain to be responsible for the work associated with the class you missed, even if your absence has
a valid reason. The DRC defines disability-related absences as when a student is absent due to the direct
effects of their disability. Disability-related absences do not apply to absences for other reasons.

Academic Honesty
As a member of the University of Georgia, all students are responsible for knowing the University's policy
on academic honesty. “I will be academically honest in all of my academic work and will not tolerate
academic dishonesty of others.” All students agree to follow this code by signing the Admissions
Application. All assignments submitted in this course must be your own unless you have received the
instructor's permission to collaborate and have properly acknowledged receiving assistance. It is the
responsibility of the instructor and the TA to uphold the University's academic honesty policy and report
the belief of dishonesty to the Office of the Vice President for Instruction. More details about the Academic
Honesty policy can be found at https://honesty.uga.edu/Academic-Honesty-Policy/Student-Honor-Code/.

Accessibility
University of Georgia always values the diversity of our students. Students who experience a barrier to full
access to this class should let the instructor know and make an appointment to meet with a staff member in
Disability Resource Center as soon as possible. More information can be found at https://drc.uga.edu/.

3 | Page
Computing Resources
• Your own Laptop (w/ or w/o GPU)
• Online computing resources:
Google Colaboratory
Amazon SageMaker
Kaggle
…
• Resources from our college and department (GACRC)

Equal Opportunity
The UGA Equal Opportunity Office (EOO) is responsible for ensuring that UGA complies with all
applicable laws and policies regarding discrimination and harassment on the basis of race, sex (including
sexual harassment and pregnancy), gender identity, sexual orientation, ethnicity or national origin,
religion, age, genetic information, disability or veteran status.
More details at https://eoo.uga.edu/About/about-the-office/

College of Engineering Calendar

The College of Engineering maintains an online calendar that includes information regarding academic
deadlines, student engagement, and success activities, student club and organization meetings, and career
preparedness and employer events.
The calendar is accessible at https://www.engineering.uga.edu/calendar. Students are encouraged to visit
the calendar each week to view upcoming events.

Classroom Technical Support

If an instructor needs assistance with your classroom technology or needs to report a technical problem,
try the CTL hotline, (706) 542-3456. This number will be answered from 7:30 am to 10:30 pm, Monday
through Friday, 10:00 am to 7:00 pm Saturday, and 11:00 am to 7:00 pm on Sunday.

*This syllabus provides a general plan for the course, the instructor may inform the class for any
necessary changes.

4 | Page