Advanced Machine Learning

Description

This course will cover the science of machine learning. It focuses on the mathematical foundations and analysis of machine learning methods and how they work.

Class meets Tuesday and Thursday from 9:30 AM to 10:45 AM in 260 Classroom Building.

To request a force-add, fill out this form: https://forms.gle/pqcbaSJmxAG3VTG4A during the first class (before 10:45 AM on 8/27/19).

• Instructor:
  Bert Huang, Assistant Professor of Computer Science
  Office hours: Tuesday 11:00am–1:00pm, Thursday 11:00am–12:00pm, Torgersen Hall 3160L
  bhuang@vt.edu
• Teaching Assistants:
  
  Alyssa Herbst, alyssa2@vt.edu
  Office hours: Wednesday 9:30am–11:30am, 12:30pm–1:30pm
  
  
  
  You Lu, you.lu@vt.edu
  Office hours: Monday 9:00am–12:00pm, Kelly Hall 219
  
  
  
  Min Oh, minoh@vt.edu
  Office hours: Thursday 4–5pm, Friday 9–11am, Torg 2160T
  
• Office Hour Chart
  

  Mon.	Tue.	Wed.	Thu.	Fri.
  9 AM – 12 PM You Lu (Kelly 219).	9:30 AM – 10:45 AM Class (NCB 260). 11 AM – 1 PM Bert (Torg 3160L).	9:30 AM – 11:30 AM Alyssa (Kelly 219). 12:30 PM – 1:30 PM Alyssa (Kelly 219).	9:30 AM – 10:45 AM Class (NCB 260). 11 AM – 12 PM Bert (Torg 3160L). 4 PM – 5 pm Min (Torg 2160T)	9 AM – 11 AM Min (Torg 2160T).

The course homepage (this page) is at http://courses.cs.vt.edu/cs5824/Fall19/.

The course Canvas site is at https://canvas.vt.edu/courses/94748 and should be visible to all users with a Virginia Tech login.

Topics

• Overview of machine learning: learning from data; overfitting, regularization, cross-validation.
• Supervised learning: decision trees, naive Bayes, logistic regression, support vector machines, neural networks
• Unsupervised and semi-supervised learning: clustering (k-means, Gaussian mixtures); principal components analysis
• Learning theory: probably approximately correct (PAC) learning, model complexity, bias and variance
• Structured models: Bayesian networks, Markov random fields, hidden Markov models
• Deep learning: convolutional neural networks, recurrent neural networks
• Other topics: reinforcement learning, machine learning applications (vision, natural language processing, recommendation)

Prerequisites

The listed prerequisite courses cover relevant material that includes: data structures, algorithms, complexity, linear algebra, and basic concepts of probability and statistics (random variables, expectation, conditional distributions, Bayes rule, sampling distributions, estimators, likelihood, and maximum likelihood). The homework assignments will include programming portions using Python.

Please speak with the instructor if you are concerned about your background. Note: If any student needs special accommodations because of any disabilities, please contact the instructor during the first week of classes.

Learning Objectives

A student who successfully completes this class should

• be familiar with a breadth of foundational machine learning concepts;
• be able to implement standard machine learning methods without the use of pre-packaged machine learning software;
• be able to make informed decisions about which machine learning methods are appropriate for different tasks;
• have awareness of the mathematical and computer science concepts underlying machine learning;
• and have the background knowledge to be able to understand new machine learning methods not covered in the course.

Reading and materials

• Our main textbook will be Understanding Machine Learning. By Shai Shalev-Shwartz and Shai Ben-David. Cambridge University Press. Digital copy available for free from the authors' page.
• Additional materials may be provided electronically, including readings, video lectures, and other media.

Homework

• Each homework assignment will include written and programing portions.
• Both written and programming assignments will be submitted electronically.
• I strongly recommend learning LaTeX to help write neat, readable math, but I will not require LaTeX.
• Programming assignments will be in Python.
• Homework assignments will be posted on Canvas.
• The last homework is a research project, in which you will replicate part of a recent machine learning research publication.

Project

For the final project, students will attempt to reproduce a finding from a machine learning research study. See instructions on the project page.

Schedule

The tentative class schedule is available here and is embedded below. We will update the schedule regularly.

Slides and other materials will be available in at http://courses.cs.vt.edu/cs5824/Fall19/slide_pdfs/.

Policies

Accessibility

If any student needs special accommodations because of any disabilities, please contact the instructor during the first week of classes. Such students are encouraged to work with The Office of Services for Students with Disabilities to help coordinate accessibity arrangements.

Regrading requests

Requests for regrading due to grading errors must be submitted in writing to the TA within one week of the release of grades.

Late homework policy

Homework submitted late without permission will be penalized according to the following formula: (Penalized score) = (Your raw score) * (1 - 0.1 * (# of days past deadline))

This formula will apply for up to three days, after which the homework will not be accepted and you will receive a grade of zero. Avoid invoking these penalties by starting early and seeking extra help.

Grading scale

• 5%: Class attendance
• 50%: Homework Sets
• 15%: Midterm Exam
• 15%: Final project
• 15%: Final exam

Based on the grading breakdown above, each student's final grade for the course will be determined by the final percentage of points earned. The grade ranges are as follows:

A:	93.3%–100%,	A-:	90.0%–93.3%,	B+:	86.6%–90.0%,	B:	83.3%–86.6%,	B-:	80.0%–83.3%,	C+:	76.6%–80.0%,
C:	73.3%–76.6%,	C-:	70.0%–73.3%,	D+:	66.6%–70.0%,	D:	63.3%–66.6%,	D-:	60.0%–63.3%,	F:	00.0%–60.0%.

Class attendance

All students are expected to attend all lectures unless they have given sufficient notice for justifiable absences. Absence will be excused for reasons including health needs, conference travel, family emergencies, and job-search interviews.

Attendance will be recorded via a randomized roll call.

Academic integrity

Students enrolled in this course are responsible for abiding by the Graduate Honor Code. Ignorance of the rules does not exclude any member of the University community from the requirements and expectations of the Honor Code. For additional information about the Honor Code, please visit: https://graduateschool.vt.edu/academics/expectations/graduate-honor-system.html

This course will have a zero-tolerance philosophy regarding plagiarism or other forms of cheating. Your homework assignments must be your own group's work, and any external source of code, ideas, or language must be cited to give credit to the original source. I will not hesitate to report incidents of academic dishonesty to the Graduate Honor System.

Principles of Community

Because the course will include in-class discussions, we will adhere to Virginia Tech's Principles of Community. The first two principles are most relevant:

• We affirm the inherent dignity and value of every person and strive to maintain a climate for work and learning based on mutual respect and understanding.
• We affirm the right of each person to express thoughts and opinions freely. We encourage open expression within a climate of civility, sensitivity, and mutual respect.

The remaining principles are also important and we will take them seriously as a class.