Machine Learning

CS4824/ECE4424 Spring 2019

Description

This course will cover the science of machine learning. It focuses on the mathematical foundations and analysis of machine learning methods and how they work.

Class meets Monday, Wednesday, and Friday from 9:05 AM to 9:55 AM in Torgersen 3100.

  • Instructor:
    Bert Huang, Assistant Professor of Computer Science
    Office hours: Monday 10:00am–12:00pm, Friday 10:00am–11:00am, Torgersen Hall 3160L
    bhuang@vt.edu
  • Teaching Assistants:
    Chidubem Arachie
    Office hours: Tuesday and Thursday, 9:15am–10:45am, Kelly Hall 219
    achid17@vt.edu

    Shuangfei Fan
    Office hours: Wednesday 12:00pm–1:00pm, Thursday 10:45am–12:45pm Kelly Hall 219
    sophia23@vt.edu

    You Lu
    Office hours: Tuesday 1:00pm–4:00pm, Kelly Hall 219
    you.lu@vt.edu

  • Office Hour Chart
    Mon. Tue. Wed. Thu. Fri.
    9:05 AM – 9:55 AM
    Class (Torg 3100).

    10 AM – 12 PM
    Bert (Torg 3160L).
    		9:15 AM – 10:45 AM
    Chidubem (Kelly 219).

    1 PM – 4 PM
    You (Kelly 219).     		9:05 AM – 9:55 AM
    Class (Torg 3100).

    12 PM – 1 PM
    Shuangfei (Kelly 219).     		9:15 AM – 10:45 AM
    Chidubem (Kelly 219).

    10:45 AM – 12:45 PM
    Shuangfei (Kelly 219).     		9:05 AM – 9:55 AM
    Class (Torg 3100).

    10 AM – 11 AM
    Bert (Torg 3160L).

The course homepage (this page) is at http://courses.cs.vt.edu/cs4824/Spring19/.

The course Canvas site is at https://canvas.vt.edu/courses/84516 and should be visible to all users with a Virginia Tech login.

Topics

  • Overview of machine learning: learning from data; overfitting, regularization, cross-validation.
  • Supervised learning: decision trees, naive Bayes, logistic regression, support vector machines, neural networks
  • Unsupervised and semi-supervised learning: clustering (k-means, Gaussian mixtures); principal components analysis
  • Learning theory: probably approximately correct (PAC) learning, model complexity, bias and variance
  • Structured models: Bayesian networks, Markov random fields, hidden Markov models
  • Deep learning: convolutional neural networks, recurrent neural networks
  • Other topics: reinforcement learning, machine learning applications (vision, natural language processing, recommendation)

Here is a flashcard game to help you study the covered concepts with classmates: http://courses.cs.vt.edu/cs4824/Spring19/flashcards/.

Prerequisites

The listed prerequisite courses cover relevant material that includes : data structures, algorithms, complexity, linear algebra, and basic concepts of probability and statistics (random variables, expectation, conditional distributions, Bayes rule, sampling distributions, estimators, likelihood, and maximum likelihood). The homework assignments will include programming portions using Python.

Please speak with the instructor if you are concerned about your background. Note: If any student needs special accommodations because of any disabilities, please contact the instructor during the first week of classes.

Learning Objectives

A student who successfully completes this class should

  • be familiar with a breadth of foundational machine learning concepts;
  • be able to implement standard machine learning methods without the use of pre-packaged machine learning software;
  • be able to make informed decisions about which machine learning methods are appropriate for different tasks;
  • have awareness of the mathematical and computer science concepts underlying machine learning;
  • and have the background knowledge to be able to understand new machine learning methods not covered in the course.

Reading and materials

  • Our main textbook will be Understanding Machine Learning. By Shai Shalev-Shwartz and Shai Ben-David. Cambridge University Press. Digital copy available for free from the authors' page.
  • Additional materials may be provided electronically, including readings, video lectures, and other media.

Homework

  • Each homework assignment will include written and programing portions.
  • Both written and programming assignments will be submitted electronically.
  • I strongly recommend learning LaTeX to help write neat, readable math, but I will not require LaTeX.
  • Programming assignments will be in Python.
  • Homework assignments will be posted on Canvas.
  • The last homework is a research project, in which you will replicate part of a recent machine learning research publication.

Project

For the final project, students will attempt to reproduce a finding from a machine learning research study. See instructions on the project page.

Schedule

The tentative class schedule is available here and is embedded below. We will update the schedule regularly.

Slides and other materials will be available in at http://courses.cs.vt.edu/cs4824/Spring19/slide_pdfs/.

Policies

Accessibility

If any student needs special accommodations because of any disabilities, please contact the instructor during the first week of classes. Such students are encouraged to work with The Office of Services for Students with Disabilities to help coordinate accessibity arrangements.

Regrading requests

Requests for regrading due to grading errors must be submitted in writing to the TA within one week of the release of grades.

Late homework policy

Homework submitted late without permission will be penalized according to the following formula: (Penalized score) = (Your raw score) * (1 - 0.1 * (# of days past deadline))

This formula will apply for up to three days, after which the homework will not be accepted and you will receive a grade of zero. Avoid invoking these penalties by starting early and seeking extra help.

Grading scale

  • 5%: Homework 0
  • 50%: Homework Sets
  • 15%: Midterm Exam
  • 15%: Final project
  • 15%: Final exam

Based on the grading breakdown above, each student's final grade for the course will be determined by the final percentage of points earned. The grade ranges are as follows:

A: 93.3%–100%, A-: 90.0%–93.3%, B+: 86.6%–90.0%, B: 83.3%–86.6%, B-: 80.0%–83.3%, C+: 76.6%–80.0%,
C: 73.3%–76.6%, C-: 70.0%–73.3%, D+: 66.6%–70.0%, D: 63.3%–66.6%, D-: 60.0%–63.3%, F: 00.0%–60.0%.

Academic integrity

The Undergraduate Honor Code pledge that each member of the university community agrees to abide by states: "As a Hokie, I will conduct myself with honor and integrity at all times. I will not lie, cheat, or steal, nor will I accept the actions of those who do."

Students enrolled in this course are responsible for abiding by the Honor Code. A student who has doubts about how the Honor Code applies to any assignment is responsible for obtaining specific guidance from the course instructor before submitting the assignment for evaluation. Ignorance of the rules does not exclude any member of the University community from the requirements and expectations of the Honor Code. For additional information about the Honor Code, please visit: https://www.honorsystem.vt.edu/

This course will have a zero-tolerance philosophy regarding plagiarism or other forms of cheating. Your homework assignments must be your own work, and any external source of code, ideas, or language must be cited to give credit to the original source. I will not hesitate to report incidents of academic dishonesty to the Office of the Undergraduate Honor System.

Principles of Community

Because the course will include in-class discussions, we will adhere to Virginia Tech's Principles of Community. The first two principles are most relevant:

  • We affirm the inherent dignity and value of every person and strive to maintain a climate for work and learning based on mutual respect and understanding.
  • We affirm the right of each person to express thoughts and opinions freely. We encourage open expression within a climate of civility, sensitivity, and mutual respect.

The remaining principles are also important and we will take them seriously as a class.