Classes – Deep Learning

Classes – Deep Learning

I will be teaching a Deep Learning Special Topics course as part of the Northwestern University Master of Science in Predictive Analytics program this Winter, 2017. Currently, as a Special Topics course, this is offered for one time only. (Doesn’t mean that we couldn’t get repeat offerings later; they’re just not scheduled right now.)

For a starting set of Reading materials, go HERE: Readings – Deep Learning.

Course Description

Deep learning has yielded multiple successful artificial intelligence (AI) applications (search, vision, translation, drug synthesis, etc.), receiving major investments from leading technology companies. It works by combining (typically neural network-based) machine learning methods with multiple representation levels, so that data is transformed through a series of processes yielding useful results. Deep learning has much in common with how the brain architecture uses multiple processing levels (as in the visual cortex), combined with feedback control mechanisms. Deep learning uses both generative and descriptive methods, where generative models learn joint probability distributions and make Bayes-rule predictions, and descriptive models learn from training examples. Students will learn which deep learning methods to select, based on dataset size, amount of known exemplars for training, classification granularity, and other factors. Students will do individual project-based work (typically using Python; optionally R) incorporating deep learning principles and practices, and will each select their own projects and datasets. Weekly readings, lecture slides, videos, discussions and example programming assignments will provide insight as to how different methods can be used and combined, contributing to project effectiveness. No final exam. Topics will include the Boltzmann machine, Deep Neural Networks, Hidden Markov Models, and other clustering and classification methods.

Prerequisites: PREDICT 422 Practical Machine Learning

Approximate Course Outline

Session 1: How the Brain Does Deep Learning and Solves Tough Problems

  • Multiple representation levels; Representation learning. Suggested Reading
  • Feedforward and feedback information flow and control loops
  • Multiple processes and sensor fusion

Session 2: State of the Art: Deep Learning and Tough Problems Today

  • Text Analytics: Google’s Knowledge Graph and RankBrain
  • Machine Vision: Face recognition, driverless cars (Google, Apple, etc.)
  • Speech Recognition; Language Translation
  • Deep Knowledge Bases: IBM’s Watson, Apple’s Siri, Cyc, and other state-of-the-art systems
  • Assess methods used for these various application

Session 3: Deep Learning; Beyond Neural Networks

  • Neural networks review; capabilities and limitations
  • Novel steps introduced via deep learning
  • Three major approaches for deep learning networks:
    • Unsupervised/generative learning
    • Supervised learning
    • Hybrid learning methods

Session 4: Unsupervised/generative learning methods

  • Clustering methods
  • Autoencoders
  • Application of autoencoders to speech recognition and other tasks

Session 5: Descriptive/Supervised learning: Backpropagation-based learning methods

Session 6: Generative/Supervised learning: Boltzmann-machine-based learning

Session 7: Hybrid learning methods

Session 8: Graphs and Deep Learning

  • Belief propagation networks; Deep belief networks (network layering/composition)
  • Directed graph models; Graph theory applications to deep learning
  • Google’s TensorFlow

Session 9: What They Don’t Teach in Any Other Deep Learning Course

  • Statistical thermodynamics and free energy: Beyond the Hopfield network and Boltzmann machines
  • Phase transitions, complexity, and order parameters – observable states in large data corpora
  • Other recent advances in deep learning algorithms/practices

Session 10: Applications Revisited (Student Presentations, based on individual research topics)


Students will use online options for various tasks, as well as have programming opportunities in both Python and R. Those students who wish to do so will have opportunities to work with the MSPA Program’s Real Time Analytics System using ElasticSearch.


There will be approximately six assigned exercises in the first six weeks of the course, giving students an opportunity to experiment with specific algorithms and/or deep learning practices, such as partitioning a classification task into multiple representation levels. Each student will select his/her own class project, and will have the option of using their own data set for the assigned exercises as well as for their final project.

Students will have options as to how they will do programming assignments: they may select whichever language and/or system that they wish to use. There will be some flexibility in the nature of the assignment to accommodate different programming preferences.

There will be no final exam. Grades will be based on exercises, discussion contributions, and final project.