Rugved Mhatre

MS. Computer Engineering @NYU Ex-Oracle

1 minute read

“High Performance Machine Learning” course (ECE-GY 9143) at NYU Tandon by Parijat Dube and Zehra Sura

nyu-header

The “High Performance Learning” course (ECE-GY 9143) at NYU Tandon by Parijat Dube and Zehra Sura offers a deeper dive into High Performance Computing for accelerating Maching Learning Algorithms.

Course Overview

This course explores High-Performance Computing (HPC) and its role in accelerating Machine Learning (ML) algorithms. It covers techniques traditionally used in supercomputing, such as GPUs and low-latency interconnects, to optimize ML performance. With the rise of large foundation models like GPT and LLAMA, efficient AI computing has become crucial. The course introduces methods for improving ML efficiency, including model compression, pruning, quantization, knowledge distillation, neural architecture search, data/model parallelism, and distributed training.

Course Work and Grading

Note: This is subject to change

  • Homework (5 assignments) - 30%
  • Final Project - 30%
  • Final Exam - 30%
  • Quizzes - 10%

Course Notes

Notes are available here.

Review

The High-Performance Machine Learning course is an intensive and highly rewarding deep dive into performance optimization and profiling for machine learning workloads. It is a must-take course for anyone interested in extracting maximum efficiency from hardware to support increasingly complex ML models.

One of the standout aspects of this course is its strong industry relevance. The professors provide valuable insights into real-world practices, bridging the gap between theoretical concepts and practical implementation. This makes the course not only academically enriching but also highly applicable to industry needs.

The assignments are challenging yet incredibly educational, functioning more like full-scale projects rather than traditional coursework. Through these hands-on experiences, students develop a deep understanding of optimization techniques, working with PyTorch and CUDA to implement cutting-edge methods such as model compression, quantization, and parallelism.

Overall, while the course demands significant effort, the knowledge and skills gained are well worth it. It is an essential course for anyone looking to specialize in high-performance computing for machine learning.

You May Also Enjoy