Parallel and Customized Computer Architecture Course

“Parallel and Customized Computer Architecture” course (ECE-GY 9413) at NYU Tandon by Brandon Reagen

The “Parallel and Customized Computer Architecture” course (ECE-GY 9413) at NYU Tandon by Brandon Reagen offers a deep dive into the cutting-edge developments that are steering the future of computer design. This course addresses the pivotal shift from Dennard’s scaling, exploring the innovations that have emerged in response to the limitations imposed by advanced process technology, focusing on the rise of multi-core CPUs, GPUs, and specialized hardware accelerators like Tensor Processing Units (TPUs).

Course Overview

Understanding the Shift to Parallelism and Specialization

The course begins with an exploration of the fundamental reasons behind the shift from traditional single-threaded processors to multi-core and many-core systems. Students will delve into the principles of parallelism, examining how multi-core CPUs and GPUs operate, their architectural differences, and the scenarios in which each excels.

Specialized Hardware Accelerators

Next, the course focuses on specialized hardware, covering various accelerators designed for specific tasks. For instance, it highlights how TPUs are tailored for machine learning workloads, providing insights into how specialization can significantly enhance performance and efficiency for particular applications.

In-Depth Paper Discussions

A substantial portion of the course is dedicated to reading and discussing seminal papers that have shaped contemporary computer architecture. These papers cover topics ranging from innovative processor designs to the latest advancements in hardware accelerators. Students are required to submit reports summarizing each paper, forming the basis for in-class discussions. Each session features a student presentation on a selected paper, emphasizing its motivation, key findings, and impact on the field.

Core Lectures on Advanced Topics

Complementing the paper discussions, core lectures provide foundational knowledge and introduce advanced topics in computer architecture. These lectures ensure that all students have a solid grounding in the principles and are well-prepared for more complex discussions.

Practical Implementation: Class Project

To reinforce the theoretical knowledge gained, students will undertake a class project. This year’s project involves implementing a cycle-level performance simulator for a vector machine, offering practical experience in performance simulation and a deeper understanding of vector machine architecture.

Readings

Vector Processor

Vector Vs. Superscalar and VLIW Architectures for Embedded Multimedia Benchmarks

Overcoming the Limitations of Conventional Vector Processors

A “New Ara” for Vector Computing: An Open Source Highly Efficient RISC-V V1.0 Vector Processor Design

WaveScalar

Dark Silicon and the End of Multicore Scaling

Understanding Sources of Inefficiency in General-Purpose Chip

The Raw Microprocessor

Polymorphous TRIPS Architecture

Bridging the Computation Gap Between Programmable Processors and Hardwired Accelerators

MANIC - A Vector-Dataflow Architecture for Ultra-Low-Power Embedded Systems

EIE - Efficient Inference Engine on Compressed Deep Neural Network

Eyeriss - A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks

Systolic Architectures

TPU

Graphicionado

GenAx - A Genome Sequencing Accelerator

Gemmini

PRIME

RPU

Characterizing and Optimizing End-to-End Systems for Private Inference

HAAC

Course Review

The “Parallel and Customized Computer Architecture” course by Prof. Brandon was the best course I took at NYU. Prof. Brandon is an outstanding teacher who made complex topics accessible and engaging. The course involved reading and discussing research papers, which was where the most significant learning occurred. These discussions were truly enlightening. The hands-on approach, particularly the semester-long project of developing a Vector Processor simulator, was the highlight, allowing us to apply theoretical knowledge practically. I highly recommend this course to anyone interested in learning the cutting-edge advancements in computer architecture.