Most online AI tutorials stop at writing clever prompts inside a browser window. Harvard University's graduate course, CS2680: Modern AI Systems, approaches artificial intelligence from the opposite direction: how autonomous agents and serving engines actually execute across computer systems. Taught by Assistant Professor Juncheng Yang at Harvard SEAS, the course connects high-level multi-agent workflows down to the underlying GPU memory, KV caching, and network scheduling layers.
Harvard SEAS graduate course bridging agent logic and low-level GPU systems.
Curriculum structure: half autonomous agent loops, half LLM serving infrastructure.
Grading based entirely on progressive programming systems assignments.
1. What CS2680 is, and what it will not teach you
Harvard's CS2680 is not a course on prompt engineering, copywriting, or generic productivity hacks. It is a rigorous computer systems class designed for engineers who want to understand the full stack of modern generative computing.
Students trace requests from the moment a user submits an intent, through the agent execution loop and tool calling runtime, down to how model weights and attention caches are partitioned across hardware clusters. There are no written exams; grades rely entirely on programming assignments and a comprehensive final systems research project.
2. Half one: agents
The first half of the syllabus focuses on the agentic application layer:
- The agent loop: How autonomous runtimes break goals into steps, manage execution state, and invoke external APIs.
- Context and tool design: Structuring tool definitions and system prompts to minimize token overhead while preventing catastrophic hallucination.
- Multi-agent orchestration: Designing patterns where specialized agents (such as planners, coders, and reviewers) coordinate without suffering deadlock or exponential token compounding.
- Memory and evaluation: Implementing short-term context compression, long-term memory retrieval, and deterministic evaluation pipelines to measure reliability.
Students analyze state-of-the-art coding agents like Claude Code, learning how industrial tools manage file systems, execute terminal commands, and recover from syntax errors.
3. Half two: serving
The second half dives deep into the high-performance infrastructure required to serve large language models economically:
- Batching and scheduling: How inference servers group incoming queries to maximize GPU compute saturation without inflating latency.
- KV-cache optimization: Managing key-value attention caches in VRAM, utilizing techniques like PagedAttention and prompt prefix caching to slash redundant compute.
- Model compression: Quantization (FP8, INT4) and pruning strategies that fit massive parameter models onto constrained hardware budgets.
- Speculative decoding and routing: Accelerating generation speed by predicting tokens with lightweight draft models, alongside dynamic routing across multi-tiered model clusters.
4. The assignments: build one agent, then make it faster, cheaper and observable
Rather than toy homework problems, the curriculum centers on a continuous software artifact that students evolve throughout the semester:
- 01
1. Build the core agent loop
Construct an autonomous runtime that decomposes user requests, calls external tools, and handles API errors.
- 02
2. Instrument full observability
Add session tracing to log token consumption, latency bottlenecks, and tool failure rates across turns.
- 03
3. Optimize token efficiency
Implement prompt prefix caching and structured tool filtering to cut unnecessary API token waste.
- 04
4. Tune the serving backend
Deploy local model weights, implement KV-cache paging, and evaluate speculative decoding throughput.
5. Who should take it, and who can skip it
CS2680 requires strong systems programming proficiency in Python, PyTorch, and low-level computer architecture. If you are an infrastructure engineer or systems architect building enterprise AI platforms, following the public syllabus and lecture notes will give you an invaluable foundation.
However, if you are a business owner or non-technical founder who simply wants to automate back-office operations, generate sales leads, or connect WhatsApp to your CRM, you do not need to study KV caches or GPU kernel scheduling.
For practical, business-focused workflow automation using reliable platforms like n8n, visit our dedicated training at course.autopilot-studio.com.
