# Generative AI | 10-423 + 10-623 + 10-723 | Spring 2026

https://www.cs.cmu.edu/~mgormley/courses/10423/schedule.html

  * [Home](https://www.cs.cmu.edu/~mgormley/courses/10423/)
  * [FAQ](https://www.cs.cmu.edu/~mgormley/courses/10423/faq.html)
  * [Syllabus](https://www.cs.cmu.edu/~mgormley/courses/10423/#syllabus)
  * [People](https://www.cs.cmu.edu/~mgormley/courses/10423/people.html)
  * [Schedule](https://www.cs.cmu.edu/~mgormley/courses/10423/schedule.html)
  * [Office Hours](https://www.cs.cmu.edu/~mgormley/courses/10423/officehours.html)
  * [Coursework](https://www.cs.cmu.edu/~mgormley/courses/10423/coursework.html)
  * [Previous](https://www.cs.cmu.edu/~mgormley/courses/10423/previous.html)
  * [Links](https://www.cs.cmu.edu/~mgormley/courses/10423/schedule.html)
    * [Piazza](https://piazza.com/cmu/spring2026/10423623723)
    * [Gradescope](https://www.gradescope.com/courses/1208127)
    * [Livestream](https://piazza.com/class/mjx40dwupwm28h/post/8)
    * [Video Recordings](https://scs.hosted.panopto.com/Panopto/Pages/Sessions/List.aspx#folderID=%222fe20532-6905-4f5e-b391-b3c901534e6b%22)
    * [OH Queue](https://ohq.eberly.cmu.edu/)


  * [![](https://www.cs.cmu.edu/~mgormley/courses/10423/img/scslogo_no_outline_simple.gif)](http://scs.cmu.edu)
  * [![](https://www.cs.cmu.edu/~mgormley/courses/10423/img/mllogo1.jpg)](http://ml.cmu.edu)


  * [Home](https://www.cs.cmu.edu/~mgormley/courses/10423/)
  * [FAQ](https://www.cs.cmu.edu/~mgormley/courses/10423/faq.html)
  * [Syllabus](https://www.cs.cmu.edu/~mgormley/courses/10423/#syllabus)
  * [People](https://www.cs.cmu.edu/~mgormley/courses/10423/people.html)
  * [Schedule](https://www.cs.cmu.edu/~mgormley/courses/10423/schedule.html)
  * [Office Hours](https://www.cs.cmu.edu/~mgormley/courses/10423/officehours.html)
  * [Coursework](https://www.cs.cmu.edu/~mgormley/courses/10423/coursework.html)
  * [Previous](https://www.cs.cmu.edu/~mgormley/courses/10423/previous.html)
  * [Links](https://www.cs.cmu.edu/~mgormley/courses/10423/schedule.html)
    * [Piazza](https://piazza.com/cmu/spring2026/10423623723)
    * [Gradescope](https://www.gradescope.com/courses/1208127)
    * [Livestream](https://piazza.com/class/mjx40dwupwm28h/post/8)
    * [Video Recordings](https://scs.hosted.panopto.com/Panopto/Pages/Sessions/List.aspx#folderID=%222fe20532-6905-4f5e-b391-b3c901534e6b%22)
    * [OH Queue](https://ohq.eberly.cmu.edu/)


  * [![](https://www.cs.cmu.edu/~mgormley/courses/10423/img/scslogo_no_outline_simple.gif)](http://scs.cmu.edu)
  * [![](https://www.cs.cmu.edu/~mgormley/courses/10423/img/mllogo1.jpg)](http://ml.cmu.edu)


# Generative AI
10-423 + 10-623 + 10-723, Spring 2026  
School of Computer Science  
Carnegie Mellon University  

* * *
No more lectures [ Open Latest Poll ](https://www.cs.cmu.edu/~mgormley/courses/10423/poll.html)   
  

###  [](https://www.cs.cmu.edu/~mgormley/courses/10423/schedule.html#notes)Important Notes
This schedule is **tentative** and subject to change. Please check back often.
You can access the OneNote notebook containing **all whiteboards** from lecture/recitation [here](https://www.cs.cmu.edu/~mgormley/courses/10423/schedule.html). The PDF version of each whiteboard is linked below.
###  [](https://www.cs.cmu.edu/~mgormley/courses/10423/schedule.html#schedule)Tentative Schedule  
| Date  | Lecture  | Readings  | Announcements  |  
| --- | --- | --- | --- |  
| 
#### Generative models of text
 |  
| Mon, 12-Jan  |  Lecture 1 : RNN LMs / Autodiff  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture1-overview.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture1-overview-ink.pdf)]   | 
  * _[Sequence Modeling: Recurrent and Recursive Nets](http://www.deeplearningbook.org/contents/rnn.html)_. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 10.1-10.5. 
  * _[A Framework for the Cooperation of Learning Algorithms](https://papers.nips.cc/paper/1990/file/a8c88a0055f636e4a163a5e3d16adab7-Paper.pdf)_. Léon Bottou, Patrick Gallinari (1991). Advances in Neural Information Processing Systems. 
  * _[PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://proceedings.neurips.cc/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf)_. Paszke et al. (2019). Advances in Neural Information Processing Systems. 

 |   |  
| Wed, 14-Jan  |  Lecture 2 : Transformer LMs  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture2-transformer.pdf)]   | 
  * _[The Long Short-Term Memory and Other GatedRNNs](http://www.deeplearningbook.org/contents/rnn.html)_. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 10.10-10.12. 
  * _[Attention Is All You Need](https://arxiv.org/abs/1706.03762)_. Vaswani et al. (2017). NeurIPS. 
  * _[The Illustrated Transformer](https://jalammar.github.io/illustrated-transformer/)_. Alammar (2018). 

 |  HW0 out   |  
| Fri, 16-Jan  |  Recitation: HW0  
[[Handout](https://colab.research.google.com/drive/1F-ik4J0hf8kUdQAH_1HdlpBufuF9j9ny?usp=sharing)]   |   |   |  
| Mon, 19-Jan  |  (MLK Day - No Class)  
 |   |   |  
| Wed, 21-Jan  |  Lecture 3 : Learning LLMs / Decoding  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture3-llms.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture3-llms-ink.pdf)]   | 
  * _[Generating Sequences With Recurrent Neural Networks](https://arxiv.org/pdf/1308.0850.pdf)_. Graves (2014). 
  * _[Recurrent neural network based language model](https://pdfs.semanticscholar.org/bba8/a2c9b9121e7c78e91ea2a68630e77c0ad20f.pdf)_. Mikolov et al. (2010). Interspeech. 
  * [_GPT-1_] _[Improving Language Understanding by Generative Pre-Training](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf)_. Radford et al. (2018). 
  * [_GPT-2_] _[Language Models are Unsupervised Multitask Learners](https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf)_. Radford et al. (2019). 

 |   |  
| Fri, 23-Jan  |  (No Recitation)  
 |   |   |  
| Mon, 26-Jan  |  Lecture 4 : Pre-training, fine-tuning / Modern Transformers  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture4-rope-gqa.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture4-rope-gqa-ink.pdf)]   | 
  * _[GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://arxiv.org/pdf/2305.13245.pdf)_. Ainslie et al. (2023). EMNLP. 
  * _[Longformer: The Long-Document Transformer](https://arxiv.org/pdf/2004.05150.pdf)_. Beltagy et al. (2020). 
  * _[RoFormer: Enhanced Transformer with Rotary Position Embedding](https://arxiv.org/pdf/2104.09864.pdf)_. Su et al. (2021). 

 |  HW0 Slot A due  HW1 out (L1-L4)   |  
| 
#### Generative models of images
 |  
| Wed, 28-Jan  |  Lecture 5 : Computer Vision: CNNs / Encoder-only Transformers / Vision Transformers  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture5-cnn-vit.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture5-cnn-vit-ink.pdf)]   | 
  * _[Convolutional Networks](http://www.deeplearningbook.org/contents/convnets.html)_. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 9.1 - 9.3. 
  * _[BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/pdf/1810.04805.pdf)_. Devlin et al. (2018). NAACL. 
  * _[An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://arxiv.org/pdf/2010.11929.pdf)_. Dosovitskiy et al. (2021). ICLR. 

 |  Quiz 1 (in-class, L1-L4)   |  
| Thu, 29-Jan  |    
 |   |  HW0 Feedback (tentative)   |  
| Fri, 30-Jan  |  Recitation: HW1  
[[Handout](https://docs.google.com/presentation/d/1IpSzQ5dkr3iO0riNfareQiif9J9O684amTBATiybuVk/edit?usp=sharing)]  [[Supplemental Material](https://drive.google.com/file/d/1PIaZ5eem1epaQRcR7Gy_XVW-QuMhmtTs/view?usp=sharing)]   |   |   |  
| Sun, 1-Feb  |    
 |   |  HW0 Slot B due (tentative)   |  
| Mon, 2-Feb  |  Lecture 6 : Generative Adversarial Networks (GANs) / PGM  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture6-gans.pdf)]   | 
  * _[Generative Adversarial Nets](https://arxiv.org/pdf/1406.2661.pdf)_. Goodfellow et al. (2014). NeurIPS. 
  * _[NuerIPS 2016 Tutorial: Generative Adversarial Networks](https://arxiv.org/pdf/1701.00160.pdf)_. Goodfellow (2017). NeurIPS Tutorials. 

 |   |  
| Wed, 4-Feb  |  Lecture 7 : Diffusion models (Part I)  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture7-diffusion.pdf)]   | 
  * _[Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://arxiv.org/pdf/1503.03585.pdf)_. Sohl-Dickstein et al. (2015). MLR. 
  * _[Denoising Diffusion Probabilistic Models](https://proceedings.neurips.cc/paper/2020/hash/4c5bcfec8584af0d967f1ab10179ca4b-Abstract.html)_. Ho et al. (2020). NeurIPS. 

 |   |  
| Fri, 6-Feb  |  (No Recitation)  
 |   |   |  
| Mon, 9-Feb  |  Lecture 8 : Diffusion models (Part II)  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture8-diffusion-vae.pdf)]   | 
  * _[Variational Inference: A Review for Statisticians](https://arxiv.org/pdf/1601.00670.pdf)_. Blei, Kucukelbir, and McAuliffe (2018). 
  * _[High-Level Explanation of Variational Inference](https://www.cs.jhu.edu/~jason/tutorials/variational.html)_. Jason Eisner (2011). 
  * _[Tutorial on Variational Autoencoders](https://arxiv.org/pdf/1606.05908.pdf)_. Carl Doersch (2016). 

 |  HW1 Slot A due  HW2 out (L5-L8)   |  
| 
#### Applying and adapting foundation models
 |  
| Wed, 11-Feb  |  Lecture 9 : Variational Autoencoders (VAEs)  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture9-vae-icl.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture9-vae-icl-ink.pdf)]   |   |   |  
| Fri, 13-Feb  |  Recitation: HW2  
[[Handout](https://docs.google.com/presentation/d/1BR_VlKfF-vupvBGgaVRr0oDnA1esJLmk/edit?usp=sharing&ouid=105692313169777035274&rtpof=true&sd=true)]   |   |   |  
| Sat, 14-Feb  |    
 |   |  HW1 Feedback (tentative)   |  
| Mon, 16-Feb  |  Lecture 10 : Parameter-efficient fine tuning  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture10-peft.pdf)]   |   |  Quiz 2 (in-class, L5-L9)   |  
| Tue, 17-Feb  |    
 |   |  HW1 Slot B due (tentative)   |  
| Wed, 18-Feb  |  Lecture 11 : In-Context Learning / Prompt Engineering / Instruction Fine-tuning / Reinforcement learning with human feedback (RLHF)  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture11-ift-rlhf.pdf)]   |   |  Project description out   |  
| Fri, 20-Feb  |  Recitation: HW3  
[[Handout](https://docs.google.com/presentation/d/1qN6VYOxE-YbT1tGA5NGXdBC6ev5PnHx417weyTRCfqA/edit?usp=sharing)]   |   |   |  
| Sat, 21-Feb  |    
 |   |  HW2 Slot A due  HW3 out (L9-L12)   |  
| 
#### Multimodal foundation models
 |  
| Mon, 23-Feb  |  Lecture 12 : Direct Preference Optimization (DPO) / Text-to-image generation / Latent diffusion model  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture12-dpo-text2img.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture12-dpo-text2img-ink.pdf)]   |   |   |  
| Wed, 25-Feb  |  Programming Test HW1/HW2  
 |   |  (Quiz 3 in-class, L9-L12)   |  
| Thu, 26-Feb  |    
 |   |  HW2 Feedback (tentative)   |  
| Fri, 27-Feb  |  Lecture 13 : Vision-language models  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture13-vlm.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture13-vlm-ink.pdf)]   |   |  Project Team Formation form released   |  
| Sun, 1-Mar  |    
 |   |  HW2 Slot B due (tentative)   |  
| Mon, 2-Mar  |  Spring Break  
 |   |   |  
| Tue, 3-Mar  |    
 |   |   |  
| Wed, 4-Mar  |  Spring Break  
 |   |   |  
| Thu, 5-Mar  |    
 |   |   |  
| Fri, 6-Mar  |  Spring Break  
 |   |   |  
| Mon, 9-Mar  |  Lecture 14 : Cross-Attention / Diffusion Transformer / Prompt-to-Prompt  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture14-ldm-dit-p2p.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture14-ldm-dit-p2p-ink.pdf)]   |   |   |  
| 
#### Scaling Up
 |  
| Wed, 11-Mar  |  Lecture 15 : Querying Transformer / Scaling Laws  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture15-querying-scaling.pdf)]   |   |   |  
| Thu, 12-Mar  |    
 |   |  HW3 Slot A due  HW4 out (L12-L14)   |  
| Fri, 13-Mar  |  Recitation: HW4  
[[Handout](https://docs.google.com/presentation/d/1oC6MwwnIM8BSjSpNY4-V4KwW2KBaXwHN/edit?slide=id.g39c497972fa_1_210#slide=id.g39c497972fa_1_210)]   |   |  Project team formation due by 2pm   |  
| Mon, 16-Mar  |  Lecture 16 : Mixture of Experts  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture16-moe.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture16-moe-ink.pdf)]   |   |  (Quiz 4 in-class, L12-L15)   |  
| Tue, 17-Mar  |    
 |   |  HW3 Feedback (tentative)   |  
| Wed, 18-Mar  |  Lecture 17 : Distributed training  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture17-distributed.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture17-distributed-ink.pdf)]   |   |   |  
| Fri, 20-Mar  |  (No Recitation)  
 |   |  HW3 Slot B due (tentative)   |  
| Mon, 23-Mar  |  Lecture 18 : Flash Attention / Efficient decoding strategies  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture18-efficient.pdf)]   |   |  HW4 Slot A due  HW623 out; Practice problems out   |  
| 
#### Advanced Topics
 |  
| Wed, 25-Mar  |  Lecture 19 : Long Context in LLM  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture19-long.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture19-long-ink.pdf)]   |   |   |  
| Fri, 27-Mar  |  Programing Test HW3/HW4  
 |   |   |  
| Sat, 28-Mar  |    
 |   |  HW4 Feedback (tentative)   |  
| Mon, 30-Mar  |  Lecture 20 : Reasoning Models  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture20-reasoning.pdf)]   |   |   |  
| Mon, 30-Mar  |  Exam (evening exam, details will be announced on Piazza)  
 |   |  HW4 Slot B due (tentative)   |  
| Wed, 1-Apr  |  Lecture 21 : State Space Models / Hybrid Models  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture21-ssm.pdf)]   |   |   |  
| Fri, 3-Apr  |  (No Recitation)  
 |   |  Project proposal due   |  
| Mon, 6-Apr  |  Lecture 22 : Real-world Issues and Considerations / What can go wrong?  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture22-practical-considerations.pdf)]   |   |  (Quiz 5 in-class, L16-L20)   |  
| Wed, 8-Apr  |  Lecture 23 : Code Generation / Autonomous Agents  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture23-code.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture23-code-ink.pdf)]   |   |   |  
| Thu, 9-Apr  |  Spring Carnival  
 |   |   |  
| Fri, 10-Apr  |  Spring Carnival  
 |   |   |  
| Mon, 13-Apr  |  Lecture 24 : Audio understanding and synthesis  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture24-audio.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture24-audio-ink.pdf)]   |   |  Project midway report due   |  
| Wed, 15-Apr  |  Lecture 25 : Generative Models for Videos  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture25-video.pdf)]   |   |   |  
| Fri, 17-Apr  |  (No Recitation)  
 |   |   |  
| Mon, 20-Apr  |  Lecture 26 : Interactive World Models + Science of Alignment  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture26-world-models.pdf)] [[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423//slides/lecture26-alignment.pdf)]   |   |  HW623 due; (Quiz 6 in-class, L21-L24)   |  
| Wed, 22-Apr  |  (No lecture)  
 |   |   |  
| Fri, 24-Apr  |  (No Recitation)  
 |   |   |  
| Sun, 26-Apr  |    
 |   |  Project poster due   |  
| Tue, 28-Apr  |  Project Final Presentations 5:30pm - 8:30pm GHC 6121 + GHC 6115  
 |   |   |  
| Thu, 30-Apr  |    
 |   |  Project final report due   |  
* * *
Last updated April 20, 2026 (03:56 PM).
This website is powered by [Jekyll](https://jekyllrb.com/) and [Bootstrap](http://getbootstrap.com).