# Generative AI | 10-423 + 10-623 + 10-723 | Fall 2025

https://www.cs.cmu.edu/~mgormley/courses/10423-f25/schedule.html

  * [Home](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/)
  * [FAQ](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/faq.html)
  * [Syllabus](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/#syllabus)
  * [People](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/people.html)
  * [Schedule](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/schedule.html)
  * [Office Hours](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/officehours.html)
  * [Coursework](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/coursework.html)
  * [Previous](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/previous.html)
  * [Links](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/schedule.html)
    * [Piazza](https://piazza.com/cmu/fall2025/10423623723)
    * [Gradescope](https://www.gradescope.com/courses/1079227)
    * [Livestream](https://piazza.com/class/me79mzsvkpg2q6/post/8)
    * [Video Recordings](https://scs.hosted.panopto.com/Panopto/Pages/Sessions/List.aspx?folderID=9042de5b-f2ce-4e07-b974-b33f00ff5eb3)
    * [OH Queue](https://ohq.eberly.cmu.edu/)


  * [![](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/img/scslogo_no_outline_simple.gif)](http://scs.cmu.edu)
  * [![](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/img/mllogo1.jpg)](http://ml.cmu.edu)


  * [Home](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/)
  * [FAQ](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/faq.html)
  * [Syllabus](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/#syllabus)
  * [People](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/people.html)
  * [Schedule](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/schedule.html)
  * [Office Hours](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/officehours.html)
  * [Coursework](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/coursework.html)
  * [Previous](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/previous.html)
  * [Links](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/schedule.html)
    * [Piazza](https://piazza.com/cmu/fall2025/10423623723)
    * [Gradescope](https://www.gradescope.com/courses/1079227)
    * [Livestream](https://piazza.com/class/me79mzsvkpg2q6/post/8)
    * [Video Recordings](https://scs.hosted.panopto.com/Panopto/Pages/Sessions/List.aspx?folderID=9042de5b-f2ce-4e07-b974-b33f00ff5eb3)
    * [OH Queue](https://ohq.eberly.cmu.edu/)


  * [![](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/img/scslogo_no_outline_simple.gif)](http://scs.cmu.edu)
  * [![](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/img/mllogo1.jpg)](http://ml.cmu.edu)


# Generative AI
[ For information about 10-x23 in Spring 2026, click here!  ](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/nextsem.html)
10-423 + 10-623 + 10-723, Fall 2025  
School of Computer Science  
Carnegie Mellon University  

* * *
Jump to Latest (Lecture 12)  [ Open Latest Poll ](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/poll.html)   
  

###  [](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/schedule.html#notes)Important Notes
This schedule is **tentative** and subject to change. Please check back often.
You can access the OneNote notebook containing **all whiteboards** from lecture/recitation [here](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/schedule.html). The PDF version of each whiteboard is linked below.
###  [](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/schedule.html#schedule)Tentative Schedule  
| Date  | Lecture  | Readings  | Announcements  |  
| --- | --- | --- | --- |  
| 
#### Generative models of text
 |  
| Mon, 25-Aug  |  Lecture 1 : RNN LMs / Autodiff  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture1-overview.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture1-overview-ink.pdf)]   | 
  * _[Sequence Modeling: Recurrent and Recursive Nets](http://www.deeplearningbook.org/contents/rnn.html)_. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 10.1-10.5. 
  * _[A Framework for the Cooperation of Learning Algorithms](https://papers.nips.cc/paper/1990/file/a8c88a0055f636e4a163a5e3d16adab7-Paper.pdf)_. Léon Bottou, Patrick Gallinari (1991). Advances in Neural Information Processing Systems. 
  * _[PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://proceedings.neurips.cc/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf)_. Paszke et al. (2019). Advances in Neural Information Processing Systems. 

 |   |  
| Wed, 27-Aug  |  Lecture 2 : Transformer LMs  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture2-transformer.pdf)]   | 
  * _[The Long Short-Term Memory and Other GatedRNNs](http://www.deeplearningbook.org/contents/rnn.html)_. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 10.10-10.12. 
  * _[Attention Is All You Need](https://arxiv.org/abs/1706.03762)_. Vaswani et al. (2017). NeurIPS. 
  * _[The Illustrated Transformer](https://jalammar.github.io/illustrated-transformer/)_. Alammar (2018). 

 |  HW0 out   |  
| Fri, 29-Aug  |  Recitation: HW0  
[[Handout](https://colab.research.google.com/drive/1mRG3B9ODxqoze-q8Y47TrItm7DZv8pgy?usp=sharing)]   |   |   |  
| Mon, 1-Sep  |  (Labor Day - No Class)  
 |   |   |  
| Wed, 3-Sep  |  Lecture 3 : Learning LLMs / Decoding  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture3-llms.pdf)]   | 
  * _[Generating Sequences With Recurrent Neural Networks](https://arxiv.org/pdf/1308.0850.pdf)_. Graves (2014). 
  * _[Recurrent neural network based language model](https://pdfs.semanticscholar.org/bba8/a2c9b9121e7c78e91ea2a68630e77c0ad20f.pdf)_. Mikolov et al. (2010). Interspeech. 
  * [_GPT-1_] _[Improving Language Understanding by Generative Pre-Training](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf)_. Radford et al. (2018). 
  * [_GPT-2_] _[Language Models are Unsupervised Multitask Learners](https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf)_. Radford et al. (2019). 

 |   |  
| Fri, 5-Sep  |  (No Recitation)  
 |   |   |  
| Mon, 8-Sep  |  Lecture 4 : Pre-training, fine-tuning / Modern Transformers  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture4-rope-gqa.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture4-rope-gqa-ink.pdf)]   | 
  * _[GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints](https://arxiv.org/pdf/2305.13245.pdf)_. Ainslie et al. (2023). EMNLP. 
  * _[Longformer: The Long-Document Transformer](https://arxiv.org/pdf/2004.05150.pdf)_. Beltagy et al. (2020). 
  * _[RoFormer: Enhanced Transformer with Rotary Position Embedding](https://arxiv.org/pdf/2104.09864.pdf)_. Su et al. (2021). 

 |  HW0 due  HW1 out (L1-L4)   |  
| 
#### Generative models of images
 |  
| Wed, 10-Sep  |  Lecture 5 : Computer Vision: CNNs / Encoder-only Transformers / Vision Transformers  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture5-vision.pdf)]   | 
  * _[Convolutional Networks](http://www.deeplearningbook.org/contents/convnets.html)_. Ian Goodfellow, Yoshua Bengio, & Aaron Courville (2016). Deep Learning, Chapter 9.1 - 9.3. 
  * _[BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/pdf/1810.04805.pdf)_. Devlin et al. (2018). NAACL. 
  * _[An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://arxiv.org/pdf/2010.11929.pdf)_. Dosovitskiy et al. (2021). ICLR. 

 |  Quiz 1 (in-class, L1-L4)   |  
| Fri, 12-Sep  |  Recitation: HW1  
[[Handout](https://docs.google.com/presentation/d/1IE8JGo4Rfki9X6iVlfAVyTJR03NvQGq6q8gCd3By73M/edit?usp=sharing)]   |   |   |  
| Mon, 15-Sep  |  Lecture 6 : Generative Adversarial Networks (GANs) / PGM  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture6-gans.pdf)]   | 
  * _[Generative Adversarial Nets](https://arxiv.org/pdf/1406.2661.pdf)_. Goodfellow et al. (2014). NeurIPS. 
  * _[NuerIPS 2016 Tutorial: Generative Adversarial Networks](https://arxiv.org/pdf/1701.00160.pdf)_. Goodfellow (2017). NeurIPS Tutorials. 

 |   |  
| Wed, 17-Sep  |  Lecture 7 : Diffusion models (Part I)  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture7-diffusion.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture7-diffusion-ink.pdf)]   | 
  * _[Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://arxiv.org/pdf/1503.03585.pdf)_. Sohl-Dickstein et al. (2015). MLR. 
  * _[Denoising Diffusion Probabilistic Models](https://proceedings.neurips.cc/paper/2020/hash/4c5bcfec8584af0d967f1ab10179ca4b-Abstract.html)_. Ho et al. (2020). NeurIPS. 

 |   |  
| Fri, 19-Sep  |  (No Recitation)  
 |   |   |  
| Mon, 22-Sep  |  Lecture 8 : Diffusion models (Part II)  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture8-diffusion-vae.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture8-diffusion-vae-ink.pdf)]   | 
  * _[Variational Inference: A Review for Statisticians](https://arxiv.org/pdf/1601.00670.pdf)_. Blei, Kucukelbir, and McAuliffe (2018). 
  * _[High-Level Explanation of Variational Inference](https://www.cs.jhu.edu/~jason/tutorials/variational.html)_. Jason Eisner (2011). 
  * _[Tutorial on Variational Autoencoders](https://arxiv.org/pdf/1606.05908.pdf)_. Carl Doersch (2016). 

 |  HW1 due  HW2 out (L5-L8)   |  
| 
#### Applying and adapting foundation models
 |  
| Wed, 24-Sep  |  Lecture 9 : Variational Autoencoders (VAEs)  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture9-vae-icl.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture9-vae-icl-ink.pdf)]   |   |   |  
| Fri, 26-Sep  |  Recitation: HW2  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25/handouts/sep26_recitation.pdf)]   |   |   |  
| Mon, 29-Sep  |  Lecture 10 : Parameter-efficient fine tuning  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture10-peft.pdf)]   |   |  Quiz 2 (in-class, L5-L9)   |  
| Wed, 1-Oct  |  Lecture 11 : In-Context Learning / Prompt Engineering / Instruction Fine-tuning / Reinforcement learning with human feedback (RLHF)  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture11-ift-rlhf.pdf)]   |   |  Project description out   |  
| Fri, 3-Oct  |  Recitation: HW3  
[[Handout](https://docs.google.com/presentation/d/1z0d00J9MhTYH2alT2Od1LD6TaNm4LU6vHX-RvIkwLD8/edit?usp=sharing)]   |   |   |  
| Sat, 4-Oct  |    
 |   |  HW2 due  HW3 out (L9-L12)   |  
| Sun, 5-Oct  |    
 |   |   |  
| 
#### Multimodal foundation models
 |  
| Mon, 6-Oct  |  Lecture 12 : Direct Preference Optimization (DPO) / Text-to-image generation / Latent diffusion model  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture12-dpo-text2img.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture12-dpo-text2img-ink.pdf)]   |   |   |  
| Wed, 8-Oct  |  Lecture 13 : Vision-language models  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture13-vlm.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture13-vlm-ink.pdf)]   |   |  (Quiz 3 in-class, L9-L12)   |  
| Fri, 10-Oct  |  (No Recitation)  
 |   |  For future semesters: Project Team Formation form released   |  
| Mon, 13-Oct  |  Fall break  
 |   |   |  
| Tue, 14-Oct  |    
 |   |   |  
| Wed, 15-Oct  |  Fall break  
 |   |   |  
| Thu, 16-Oct  |    
 |   |   |  
| Fri, 17-Oct  |  Fall break  
 |   |   |  
| Mon, 20-Oct  |  Lecture 14 : Latent diffusion model / Diffusion Transformer / Prompt-to-Prompt  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture14-ldm-dit-p2p.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture14-ldm-dit-p2p-ink.pdf)]   |   |   |  
| 
#### Scaling Up
 |  
| Wed, 22-Oct  |  Lecture 15 : Querying Transformer / Scaling Laws  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture15-querying-scaling.pdf)]   |   |   |  
| Thu, 23-Oct  |    
 |   |  HW3 due  HW4 out (L12-L14)   |  
| Fri, 24-Oct  |  Recitation: HW4  
[[Handout](https://docs.google.com/presentation/d/1OaZ6OLIlTNQOlcCRDf0qfTZfZzYPoSRR/edit?usp=sharing)]   |   |  Project team formation due by 2pm   |  
| Mon, 27-Oct  |  Lecture 16 : Mixture of Experts  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture16-moe.pdf)]   |   |  (Quiz 4 in-class, L12-L15)   |  
| Wed, 29-Oct  |  Lecture 17 : Distributed training  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture17-distributed.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture17-distributed-ink.pdf)]   |   |   |  
| Fri, 31-Oct  |  (No Recitation)  
 |   |   |  
| Mon, 3-Nov  |  Lecture 18 : Flash Attention / Efficient decoding strategies  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture18-efficient.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture18-efficient-ink.pdf)]   |   |  HW4 due  HW623 out; Practice problems out   |  
| 
#### Advanced Topics
 |  
| Wed, 5-Nov  |  Lecture 19 : Long Context in LLM  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture19-long.pdf)]   |   |   |  
| Fri, 7-Nov  |  (No Recitation)  
 |   |   |  
| Mon, 10-Nov  |  Lecture 20 : Reasoning Models  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture20-reasoning.pdf)]   |   |   |  
| Mon, 10-Nov  |  Exam (evening exam, details will be announced on Piazza Starting at 7pm Ending at 9pm)  
 |   |   |  
| Wed, 12-Nov  |  Lecture 21 : State Space Models / Hybrid Models  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture21-ssm.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture21-ssm-ink.pdf)]   |   |   |  
| Thu, 13-Nov  |    
 |   |   |  
| Fri, 14-Nov  |  (No Recitation)  
 |   |  Project proposal due   |  
| Sat, 15-Nov  |    
 |   |   |  
| Mon, 17-Nov  |  Lecture 22 : Real-world Issues and Considerations / What can go wrong?  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture22-practical-considerations.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture22-practical-considerations-ink.pdf)]   |   |  (Quiz 5 in-class, L16-L20)   |  
| Tue, 18-Nov  |    
 |   |   |  
| Wed, 19-Nov  |  Lecture 23 : Code Generation / Autonomous Agents  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture23-code.pdf)]   |   |   |  
| Fri, 21-Nov  |  (No Recitation)  
 |   |   |  
| Mon, 24-Nov  |  Lecture 24 : Audio understanding and synthesis  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture24-audio.pdf)] [[Slides (Inked)](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture24-audio-ink.pdf)]   |   |  Project midway report due   |  
| Wed, 26-Nov  |  Thanksgiving Break - No Class  
 |   |   |  
| Thu, 27-Nov  |  Thanksgiving Break - No Class  
 |   |   |  
| Fri, 28-Nov  |  Thanksgiving Break - No Class  
 |   |   |  
| Mon, 1-Dec  |  Lecture 25 : Generative Models for Videos  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture25-video.pdf)]   |   |  HW623 due; (Quiz 6 in-class, L21-L24 for 10-723 only)   |  
| Wed, 3-Dec  |  Lecture 26 : Interactive World Models + Science of Alignment  
[[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture26-world-models.pdf)] [[Slides](https://www.cs.cmu.edu/~mgormley/courses/10423-f25//slides/lecture26-alignment.pdf)]   |   |   |  
| Fri, 5-Dec  |  (No Recitation)  
 |   |   |  
| Sun, 7-Dec  |    
 |   |  Project poster due   |  
| Tue, 9-Dec  |  Project Final Presentations 8:30am - 11:30am GHC 6121 + GHC 6115  
 |   |   |  
| Thu, 11-Dec  |    
 |   |  Project final report due   |  
* * *
Last updated January 11, 2026 (08:01 PM).
This website is powered by [Jekyll](https://jekyllrb.com/) and [Bootstrap](http://getbootstrap.com).