# Schedule - Large Language Models: Methods and Applications / Fall 2025 https://2025.cmu-llms.org/schedule/ | ![](https://2025.cmu-llms.org/_images/logo.png) | [Carnegie Mellon University](https://www.lti.cs.cmu.edu/) [**Large Language Models: Methods and Applications**](https://2025.cmu-llms.org/ "Large Language Models: Methods and Applications / Fall 2025 - Carnegie Mellon University") Fall 2025 | | --- | --- | [ ](https://2025.cmu-llms.org/schedule/) # Main Navigation * [ Home ](https://2025.cmu-llms.org/) * [ Syllabus ](https://2025.cmu-llms.org/syllabus/) * [ Schedule ](https://2025.cmu-llms.org/schedule/) * [ Assignments ](https://2025.cmu-llms.org/assignments/) * [ Useful Links ](https://2025.cmu-llms.org/materials/) # Schedule The schedule is updated on the onging basis as the semester progresses. * Event Date Description Course Material * Assignment 08/25/2025 Monday Assignment #0 released! [[Assignment #0](https://2025.cmu-llms.org/assignments/00)] * Lecture 08/26/2025 Tuesday Building blocks of modern LLMs [[Slides (class overview)](https://cmu.box.com/s/024zqfi8eqhwtztr5i3o9hdxr6gdrvth%20%0A%20%20%20%20 "Download Slides \(class overview\)")] [[Slides (lm basics)](https://cmu.box.com/s/2dwogftdasf3ulx83760gg8ry2z6k1aw%20%0A%20%20%20%20 "Download Slides \(lm basics\)")] Readings: * [Jurasky and Martin, Chapter 7](https://web.stanford.edu/~jurafsky/slp3/7.pdf) * [Illustrated Transformer](http://jalammar.github.io/illustrated-transformer/) * Lecture 08/28/2025 Thursday Transformer architecture and pre-training learning objectives [[Slides (lm architectures)](https://cmu.box.com/s/8mpj7subu7nugd9xiqnrcn3y2zghzdqp%20%0A%20%20%20%20 "Download Slides \(lm architectures\)")] Readings: * [What Language Model Architecture and Pretraining Objective Work Best](https://proceedings.mlr.press/v162/wang22u/wang22u.pdf) * Due 08/28/2025 14:00 Thursday Assignment #0 due * Assignment 08/31/2025 Sunday Preparing and Exploring Pre-Training Data released! [[Preparing and Exploring Pre-Training Data](https://2025.cmu-llms.org/assignments/01)] * Lecture 09/02/2025 Tuesday Pre-training data curation and tokenization [[Slides (pre-training data)](https://cmu.box.com/s/5e88ssszlrcx9oys59vq9mk8lk9pij7k%20%0A%20%20%20%20 "Download Slides \(pre-training data\)")] Readings: * [Documenting C4](https://arxiv.org/pdf/2104.08758) * [Decline of AI Data Commons](https://arxiv.org/pdf/2407.14933) * [ClueWeb22](https://arxiv.org/pdf/2211.15848) * Lecture 09/04/2025 Thursday Architecture Advancement on Transformers [[Slides (transformer advancements)](https://cmu.box.com/s/qw8wqfubo076asaepegrnsst6xfzpf8f%20%0A%20%20%20%20 "Download Slides \(transformer advancements\)")] Readings: * [Activation Functions](https://arxiv.org/abs/1710.05941) * [Layer norms](https://proceedings.mlr.press/v119/xiong20b/xiong20b.pdf) * [Grouped-Query Attention](https://arxiv.org/pdf/2305.13245) * [RoPE embeddings](https://arxiv.org/pdf/2104.09864) * Lecture 09/09/2025 Tuesday Automatic evaluation of LLMs [[Slides (automatic evaluation)](https://cmu.box.com/s/1hcd7q8qpc0jh68m5s22ciyheb9vk2jo%20%0A%20%20%20%20 "Download Slides \(automatic evaluation\)")] Readings: * [GEM Benchmark](https://arxiv.org/pdf/2102.01672.pdf) * [BIG-Bench, chapters 1 and 2](https://arxiv.org/pdf/2206.04615) * [Holistic Evaluation of Language Models](https://arxiv.org/pdf/2211.09110) * [Evaluation Challenges](https://aclanthology.org/D17-1238.pdf) * Lecture 09/11/2025 Thursday In-context learning, task-oriented finetuning, and parameter-efficient tuning methods [[Slides (finetuning)](https://cmu.box.com/s/8x5xnusopr6gwatqq9kh3vbbkmnpnd9c%20%0A%20%20%20%20 "Download Slides \(finetuning\)")] Readings: * [Prompt Tuning](https://arxiv.org/abs/2104.08691) * [Blog post on param-efficient tuning](https://www.leewayhertz.com/parameter-efficient-fine-tuning/) * [Really detailed survey paper](https://arxiv.org/pdf/2203.06904) * Due 09/11/2025 23:59 Thursday Assignment #1 due * Assignment 09/14/2025 Sunday Implementing a Transformer from Scratch and Doing Inference with the HuggingFace API released! [[Implementing a Transformer from Scratch and Doing Inference with the HuggingFace API](https://2025.cmu-llms.org/assignments/02)] * Lecture 09/16/2025 Tuesday Should we expect a half of a translation model to reason? - guest lecture by Ondrej Bojar [[Slides (reasoning)](https://cmu.box.com/s/5zljqnpz4nwdgiwwq8lnyxresasep26u%20%0A%20%20%20%20 "Download Slides \(reasoning\)")] Readings: * [Wishful Thinking (pp. 55-62)](https://ufal.mff.cuni.cz/books/preview/2021-kocmi_full.pdf#page=63.16) * [ELOQUENT 2025 Sensemaking](https://arxiv.org/pdf/2507.12143) * [Sequence Length](https://aclanthology.org/2021.emnlp-main.650.pdf) * [Evaluation Malpractices](https://aclanthology.org/2024.eacl-long.5.pdf) * [GSM-Symbolic](https://openreview.net/pdf?id=AjXkRZIvjB) * Lecture 09/18/2025 Thursday Evaluation and Learning in Human-Agent Interaction - guest lecture by Hao Zhu [[Slides (agent-human interaction)](https://cmu.box.com/s/ojqkxg24aidyuvns7calezoj5x2qwhhf%20%0A%20%20%20%20 "Download Slides \(agent-human interaction\)")] Readings: * [AI as thought partners](https://www.nature.com/articles/s41562-024-01991-9) * [Challenges in Human-AI Communication](https://arxiv.org/abs/2412.10380) * [NAACL 2025 Social Intelligence in the Age of LLMs](https://docs.google.com/presentation/d/1s18RGtlXXTxAqdwzGdzTy1nsmkmdUd8VmHmTFIatlH0/edit) * Lecture 09/23/2025 Tuesday Surprising Behaviours of In-Context Learning and Model Alignment [[Slides (in-context learning)](https://cmu.box.com/s/pimjo5f9992d5zdcj728z4artu8lxmhr%20%0A%20%20%20%20 "Download Slides \(in-context learning\)")] Readings: * [Chain-of-thought prompting](https://arxiv.org/pdf/2201.11903) * [What makes in-context learning work?](https://aclanthology.org/2022.emnlp-main.759.pdf) * [Are aligned models adversarially aligned?](https://proceedings.neurips.cc/paper_files/paper/2023/file/c1f0b856a35986348ab3414177266f75-Paper-Conference.pdf) * Lecture 09/25/2025 Thursday Tool Use in LLMs [[Slides (tool use)](https://cmu.box.com/s/w05hj9qo07ii1xccwzzblyw6ow5hdt5j%20%0A%20%20%20%20 "Download Slides \(tool use\)")] Readings: * [What are tools anyway](https://arxiv.org/pdf/2403.15452) * [Toolformer](https://openreview.net/pdf?id=Yacmpz84TH) * [Dense Retrieval Training](https://arxiv.org/pdf/2007.00808) * Lecture 09/30/2025 Tuesday Retrieval-augmented generation - guest lecture by Akari Asai [[Slides (rag)](https://cmu.box.com/s/02vck4qnr4ikrk9aw13t8nlvs5hajepm%20%0A%20%20%20%20 "Download Slides \(rag\)")] Readings: * [RAG](https://arxiv.org/abs/2005.11401) * [Generalization through Memorization](https://arxiv.org/abs/1911.00172) * [When Not to Trust Language Models](https://arxiv.org/abs/2212.10511) * [Self-RAG](https://arxiv.org/abs/2310.11511) * Lecture 10/02/2025 Thursday Scaling Up LLMs - Scaling laws [[Slides (scaling laws)](https://cmu.box.com/s/63yap5np284nlj0f2wt1shh30ub89030%20%0A%20%20%20%20 "Download Slides \(scaling laws\)")] Readings: * [Scaling laws](https://arxiv.org/abs/2001.08361) * [Chinchilla](https://arxiv.org/abs/2203.15556) * Assignment 10/05/2025 Sunday Retrieval-augmented Generation and Tool-use released! [[Retrieval-augmented Generation and Tool-use](https://2025.cmu-llms.org/assignments/03)] * Due 10/05/2025 23:59 Sunday Assignment #2 due * Lecture 10/07/2025 Tuesday Scaling up LLMs - Optimization and Parallel Training [[Slides (scaling)](https://cmu.box.com/s/rh6tw3zvun2fupf7ewyzv5utklon27nv%20%0A%20%20%20%20 "Download Slides \(scaling\)")] Readings: * [GPipe](https://arxiv.org/abs/1811.06965) * [DeepSpeed](https://arxiv.org/abs/1910.02054) * [Adam](https://arxiv.org/abs/1412.6980) * Lecture 10/09/2025 Thursday Interpretability methods [[Slides (interpretability)](https://cmu.box.com/s/3ct87zi2ph50sr2zlh5u1juiyrija064%20%0A%20%20%20%20 "Download Slides \(interpretability\)")] Readings: * [Edge Probing](https://arxiv.org/abs/1905.06316) * [Loss Landscape](https://arxiv.org/abs/1712.09913) * Lecture 10/14/2025 Tuesday Fall break - no class * Lecture 10/16/2025 Thursday Fall break - no class * Exam 10/21/2025 14:00 Tuesday Midterm * Lecture 10/23/2025 Thursday Bias and ethical issues [[Slides (ethics)](https://cmu.box.com/s/35rnype19f1nuqegypz6819mqdh5ovtg%20%0A%20%20%20%20 "Download Slides \(ethics\)")] Readings: * [Redteaming LLMs](https://arxiv.org/abs/2209.07858) * Due 10/23/2025 23:59 Thursday Assignment #3 due * Assignment 10/26/2025 Sunday Improving Training Efficiency released! [[Improving Training Efficiency](https://2025.cmu-llms.org/assignments/04)] * Lecture 10/28/2025 Tuesday Attacking LLM systems + benchmark leakage [[Slides (attacking llms)](https://cmu.box.com/s/cgw05cpgaswhzfb17gpdv8tgux6jchyo%20%0A%20%20%20%20 "Download Slides \(attacking llms\)")] Readings: * [How does LLM safety training fail](https://openreview.net/pdf?id=jA235JGM09) * [Extracting training data from production models](https://arxiv.org/abs/2311.17035) * Lecture 10/30/2025 Thursday LLMs for Recommendation Systems - Guest lecture by Neil Shah [[Slides (recommenders)](https://cmu.box.com/s/gutuf1ohvg88bwx3mszyvaw3fa89tx0p%20%0A%20%20%20%20 "Download Slides \(recommenders\)")] Readings: * [Sequential Recommendation](https://arxiv.org/pdf/1808.09781) * [LLM2Vec](https://arxiv.org/pdf/2404.05961) * [Rethinking Search](https://arxiv.org/pdf/2105.02274) * [Pairwise Ranking Prompting](https://arxiv.org/pdf/2306.17563) * Lecture 11/04/2025 Tuesday Democracy day - no class * Lecture 11/06/2025 Thursday Reinforcement Learning from Human Feedback (RLHF) [[Slides (RLHF)](https://cmu.box.com/s/uza00oi1h4k2ybl0u6c9gz5iv127rk2v%20%0A%20%20%20%20 "Download Slides \(RLHF\)")] Readings: * [RLHF](https://arxiv.org/abs/2203.02155) * [DPO](https://arxiv.org/abs/2305.18290) * Due 11/06/2025 23:59 Thursday Assignment #4 due * Assignment 11/09/2025 Sunday Mini-project released! [[Mini-project](https://2025.cmu-llms.org/assignments/06)] * Lecture 11/11/2025 Tuesday Efficient Inference Methods [[Slides (Efficient Inference)](https://cmu.box.com/s/frt5wt4bp7103falhs265hd0uqunap3e%20%0A%20%20%20%20 "Download Slides \(Efficient Inference\)")] Readings: * [Speculative Decoding](https://arxiv.org/abs/2211.17192) * [FlashAttention](https://arxiv.org/abs/2205.14135) * [PagedAttention](https://arxiv.org/abs/2309.06180) * Lecture 11/13/2025 Thursday Chatbots and LLM Agents [[Slides (chatbots, agents, human evaluation)](https://cmu.box.com/s/lk3pnxmnba4xttjhj4yg8ah35gccofi1%20%0A%20%20%20%20 "Download Slides \(chatbots, agents, human evaluation\)")] – Readings: * [Chatbot Arena](https://arxiv.org/abs/2403.04132) * [Evaluating human evaluation](https://arxiv.org/abs/2107.00061) * Lecture 11/18/2025 Tuesday Long-context models [[Slides (2024 version)](https://storage.googleapis.com/cmu-llms/2024/2024-11-21-long-context-llms.pdf%20%0A%20%20%20%20 "Download Slides \(2024 version\)")] Readings: * [Lost in the middle](https://arxiv.org/abs/2307.03172) * [ProLong](https://arxiv.org/abs/2410.02660) * Lecture 11/20/2025 Thursday Efficient Pretraining with Sparse Models [[Slides (efficient pre-training)](https://cmu.box.com/s/6azzspcejv7scy49vi0ncbly60dtpbo8%20%0A%20%20%20%20 "Download Slides \(efficient pre-training\)")] Suggested Readings: * [Review of MoE](https://arxiv.org/abs/2209.01667) * [Switch Transformer](https://arxiv.org/abs/2101.03961) * [MatFormer](https://arxiv.org/pdf/2310.07707) * Lecture 11/25/2025 Tuesday Training with Synthetic Data [[Slides (synthetic data)](https://cmu.box.com/s/304w2ftvklnr7q57sqmvhjx9k2rl1vcg%20%0A%20%20%20%20 "Download Slides \(synthetic data\)")] Readings: * [MATES](https://arxiv.org/abs/2406.06046) * [Montessori-Instruct](https://arxiv.org/abs/2410.14208) * Lecture 11/27/2025 Thursday Thanksgiving - no class * Lecture 12/02/2025 Tuesday Mini-Project Presentation [[Schedule](https://docs.google.com/spreadsheets/d/1wkqu1dc8GwtUFiWYVLaDDPRzILbEwz-kAY5mt1-oj2M/edit?usp=sharing%20%0A%20%20%20%20 "Download Schedule")] [[Upload dir](https://drive.google.com/drive/folders/1XrCq-kA5hZqg3oyED0bg840Phgp2Y2n2?usp=sharing%20%0A%20%20%20%20 "Download Upload dir")] * Lecture 12/04/2025 Thursday Mini-Project Presentation [[Schedule](https://docs.google.com/spreadsheets/d/1wkqu1dc8GwtUFiWYVLaDDPRzILbEwz-kAY5mt1-oj2M/edit?usp=sharing%20%0A%20%20%20%20 "Download Schedule")] [[Upload dir](https://drive.google.com/drive/folders/1BTjLcbmNJLcporBMzCwhLL848Qsx27qh?usp=sharing%20%0A%20%20%20%20 "Download Upload dir")] * Due 12/07/2025 23:59 Sunday Mini-project due * Exam 12/09/2025 13:00 Tuesday Final Exam Language Technologies Institute School of Computer Science Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213 * [ lti.cs.cmu.edu/ ](https://www.lti.cs.cmu.edu/)