# Advances in Computer Vision – Scene Representation Group https://scenerepresentations.org/courses/2025/spring/advances-in-cv/#syllabus [Skip to content](https://scenerepresentations.org/courses/2025/spring/advances-in-cv/#main-content) [ Scene Representation Group ](https://scenerepresentations.org/) [Publications](https://scenerepresentations.org/publications) [Talks](https://scenerepresentations.org/talks) [Teaching](https://scenerepresentations.org/courses) [People](https://scenerepresentations.org/people) [ Massachusetts Institute of Technology ](https://web.mit.edu "The Massachusetts Institute of Technology") [ ](https://www.csail.mit.edu/ "MIT Computer Science & Artificial Intelligence Lab") 1. [Teaching](https://scenerepresentations.org/courses) 2. Advances in Computer Vision 6.8300 spring 2025 Advances in Computer Vision - MIT 6.8300 * [Details](https://scenerepresentations.org/courses/2025/spring/advances-in-cv/#details) * [Syllabus](https://scenerepresentations.org/courses/2025/spring/advances-in-cv/#syllabus) * [Related Courses](https://scenerepresentations.org/courses/2025/spring/advances-in-cv/#related-courses) * [Gradescope](https://www.gradescope.com/courses/972401) * [Slides](https://drive.google.com/drive/folders/1_t7-165L6jqbc2MANXRj7brrWXrx-SfM?usp=sharing) * [Piazza](https://piazza.com/class/m5fhhrt0jzc6hx) * [Canvas](https://canvas.mit.edu/courses/31251) **Note:** This is the **Spring 2025** version of this course. For the current offering, please visit the [Spring 2026 course page](https://scenerepresentations.org/courses/2026/spring/advances-in-cv/). ### Course Contents This course dives into advanced concepts in computer vision. A first focus is geometry in computer vision, including image formation, represnetation theory for vision, classic multi-view geometry, multi-view geometry in the age of deep learning, differentiable rendering, neural scene representations, correspondence estimation, optical flow computation, and point tracking. Next, we explore generative modeling and representation learning including image and video generation, guidance in diffusion models, conditional probabilistic models, as well as representation learning in the form of contrastive and masking-based methods. Finally, we will explore the intersection of robotics and computer vision with "vision for embodied agents", investigating the role of vision for decision-making, planning and control. ### Prerequisites The formal prereqs of this course are: 6.7960 Deep Learning, (6.1200 or 6.3700), (18.06 or 18.C06). This class is an advanced graduate-level class. You have to have working knowledge of the following topics, i.e., be able to work with them in numpy / scipy / pytorch. There will be no explainer on this and TAs will not be able to help you with these basics. Deep Learning: Proficiency in Python, Numpy, and PyTorch, vectorized programming, and training deep neural networks. Convolutional neural networks, transformers, MLPs, backpropagation. Linear Algebra: Vector spaces, matrix-matrix products, matrix-vector products, change-of-basis, inner products and norms, Eigenvalues, Eigenvectors, Singular Value Decomposition, Fourier Transform, Convolution. ### Schedule 6.8300 will be held as 1.5 hour long lectures in room **26-100** : | **Tuesday** | 1:00 – 2:30 pm | | --- | --- | | **Thursday** | 1:00 – 2:30 pm | ### Collaboration Policy Problem sets should be written up individually and should reflect your own individual work. However, you may discuss with your peers, TAs, and instructors. You should not copy or share complete solutions or ask others if your answer is correct, whether in person or via Piazza or Canvas. If you work on the problem set with anyone other than TAs and instructors, list their names at the top of the problem set. ### Office Hours | **Monday** | 1:00 – 2:00 pm | Ben | 32-G575 | | --- | --- | --- | --- | | | 2:00 – 3:00 pm | Ariba | [Zoom](https://mit.zoom.us/j/95116191629) | | | 4:00 – 5:00 pm | Tianyuan | 45-205 | | **Tuesday** | 9:30 – 10:30 am | Ate | [Zoom](https://mit.zoom.us/j/94436710929) | | **Wednesday** | 9:00 – 10:00 am | Vincent | 45-741B | | | 1:30 – 2:30 pm | Vivek | 32-D451 | | | 2:30 – 3:30 pm | Jane | 32-D451 | | **Thursday** | 3:00 – 4:00 pm | Christian | 32-370 | | **Friday** | 1:00 – 2:00 pm | Chenyu | 32-262 | | | 3:00 – 4:00 pm | Isabella | 32-D451 | | | 4:00 – 5:00 pm | Adriano | [Zoom](https://mit.zoom.us/j/95284337171) | ### AI Assistants Policy Our policy for using ChatGPT and other AI assistants is identical to our policy for using human assistants. Just like you can come to office hours and ask a human questions (about the lecture material, clarifications about problem set questions, tips for getting started, etc), you are very welcome to do the same with AI assistants. But: just like you are not allowed to ask an expert friend to do your homework for you, you also should not ask an expert AI. If it is ever unclear, just imagine the AI as a human and apply the same norm as you would with a human. If you work with any AI on a problem set, briefly describe which AI and how you used it at the top of the problem set (a few sentences is enough). ### Grading Policy Grading will be split between four module-specific problem sets and a final project: | 65% | **Problem Sets** 5 problem sets note our separate policies on [Collaboration](https://scenerepresentations.org/courses/2025/spring/advances-in-cv/#collaboration-policy), [AI Assistants](https://scenerepresentations.org/courses/2025/spring/advances-in-cv/#ai-policy), and [Late Submissions](https://scenerepresentations.org/courses/2025/spring/advances-in-cv/#late-policy). | | --- | --- | | 35% | **Final Project** Proposal (10%) + Blog Post (90%) | The final project will be a **research project on perception** of your choice: * You will run experiments and do analysis to explore your research question. * You will write up your research in the format of a blog post. Your post will include an explanation of background material, new investigations, and results you found. * You are encouraged to include plots, animations, and interactive graphics to make your findings clear. [Here](https://distill.pub/) [are](https://www.engraved.blog/why-machine-learning-algorithms-are-hard-to-tune/) [some](http://karpathy.github.io/2015/05/21/rnn-effectiveness/) [examples](https://ai.facebook.com/blog/dino-paws-computer-vision-with-self-supervised-transformers-and-10x-more-efficient-training) of well-presented research. The final project will be graded for clarity and insight as well as novelty and depth of the experiments and analysis. Detailed guidance will be given later in the semester. ### Late Submissions Policy Homeworks will not be accepted more than 7 days after the deadline. The grade on a homework received n days after the deadline (n<=7) will be multiplied by (1-n/14). We will round up to units of full days; submitting 1 hour late counts as using 1 late day. Ten penalty days will be automatically waived for each student. For example, let's say a student perfectly solves the first three homeworks but submits the first homework 8 days late, the second homework six days late and the third homework five days late. Then the student will score zero on the first homework, 100% on the second homework (using up six late days) and 100% * (1-1/14) = 92.9% on the third homework (using up the last four remaining late days). The slack days are meant to be used for all the normal circumstances of life: being behind on work, forgetting the deadline, having a conference to attend, etc. We will not grant further extensions for these routine issues. For any extension request, such as serious medical issues or major life events, please contact [S3](https://studentlife.mit.edu/s3) (for undergrads) or [GradSupport](https://oge.mit.edu/student-support-development/gradsupport/) (for grad students) and we will work with them to find a good solution. We will not be able to support course incompletes. ### FAQ | Q | Can I take this course if I have _not_ taken 6.7960 Deep Learning or a comparable class? | | --- | --- | | A | I advise against it. There will be homework assignments where you will be asked to re-implement deep learning papers by yourself. If you don't have working knowledge in Deep Learning using Pytorch, you are unlikely to perform well on these assignments. We will generally not discuss topics that were discussed in the Deep Learning class, i.e., we will not be reiterating Transformers, CNNs, how to train these models, etc, but will assume that you are already familiar with them. | | Q | Is this class a CI-M class? | | A | No, this is a graduate class. | | Q | Is 6.8301 (the undergraduate version) taught this semester? | | A | The undergraduate version is taught this semester as well. For logistical reasons, it had to be renamed to [6.S058, and is taught by Profs. Bill Freeman and Phillip Isola](https://introtocv.github.io/). 6.S058 is a CI-M class, and does _not_ have a prerequisite on 6.7960 Deep Learning. | | Q | Is attendance required? Will lectures be recorded? | | A | Attendance is at your discretion. Yes, lectures will be recorded and uploaded. | **Note:** This is the **Spring 2025** version of this course. For the current offering, please visit the [Spring 2026 course page](https://scenerepresentations.org/courses/2026/spring/advances-in-cv/). ## Syllabus | ### Module 0: Introduction to Computer Vision | | --- | | #### Introduction to Vision Tue, Feb. 4th | * Administrativa & Logistics * Historical perspective on vision: problems identified so far * What is vision? * Outlook | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=411d3828-5e30-41ab-b70b-b2780084a1ed&start=270) * [Slides ](https://drive.google.com/file/d/15ktSBIBs9tjpjzkwqt7mC0e1zBRFGhkY/view?usp=drive_link) | | ### Module 1: Module 1: Geometry, 3D and 4D | | #### What is an Image: Pinhole Cameras & Projective Geometry Thu, Feb. 6th | * Image as a 2D signal * Image as measurements of a 3D light field * Pinhole camera and perspective projection * Camera motion and poses | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=e124d25b-54e9-4236-ae05-b2780084a983) * [Slides ](https://drive.google.com/file/d/16Cp18-VFONBCAoWbcAqO8LNOK46fPVht/view?usp=drive_link) * [ pset 1 out ](https://github.com/6-8300/pset-1) | | #### Linear Image Processing & Transformations Tue, Feb. 11th | * Images as functions: continuous vs discrete * Function spaces and Fourier transform overview * Image filtering: gradients, Laplacians, convolutions * Multi-scale processing: Laplacian and multi-scale pyramids | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=1f212038-b129-492e-aac0-b27f0084dcca) * [Slides ](https://drive.google.com/file/d/17y9FxhfEPNb6T8iCTTapTDTKJETjz_hE/view?usp=drive_link) | | #### Representation Theory in Vision Thu, Feb. 13th | * Groups * Group Representations * Steerable Bases * Invariant Operators * Finding Steerable Bases via the Eigendecomposition of Invariant Operators | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=098e739d-dfed-4a10-93c5-b27f0084e42f) * [Slides ](https://drive.google.com/file/d/187OiFk9RNXUUSTr1EtigSTVaAraCoGOw/view?usp=drive_link) | | #### No Class (Monday Schedule) Tue, Feb. 18th holiday | | * pset 1 due * [ pset 2 out ](https://github.com/6-8300/pset-2/) | | #### Geometric Deep Learning and Vision Thu, Feb. 20th | * Equivariance and invariance * Regular Group Convolutions * Steerable Group Convolutions * Challenges of applying geometric techniques to vision tasks | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=5fdbd9ac-5a85-4c96-b34a-b2860084959c) * [Slides ](https://drive.google.com/file/d/18DcTEWqxzIixsVEsWHaqVp1wOTFgcF-C/view?usp=drive_link) | | #### Optical Flow Tue, Feb. 25th | * What is optical flow? * Color Constancy Assumption * Infinitesimal Optical Flow * Multi-Scale Cost and Correlation Volumes * Learning-based optical flow * RAFT | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=9a9de305-4273-4d41-a4db-b28d0084a74b&start=378) * [Slides ](https://drive.google.com/file/d/18IliY17B-9VRiIbk7odvPnoq4bIHddhd/view?usp=drive_link) | | #### Point Tracking, Scene Flow and Feature Matching Thu, Feb. 27th | * Point Tracking * Scene Flow * Connection of Scene Flow and Pixel Motion, FlowMap * Sparse Correspondence and Invariant Descriptors * SIFT | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=a6295992-0b71-4021-b1c6-b28d0084990c&start=340) * [Slides ](https://drive.google.com/file/d/18Mixij7hVWhMmt3RBfGwvlvzLLCB2soU/view?usp=drive_link) * pset 2 due * [ pset 3 out ](https://github.com/6-8300/pset-3) | | #### Multi-View Geometry Tue, March 4th | * Triangulation in Light Fields: Infinetismal perspective * Finite Triangulation * Epipolar Geometry * Eight-point algorithm and bundle adjustment * Learning-Based Approaches: Dust3r & Mast3r | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=017a9bc8-fbcf-4f79-8cf2-b29400849dc9&start=326.350904) * [Slides ](https://drive.google.com/file/d/18NEvg7T75CridpW0GIHK-tS-TiBQvGSd/view?usp=drive_link) | | #### Differentiable Rendering: Data Structures and Signal Parameterizations Thu, March 6th | * Surface-Based Representations * (Volumetric) Field Representations * Grid-based and adaptive data structures * Neural Fields * Hybrid Neural / discrete fields | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=eb46f1c9-5301-49b4-85a1-b2940084b29d&start=0) * [Slides ](https://drive.google.com/file/d/18VBGC8wzggKv7K8Jov2XdFFMC6dL53M3/view?usp=drive_link) | | #### Guest Lecture by Eric Brachmann: Deep Learning for 3D Reconstruction Tue, March 11th guest lecture | * Eric has done some amazing work on a new generation of structure-from-motion algorithms that leverage deep learning in new and inspiring ways. Here, he's speaking about some of his most recent work. | * [Recording ](https://drive.google.com/file/d/1yI7g2TCk1g4l4ZnuRvv2IpJR99nAsQ8Z/view?usp=drive_link) * pset 3 due * [ pset 4 out ](https://github.com/6-8300/pset-4) | | #### Differentiable Rendering: Novel View Synthesis Thu, March 13th | * Sphere tracing and volume rendering * Differentiable rendering techniques | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=4711cc6e-b88c-4102-abd6-b29b00848f27&start=294.752913) * [Slides ](https://drive.google.com/file/d/18gA0dwzT8iXCFmvvL2bis5RDb7B5hFQi/view?usp=drive_link) | | #### Differentiable Rendering: Novel View Synthesis 2 Tue, March 18th | * Gaussian splatting * Advanced differentiable rendering methods | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=b531755f-e6ea-4e12-a13b-b2a20084a8ec&start=340) * [Slides ](https://drive.google.com/file/d/18q675zGa69octxqpb7IpLPxYyz4x6UJ5/view?usp=sharing) * [ Final Project Guidelines Released ](https://drive.google.com/file/d/18ilI716Sxeo481YriD2eHn3mutFcHtWq/view?usp=drive_link) | | #### Differentiable Rendering: Prior-Based 3D Reconstruction and Novel View Synthesis Thu, March 20th | * Global inference techniques * Light field inference and generative models | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=84df421f-6123-4c3f-9da3-b2a20084bce9&start=82.76796460253344) * [Slides ](https://drive.google.com/file/d/18rFFWRpPYTwqyaJ1M9E-BN92GvF-c8a7/view?usp=drive_link) * pset 4 due | | #### Student Holiday: Spring Break Tue, March 25th holiday | | | | #### Student Holiday: Spring Break Thu, March 27th holiday | | | | ### Module 2: Module 2: Unsupervised Representation Learning and Generative Modeling | | #### Introduction to Representation Learning and Generative Modeling Tue, April 1st | * What makes a good representation? How do we know that we found one? * Generative modeling: density estimation, uncertainty modeling * Representation learning: task-relevant encoding * Surrogate tasks: compression, denoising, imputation | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=14ea382e-9e8b-47dc-a2ef-b2b00084961c&start=284.62439283025714) * [Slides ](https://drive.google.com/file/d/18rb6OYA4RS1FnzdZoPrk2L2oPCl4wFz7/view?usp=sharing) * [ pset 5 out ](https://github.com/6-8300/pset-5) | | #### Peter Holderrieth: Diffusion Models 1 Thu, April 3rd | * Mathematical Foundations of Diffusion Models * ODE and SDE perspective of Diffusion * Score Matching and Flow | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=2465a76f-e509-4972-943d-b2b00084b527&start=304.8138225481773) * [Slides ](https://drive.google.com/file/d/18tuLxAOPB1bduOOrPu9ZR9mglowE7lzW/view?usp=drive_link) | | #### Peter Holderrieth: Diffusion Models 2 Tue, April 8th | * Classifier-Free Guidance * Case study: SOTA models in image and video generation * SOTA architectures | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=23dfd4ec-f68c-495c-b1b0-b2b70084a3bf&start=170) * [Slides ](https://drive.google.com/file/d/18xiCo_E7-9FMG_Q6_tRrekyc5f0F-w08/view?usp=drive_link) | | #### Lecture Canceled Thu, April 10th | | | | #### Diffusion Models 3 Tue, April 15th | * A spectral perspective on image and video diffusion * Why do Diffusion Models generalize? | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=6afc91ce-c35a-40ac-befb-b2be0084b58f) * [Slides ](https://drive.google.com/file/d/19BIbJQ3zOi5arKzsvr9sElr4VQ-8i4gk/view?usp=drive_link) * pset 5 due * Project proposal due | | #### Sequence Generative Models Thu, April 17th | * Auto-regressive and full-sequence models * Compounding errors and stability * Diffusion Forcing * History Guidance | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=a13d924f-92fb-4989-8ec4-b2be0084b8c7&start=0) * [Slides ](https://drive.google.com/file/d/19NI8hUGI0Ml9G0GDPTWxvgMyRe138KmV/view?usp=sharing) | | #### Sequence Generative Models II Tue, April 22nd | * Another perspective on Sequence generation * History Guidance | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=7ef5f1ed-0515-4f63-b485-b2c50084aa54&start=0) * [Slides ](https://drive.google.com/file/d/19NI8hUGI0Ml9G0GDPTWxvgMyRe138KmV/view?usp=drive_link) | | #### Non-Generative Representation Learning Thu, April 24th | * Alternative representation learning techniques * Applications in computer vision | | | #### Guest Lecture by Mathilde Caron: Self-Supervised Learning for Vision Tue, April 29th guest lecture | * Mathilde is the author of many of the most influential self-supervised learning methods. This lecture was an amazing look into how she came up with them and what inspired her. | * [Recording ](https://drive.google.com/file/d/1kP7vwg4eQ07JZRLT6ZtMXJHYatEAiC0T/view?usp=sharing) | | ### Module 3: Module 3: Vision for Embodied Agents | | #### Introduction to Robotic Perception Thu, May 1st | * Definition and challenges of embodied agents * Intersection with vision * Controlling Robots from Vision | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=e2676fe4-08d3-473d-85d3-b2cc0084b813) * [Slides ](https://drive.google.com/file/d/1U4IBaR-g22QNZU-Fpgl_mfnX5MH43opw/view?usp=sharing) | | #### Lecture Canceled Tue, May 6th | | | | #### Learning Skills from Demonstrations Thu, May 8th | * Behavior Cloning and Imitation Learning from Vision | * [Recording ](https://mit.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=5b2ea5f5-040b-4a86-87f6-b2d30084efc9) * [Slides ](https://drive.google.com/file/d/1v4otlDEAn4eu__XZvci9ZggLKKRjcBYQ/view?usp=sharing) | | #### Lecture Canceled Tue, May 13th | | * Final project due | ### Related Courses and Credits * [ **Computer Vision**](https://www.cs.ox.ac.uk/teaching/courses/2024-2025/vision/) Oxford University, Prof. Christian Rupprecht * [ **FFTs in Graphics and Vision**](https://www.cs.jhu.edu/~misha/Spring23/) Johns Hopkins University, Prof. Misha Kazhdan * [ **Learning for 3D Vision**](https://learning3d.github.io/) CMU 16-889, Prof. Shubham Tulsiani * [ **Advances in Computer Vision**](http://6.869.csail.mit.edu/sp22/) MIT 6.819/6.869, Profs. Bill Freeman, Phillip Isola, Antonio Torralba * [ **Deep Learning II, Part "Geometric Deep Learning"**](https://uvadl2c.github.io/) University of Amsterdam 52042DEL6Y, Prof. Erik Bekkers * [ **Computer Graphics in the Era of AI**](http://cs348i.stanford.edu/) Stanford CS348I, Profs. C. Karen Liu and Jiajun Wu * [ **Computer Vision**](https://uni-tuebingen.de/fakultaeten/mathematisch-naturwissenschaftliche-fakultaet/fachbereiche/informatik/lehrstuehle/autonomous-vision/lectures/computer-vision/) University of Tübingen ML-4360, Prof. Andreas Geiger ### Image Attribution The header background image is by Los Angeles–based designer and photographer [Anastasiya Badun](https://bio.site/_badun), shared [unsplash](https://unsplash.com/photos/a-close-up-of-an-eye-with-a-black-background-2wDxCCw83HM). We chose the image because of its thematic link to vision, and its overall abstract aesthetic. © 2021 – 2026 Scene Representation Group [About MIT](https://web.mit.edu/ "Learn more about MIT") [ About CSAIL ](https://www.csail.mit.edu/ "Learn more about CSAIL")