# CSE455: Computer Vision https://courses.cs.washington.edu/courses/cse455/20sp/ ![](https://courses.cs.washington.edu/courses/cse455/20sp/images/visiontitle.png) ### Course Information: **Course number** : CSE 455 **Quarter** : Spring 2020 **Instructor** : [Linda Shapiro](https://homes.cs.washington.edu/~shapiro/) (shapiro@cs.washington.edu, CSE 634) **Office hours** (for concepts, not code): Wednesdays 11:00 a.m. -12:00 p.m. Zoom link below. **Class time** : Tuesdays, Thursdays 10:00-11:20 a.m. **Class location** : [Zoom Link](https://washington.zoom.us/s/553420896) (password [here](https://canvas.uw.edu/courses/1371994/)). Lectures will be recorded and available on [canvas](https://canvas.uw.edu/courses/1371994/external_tools/21130). **Contact** : Cse455-staff@cs.washington.edu **Discussion and Questions** : [Canvas](https://canvas.uw.edu/courses/1371994) **Grading policy** : Homeworks (80%) and Final Project (20%). There will be no written exams. **TAs** : * Beibin Li (beibin@cs.washington.edu) Office Hours: Tuesdays 3:00-4:00 p.m. * Bindita Chaudhuri (bindita@cs.washington.edu) Office Hours: Thursdays 2:00-3:00 p.m. * Shima Nofallah (shima@cs.washington.edu) Office Hours: Mondays 2:00-3:00 p.m. Zoom links for ALL office hours can be found [here](https://canvas.uw.edu/courses/1371994/discussion_topics/5315882). Slides are a mishmash of lots of other people's work. Special thanks to: Rob Fergus, Joseph Redmon, Harvey Rhody, Rick Szeliski, Ali Farhadi, Robert Collins, Ricardo Martin Brualla. ### **Homeworks:** We will have 7 homework assignments (including 1 optional), which will be listed below as they are assigned. The assignments will be given out every week starting week 1 for week 2's assignment, so we can get going. Please let the TAs know if you have any issues with the homeworks. Note that there is a deadline for each homework. Anything uploaded after the deadline will be marked late. Please be careful to not overwrite an in-time assignment with a late assignment when uploading near the deadline. Each student has 4 penalty-free late days for the whole quarter; other than that any late submission will be penalized (10% of total points will be deducted from the obtained points) for each day it is late. #### Assignments: * [Homework 0: Fun with color!](https://github.com/UW-CSE-455-2020SP/homeworks), due April 14 at 11:59 p.m. * [Homework 1: Image resizing](https://github.com/UW-CSE-455-2020SP/homeworks/tree/master/src/hw1), due April 21 at 11:59 p.m. * [Homework 2: Image filtering](https://github.com/UW-CSE-455-2020SP/homeworks/tree/master/src/hw2), due April 28 at 11:59 p.m. * [Homework 3: Panoramas](https://github.com/UW-CSE-455-2020SP/homeworks/tree/master/src/hw3), due May 12 at 11:59 p.m. * [Homework 4: Optical Flow](https://github.com/UW-CSE-455-2020SP/homeworks/tree/master/src/hw4), due May 21 at 11:59 p.m. * [Homework 5: Neural Networks](https://github.com/UW-CSE-455-2020SP/homeworks/tree/master/src/hw5), due May 28 at 11:59 p.m. * Optional: [Homework 6: PyTorch](https://github.com/UW-CSE-455-2020SP/homeworks/tree/master/src/hw6), due June 4 at 11:59 p.m. Turn in all homeworks on [canvas](https://canvas.uw.edu/courses/1371994/assignments). ### **Final Project:** There will be a final project worth 20% of your final grade. The project should be done in **teams of about 4 students** , except in unusual circumstances. The TAs will help you form teams (who will collaborate among themselves virtually) and help with your progress during the office hours. Canvas groups will be created for your teams, where you can submit your documents. Don't worry about multiple submissions, canvas will automatically show one submission (the latest one) per team. The project timeline is given below. **Projects must involve machine learning** , but otherwise are up to you to define. Each project should have a significant technical component, software implementation, or large-scale study. Projects can focus on developing new techniques or tools in computer vision or applying existing tools to a new domain. You can take a look [here](https://courses.cs.washington.edu/courses/cse455/20sp/projectpage.html) for some guidance. #### Specifications: * Use this [discussion post](https://canvas.uw.edu/courses/1371994/discussion_topics/5357250) to discuss about project teams, topics etc. * You can use [Google Colab](https://colab.research.google.com/) for computing resources, they generally have GPUs available all the time. * Submit your files on [canvas](https://canvas.uw.edu/courses/1371994/assignments). * All your written documents (except presentation) should be in [NIPS format](https://nips.cc/Conferences/2018/PaperInformation/StyleFiles). * You can use [Overleaf](https://www.overleaf.com/) to collaborate on your written documents. * Here are some [tutorials](https://www.overleaf.com/learn/latex/Tutorials) on Overleaf/Latex. * CSE students get free access to Overleaf Pro if you sign up with your cs.washington.edu email. #### Project timeline: * **Group formation** * Preliminary deadline: April 20 @ 11:59 pm * Update the spreadsheet (linked in the canvas discussion post) with the names of your current team members. * If you have decided your project topic, update that as well so that other students may be interested to join your team. * Final deadline: April 24 @ 11:59 pm * If you are in a team of 4 students, you are good to go. Discuss the next steps among yourselves. * If you are in a team of less than 4 students, you can also approach the students without any team to get them to join you. * If you are not in a team, you can approach the students who have already formed teams to join them. * We will probably assign the remaining students randomly to existing teams after the deadline. * **Project proposal** (_1 point_) * Deadline: May 1 @ 11:59 pm * Submit a 1 page PDF named _proposal.pdf_ specifying your team member names, brief description and goal of the project, and milestones. * We will give you feedback on canvas as comments. * **Project status report** (_4 points_) * Deadline: May 15 @ 11:59 pm * Submit a 2-3 page PDF named _status_report.pdf_ containing your project description, what you have done till date, and future work. * **In-class project presentation** (_15 points_) * Time: June 8, 10:30 am - 12:20 pm (most likely) * Zoom link: available on [Canvas](https://canvas.uw.edu/courses/1371994) * Duration: 5 minutes per team. No limit on number of slides. * Introduce the project, show the methodology, then your results, and finally some conclusion. **Extra credit (5 points): A live demo of some sort**. * Submit your slides in the form of a PDF (don't worry about animations if any). Name the PDF _presentation.pdf_. * Submission deadline: June 8 @ 1 pm * **Project final report** (_30 points_) * Deadline: June 9 @ 12 noon * Submit a 5-10 page PDF named _final_report.pdf_ using [this](https://media.nips.cc/Conferences/NIPS2018/Styles/nips_2018.pdf) format, containing * Title (name of the project) and team members * Introduction * Related work, with references to papers, webpages etc. * Methodology, with an algorithm or a diagram * Experimental results (preferably both quantitative and qualitative) * **Screenshots of your code/demo** * Discussion of results - strengths/weaknesses, what worked/what didn't * Conclusion and future work (what you could have done given more time) * **No late days allowed!** ### **Topics** * Introduction * Color and Texture * Transforms and Resizing * Filters and Convolutions * Edges and Lines * Interest Operators, Image Matching and Stitching * Optical Flow * 3D * Stereo * ML overview including Neural Nets * Object Detection and Recognition with ML * Convolutional Neural Networks * CNN Applications ### **Lecture Notes:** | **Date** | **Lecture Notes in PPT** | **Lecture Notes in PDF** | **Readings** | | --- | --- | --- | --- | | March 31 | [Course Details](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/0_CourseDetails_20.pptx) | [Course Details](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/0_CourseDetails_20.pdf) | None | | April 2 | [Introduction](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/1_Intro20.pptx) | [Introduction](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/1_Intro20.pdf) | None | | April 7 | [Color and Texture](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/2_Color_Texture_FIN_20.pptx); [ink](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/2_Color_Texture_FIN_20ink.pptx) | [Color and Texture](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/2_Color_Texture_FIN_20.pdf) | Shapiro & Stockman Ch 6 and 7 | | April 9 | [Resizing](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/3_Resizing_20.pptx); [ink](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/3_Resizing_20ink.pptx) | [Resizing](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/3_Resizing_20.pdf) | None | | April 14 | [Filtering](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/4_Filters_20.pptx); [ink](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/4_Filters_20ink.pptx) | [Filtering](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/4_Filters_20.pdf) | Shapiro & Stockman 5.4-5.5, Szeliski 3.2, 3.3.1, [Morphological operations](https://courses.cs.washington.edu/courses/cse576/19sp/notes/Basics1_white.pdf) | | April 16 | [Edges and Lines](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/5_Edges_20.pptx); [ink](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/5_Edges_20ink.pptx) | [Edges and Lines](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/5_Edges_20.pdf) | Shapiro & Stockman 5.6-8 and 10.3.4, Szeliski 4.2.1 | | April 20 | [Interest Operators](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/6_Harris_20.pptx); nk](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/6_Harris_20ink.pptx) | [Interest Operators](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/6_Harris_20.pdf) | Szeliski 4.1.1, [Harris Paper](https://courses.cs.washington.edu/courses/cse576/06sp/papers/harris.pdf), [SIFT paper](https://courses.cs.washington.edu/courses/cse576/06sp/papers/SIFT.pdf) | | April 22 | [Describe and Match](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/7_DescribeMatch_20.pptx); [ink](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/7_DescribMatch_20ink.pptx) | [Describe and Match](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/7_DescribeMatch_20.pdf) | Szeliski 4.1.2-4.1.3, [SIFT paper](https://courses.cs.washington.edu/courses/cse576/06sp/papers/SIFT.pdf) | | April 28 | [Matching](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/8_Match_20.pptx); [ink](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/8_Match_20ink.pptx) | [Matching](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/8_Match_20.pdf) | Szeliski 6.1 | | April 30 | [Regions, Recognition, and Matching](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/9_RegMatchRec_20.pptx); [ink](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/9_RegMatchRec_20ink.pptx) | [Regions, Recognition, and Matching](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/9_RegMatchRec_20.pdf) | [Kadir paper](https://homes.cs.washington.edu/~shapiro/EE596/notes/kadir.pdf); [Fergus paper](https://homes.cs.washington.edu/~shapiro/EE596/notes/fergus.pdf); S&S Ch 8 | | May 5 | [CBIR and EM](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/10_CBIR_EM_20.pptx); [ink](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/10_CBIR_EM_20ink.pptx) | [CBIR and EM](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/10_CBIR_EM_20.pdf) | S&S Ch 8; [ Yi's EM Classifier Paper](https://homes.cs.washington.edu/~shapiro/icprem2004.pdf) | | May 7 | [Learning 1](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/11_Learning1_20.pptx); [ink](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/11_Learning1_20ink.pptx) | [Learning 1](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/11_Learning1_20.pdf) | Related to S&S Ch 4 | | May 12 | [Motion](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/12_Motion_20.pptx); [ink](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/12_Motion_20ink.pptx) | [Motion](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/12_Motion_20.pdf) | Szeliski 8.4-8.5, S&S 9.3.4-6, [Pyramid Paper](http://homes.cs.washington.edu/~shapiro/EE596/notes/bergen_eccv92.pdf); [Ming Ye's paper](http://courses.cs.washington.edu/courses/cse455/16wi/notes/mingpami.pdf); [Motion Layers Paper](http://homes.cs.washington.edu/~shapiro/EE596/notes/wangadelson.pdf) | | May 14 | Continue Learning 1 | | | | May 19 | [Learning 2](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/13_Learning2_20.pptx); [ink](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/13_Learning2_20ink.pptx) | [Learning 2](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/13_Learning2_20.pdf) | | | May 21 | [Object Recognition](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/14_ObjRecog_20.pptx); [ink](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/14_ObjRecog_20ink.pptx) | [Object Recognitin](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/14_ObjRecog_20.pdf) | [Girshick Paper](https://courses.cs.washington.edu/courses/cse455/16wi/notes/richfeatures.pdf); [Redmon's YOLO Paper](https://grail.cs.washington.edu/wp-content/uploads/2016/09/redmon2016yol.pdf) | | May 26 | [CNN Basics (Beibin)](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/CNNBasics.pptx); [GANs (Bindita)](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/GAN_cse455.pptx) | [CNN Basics (Beibin)](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/CNNBasics.pdf); [GANs (Bindita)](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/GAN_cse455.pdf) | | | May 28 | [Bindita's Facial Expression Research](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/Bindita_research.pptx); [Shima's Cancer Diagnoisis Research](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/CNN_Shima.pptx); [Beibin's ASD Research](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/affective_for_asd.pptx) | [Bindita's Facial Expression Research](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/Bindita_research.pdf); [Shima's Cancer Diagnoisis Research](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/CNN_Shima.pdf); [Beibin's ASD Research](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/affective_for_asd.pdf) | | | June 2 | [Stereo and 3D](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/15_Stereo3D_20.pptx); [ink](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/15_Stereo3D_20ink.pptx) | [Stereo and 3D](https://courses.cs.washington.edu/courses/cse455/20sp/notes/Linda/15_Stereo3D_20.pdf) | Shapiro and Stockman Chs 12-13 | | June 4 | NO CLASS | Work on your projects. | | | June 8 at 10:30am | PROJECT PRESENTATIONS | 5 minutes per group | | ### **Text Books:** * [Computer Vision](https://courses.cs.washington.edu/courses/cse576/99sp/book.html), Linda G. Shapiro and George Stockman, Prentice-Hall, 2001. * [Computer Vision: Algorithms and Applications](http://szeliski.org/Book/), Rick Szeliski, 2010. ### **Course Administration and Policies** * Collaboration is encouraged! Feel free to discuss howemork and class material with other students. However, make sure you understand the concepts. Each student will complete and submit their own work. Do not directly or indirectly copy other students' work. * If you are working together or helping another student, work on teaching them concepts and answering general questions, not directly telling them what code to write. You're all smart; you should understand the line between productive collaboration and giving someone answers. [Privacy policy](http://www.washington.edu/online/privacy "University of Washington Online Privacy Statement") and [Terms of use](http://www.washington.edu/online/terms "University of Washington Website Terms and Conditions of Use")