# CS 276: Information Retrieval and Web Search

https://web.stanford.edu/class/cs276/#course_schedule

[ ![](https://web.stanford.edu/class/cs276/images/nlp-logo.gif) ](http://nlp.stanford.edu/)
# CS 276 / LING 286: Information Retrieval and Web Search
![](https://web.stanford.edu/class/cs276/images/wordle.jpg)   
[Image credit](http://www.ics.uci.edu/~djp3/classes/2009_01_02_INF141/)
## Course Description
Information retrieval is the process through which a computer system can respond to a user's query for text-based information on a specific topic. IR was one of the first and remains one of the most important problems in the domain of natural language processing (NLP). Web search is the application of information retrieval techniques to the largest corpus of text anywhere — the web — and it is the context where many people interact with IR systems most frequently.   
  
In this course, we will cover basic and advanced techniques for building text-based information systems, including the following topics:   
  

  * Efficient text indexing
  * Boolean and vector-space retrieval models
  * Evaluation and interface issues
  * IR techniques for the web, including crawling, link-based algorithms, and metadata usage
  * Document clustering and classification
  * Traditional and machine learning-based ranking approaches


## Course Instructors
[ ![](https://web.stanford.edu/class/cs276/images/Christopher_Manning_027_1154x1154.jpg) Chris Manning ](http://nlp.stanford.edu/manning/)
[ ![](https://web.stanford.edu/class/cs276/images/pandu_nayak_cropped.jpeg) Pandu Nayak ](http://pandunayak.com)
## Teaching Assistants
[ ![](https://web.stanford.edu/class/cs276/images/DP-downscaled.jpg) Ashwin Paranjape (head TA) ](http://stanford.edu/~ashwinpp/)
[ ![](https://web.stanford.edu/class/cs276/images/chris_chute_photo.jpg) Chris Chute ](http://chrischute.com)
[ ![](https://web.stanford.edu/class/cs276/images/fei-jia.jpg) Fei Jia ](https://www.linkedin.com/in/fei-jia/)
[ ![](https://web.stanford.edu/class/cs276/images/2241473_th1.jpg) Rohan Sampath ](https://www.linkedin.com/in/rohan-sampath-87463744/)
[ ![](https://web.stanford.edu/class/cs276/images/NeethuRenjith.jpg) Neethu Renjith ](https://www.linkedin.com/in/neethu-renjith-8125b5103/)
## Class time & location
Spring quarter 2019  
Lecture times: Tues/Thurs, 4:30–5:50pm, April 1 to June 5  
Location: Gates B1 (Basement) 
## Grading & course policies
See the [course policies](https://web.stanford.edu/class/cs276/course_policies.html) page for details on grading, late days, and other policies. 
## Required textbook
Introduction to Information Retrieval, by C. Manning, P. Raghavan, and H. Schütze (Cambridge University Press, 2008).  
  
This book is available from [Amazon](https://www.amazon.com/Introduction-Information-Retrieval-Christopher-Manning/dp/0521865719), the Stanford bookstore, or your favorite book purveyor. You can also download and print chapters for free at the [book website](http://nlp.stanford.edu/IR-book). (We’d appreciate any reports of typos or of higher-level problems for the third printing.)  
  
This book will be referred to as IIR in the reading assignments listed in the [course schedule](https://web.stanford.edu/class/cs276/#course_schedule) section. 
## Other useful references
  * (MG) Managing Gigabytes, by I. Witten, A. Moffat, and T. Bell.
  * (IRAH) Information Retrieval: Algorithms and Heuristics, by D. Grossman and O. Frieder.
  * (MIR) Modern Information Retrieval, by R. Baeza-Yates and B. Ribeiro-Neto.
  * (FSNLP) Foundations of Statistical Natural Language Processing, by C. Manning and H. Schütze.
  * (SE) Search Engines: Information Retrieval in Practice, by B. Croft, D. Metzler, and T. Strohman.
  * (IRIE) Information Retrieval: Implementing and Evaluating Search Engines, by S. Büttcher, C. Clarke, and G. Cormack.


## Prerequisites
  * Core programming and algorithm skills  
CS 107, CS 161, and ideally other courses in the ["core"](http://cs.stanford.edu/degrees/undergrad/Requirements.shtml#Core) for CS majors provide good preparation. Note that we will be using [bitwise operations](https://en.wikipedia.org/wiki/Bitwise_operation) in several labs and assignments, so it's a good idea to brush up on these concepts and their syntax if you're rusty on low-level data manipulation.
  * Basic probability and statistics  
You should have a good grasp of the fundamentals of probability distributions and basic statistical calculations (mean, standard deviation, etc.) at the level of a course like CS 109.
  * Proficiency in Python  
All class assignments this year will be in Python.


## Programming Tutorials
  * Python for programmers  
While Python is wildly popular, this class was traditionally taught with programming assignments in Java. Here are a few [ Python Tutorials for programmers](https://wiki.python.org/moin/BeginnersGuide/Programmers).
  * Bit Manipulation in Python  
[Bit Manipulation in Python](https://wiki.python.org/moin/BitManipulation). Although you might not need any of it, it might come in handy to brush up your bit manipulation skills.
  * Jupyter notebook  
This years programming assignments make use of Jupyter Notebooks. If you are not familiar with them, here's a few pointers. 
    * [ Get started guide ](https://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/execute.html)
    * [ Official documentation ](https://jupyter-notebook.readthedocs.io/en/stable/)


[ OH calendar ](https://calendar.google.com/calendar/embed?src=on10c2p6iarm6udauphnk8nauk%40group.calendar.google.com&ctz=America%2FLos_Angeles) [ Lecture videos ](https://mvideox.stanford.edu/Course/1314) [ Canvas ](https://canvas.stanford.edu) [ Piazza forum ](https://piazza.com/stanford/spring2019/cs276)
Note:   
Some of the slides and video links are from previous offering of the course. We leave them here for your reference and they will be updated/replaced by each lecture. ***** marks the latest updated slides.   
The **complementary videos** are on Canvas, and the **slides** of the videos are linked below.   

## Course Schedule
  
  
| Week  | Date  | Event  | Description & materials  | Readings & other resources  |  
| --- | --- | --- | --- | --- |  
| Week 1  | Tues. 4/2  | Lecture (Pandu)  |  **Introduction to the course**  
  

  * Videos: ["Semistructured Data"](https://web.stanford.edu/class/cs276/complementary_video_slides/05-01-07-IR-SemistructuredData.pptx)
  * Slides: [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture1-course-intro.pptx) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture1-course-intro-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture1-course-intro-1per.pdf)

 | 
  * [IIR chapter 1](http://nlp.stanford.edu/IR-book/pdf/01bool.pdf)
  * MG section 3.2
  * MIR section 8.2
  * [Shakespeare plays](http://www.rhymezone.com/shakespeare/)

 |  
|   | Thurs. 4/4  | Lecture (Chris)  |  **Inverted Indices** : Dictionary and postings lists, boolean querying    
  

  * Videos: ["Document Encodings" ](https://web.stanford.edu/class/cs276/complementary_video_slides/IR-02-01-document_ingestions.pptx), ["Tokens"](https://web.stanford.edu/class/cs276/complementary_video_slides/IR-02-02-tokens.pptx), ["Terms"](https://web.stanford.edu/class/cs276/complementary_video_slides/IR-02-03-terms.pptx), ["Stemming"](https://web.stanford.edu/class/cs276/complementary_video_slides/IR-02-04-stemming.pptx), ["Skip Lists"](https://web.stanford.edu/class/cs276/complementary_video_slides/IR-02-05-skip_lists.pptx)
  * Slides: [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture2-intro-boolean.pptx) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture2-intro-boolean-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture2-intro-boolean-1per.pdf)

 | 
  * [IIR chapter 2](http://nlp.stanford.edu/IR-book/pdf/02voc.pdf)
  * MG sections 3.6, 4.3
  * MIR section 7.2
  * [Porter's stemmer (MIR)](http://www.sims.berkeley.edu/~hearst/irbook/porter.html)
  * [Porter stemming algorithm (Official)](http://www.tartarus.org/~martin/PorterStemmer/)
  * [A skip list cookbook (Pugh 1990)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.17.524)
  * [Fast phrase querying with combined indexes (Williams, Zobel, Bahle 2004)](http://portal.acm.org/citation.cfm?id=1028102)
  * [Efficient phrase querying with an auxiliary index (Bahle, Williams, Zobel 2002)](http://portal.acm.org/citation.cfm?id=564415)

 |  
| Week 2  | Tues. 4/9  | Lecture (Pandu)  |  **Index Construction**   
  

  * Videos: "Index Construction"
  * Slides: [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture3-indexconstruction.ppt) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture3-indexconstruction-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture3-indexconstruction-1per.pdf)

 | 
  * [IIR chapter 4](http://nlp.stanford.edu/IR-book/pdf/04const.pdf)

 |  
|   | Tues. 4/9  | PA1 release  |  **[Programming assignment #1](https://web.stanford.edu/class/cs276/pa/pa1.zip) released**  
 |   |  
|   | Thurs. 4/11  | Lecture (Chris)  |  **Algorithms for postings list compression**   
  

  * Videos: "Index Compression"
  * Slides: [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture4-compression.pptx) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture4-compression-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture4-compression-1per.pdf)

 | 
  * [IIR chapter 5](http://nlp.stanford.edu/IR-book/pdf/05comp.pdf)
  * MG sections 3.3-3.4
  * [Compression of inverted indexes for fast query evaluation (Scholer et al. 2002)](http://portal.acm.org/citation.cfm?id=564416)
  * [Inverted index compression using word-aligned binary codes (Anh and Moffat 2005)](http://www.springerlink.com/content/j66851228120170t/)
  * [Inverted index compression and query processing with optimized document ordering (Yan et al. 2009)](http://engineering.nyu.edu/~suel/papers/comp.pdf)

 |  
| Week 3  | Tues. 4/16  | Lecture (Pandu)  |  **Spelling correction**  
  

  * Videos: "Dictionaries and Tolerant Retrieval"
  * Slides: [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture5-spell_correction.ppt)| [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture5-spell_correction-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture5-spell_correction-1per.pdf)

 | 
  * [IIR chapter 3](http://nlp.stanford.edu/IR-book/pdf/03dict.pdf)
  * MG section 4.2
  * [How to write a spelling corrector (Peter Norvig)](http://norvig.com/spell-correct.html)
  * [Techniques for automatically correcting words in text (Kukich 1992)](http://portal.acm.org/citation.cfm?id=146380)
  * [Finding approximate matches in large lexicons (Zobel and Dart 1995)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.18.9400)
  * [Efficient Generation and Ranking of Spelling Error Corrections (Tillenius)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.49.1392)

 |  
|   | Tues. 4/16  | PS1 release  | **[Problem set #1](https://web.stanford.edu/class/cs276/ps/ps1_updated.pdf) released**  |   |   |  
|   | Tues. 4/16  | Query quiz release  | **[Query quiz](https://forms.gle/Timnnu3NEashupMeA) released**  |   |   |  
|   | Thurs. 4/18  | Lecture (Pandu)  |  **Scoring, term weighting and the vector space model**  
  

  * Videos: "Computing Scores"
  * Slides: [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture6-tfidf.ppt) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture6-tfidf-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture6-tfidf-1per.pdf)

 | 
  * [IIR chapter 7](http://nlp.stanford.edu/IR-book/pdf/07system.pdf)
  * [IIR chapter 11](http://nlp.stanford.edu/IR-book/pdf/11prob.pdf)

 |  
|   | Sun. 4/20  | Query quiz due  | **Query quiz due**  |   |   |  
| Week 4  | Tues. 4/23  | PA1 due  | **Programming assignment #1 due**  |   |   |  
|   | Tues. 4/23  | Guest lecture  |  _Guest lecture by Joachim Kupke (Principal Software Engineer, Google)_  
  
**NOTE: attendance required** for on-campus students  |   |  
|   | Tues. 4/23  | PA2 release  |  **[Programming assignment #2](https://web.stanford.edu/class/cs276/pa/pa2.zip) released**  
 |   |  
|   | Thurs. 4/25  | Lecture (Chris)  |  **Probabilistic IR: the binary independence model, BM25, BM25F**  
  

* Videos: "Vector Space Model"
* Slides: [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture7-probir.pptx) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture7-probir-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture7-probir-1per.pdf)
 | 
  * [IIR chapter 6](http://nlp.stanford.edu/IR-book/pdf/06vect.pdf)
  * [IIR chapter 11](http://nlp.stanford.edu/IR-book/pdf/11prob.pdf)

 |  
| Week 5  | Tues. 4/30  | PS1 due  | **Problem set #1 due**  |   |  
|   | Tues. 4/30  | Lecture (Chris)  |  **Evaluation methods & NDCG**  
  

  * Videos: "Result Summaries"
  * Slides: [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture8-evaluation.pptx) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture8-evaluation-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture8-evaluation-1per.pdf)

 | 
  * [IIR chapter 8](http://nlp.stanford.edu/IR-book/pdf/08eval.pdf)
  * MG section 4.5
  * MIR chapter 3

 |  
|   | Tues. 4/30  | Ranking quiz release  | **[Ranking quiz](https://web.stanford.edu/class/cs276/cgi-bin/queryrank/queryrank.php) released**  |   |  
|   | Thurs. 5/2  | Lecture (Pandu)  |  **Systems issues in efficient retrieval and scoring**  
  
Slides: 
  * [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture10-efficient-scoring.ppt) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture10-efficient-scoring-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture10-efficient-scoring-1per.pdf)

 | 
  * [IIR chapter 6](http://nlp.stanford.edu/IR-book/pdf/06vect.pdf)
  * [IIR chapter 7](http://nlp.stanford.edu/IR-book/pdf/07system.pdf)
  * [ Efficient Query Evaluation using a Two-Level Retrieval Process (Broder et al. 2003)](http://cis.poly.edu/westlab/papers/cntdstrb/p426-broder.pdf)

 |  
| Week 6  | Tues. 5/7  | PA2 due  | **Programming assignment #2 due**  |   |  
|   | Tues. 5/7  | Lecture (Pandu)  |  **Classification and clustering in vector spaces(Naive Bayes, kNN, decision boundaries)**  
  
Slides: 
  * [PPT](https://web.stanford.edu/class/cs276/handouts/lecture12-textcat.pptx) | [PDF/6](https://web.stanford.edu/class/cs276/handouts/lecture12-textcat-6up.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/handouts/lecture12-textcat.pdf)


  * Videos: "Naive Bayes"

 | 
  * [IIR chapter 13](http://nlp.stanford.edu/IR-book/pdf/13bayes.pdf)
  * [IIR chapter 14](http://nlp.stanford.edu/IR-book/pdf/14vcat.pdf)
  * [Reuters-21578](http://www.daviddlewis.com/resources/testcollections/reuters21578/)
  * [Machine learning in automated text categorization (Sebastiani 2002)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.17.6513)
  * [A re-examination of text categorization methods (Yang et al. 1999)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.11.9519)
  * [A Comparison of event models for naive Bayes text classification (McCallum et al. 1998)](http://citeseer.ist.psu.edu/mccallum98comparison.html)
  * [Tackling the poor assumptions of Naive Bayes classifier (Rennie et al. 2003)](http://www.stanford.edu/class/cs276/handouts/rennie.icml03.pdf)
  * [Machine learning in automated text categorization (Sebastiani 2002)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.17.6513)
  * [ A re-examination of text categorization methods (Yang et al. 1999)](http://citeseer.ist.psu.edu/yang99reexamination.html)
  * [Evaluating and optimizing autonomous text classification systems (Lewis 1995)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.33.867)
  * Tom Mitchell. _Machine Learning._ McGraw-Hill, 1997.
  * Trevor Hastie, Robert Tibshirani, Jerome Friedman. _Elements of Statistical Learning: Data Mining, Inference, and Prediction_. Springer-Verlag, New York, 2001.
  * Open Calais
  * Weka

 |  
|   | Thurs. 5/9  | Lecture (Chris)  |  **Text classification**  
  
Slides: 
  * [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture13-DTrees-textcat.pptx) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture13-DTrees-textcat-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture13-DTrees-textcat-1per.pdf)

 | 
  * [IIR chapter 15](http://nlp.stanford.edu/IR-book/pdf/15svm.pdf)
  * [Reuters-21578](http://www.daviddlewis.com/resources/testcollections/reuters21578/)
  * [A tutorial on support vector machines for pattern recognition (Burges 1998)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.18.1083)
  * [Using SVMs for text categorization (Dumais 1998)](http://ezproxy.stanford.edu:2072/xpls/abs_all.jsp?isnumber=15361&arnumber=708428&count=14&index=5)
  * [Inductive learning algorithms and representations for text categorization (Dumais et al. 1998)](http://portal.acm.org/citation.cfm?id=288651)
  * [A Re-examination of text categorization methods (Yang et al. 1999)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.30.5594)
  * [Text categorization based on regularized linear classification methods (Zhang et al. 2001)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.5553)
  * [A loss function analysis for classification methods in text categorization (Li et al. 2003)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.8.2881)
  * Trevor Hastie, Robert Tibshirani, Jerome Friedman. _Elements of Statistical Learning: Data Mining, Inference, and Prediction_. Springer-Verlag,New York, 2001.
  * Thorsten Joachims. _Learning to Classify Text using Support Vector Machines_. Kluwer, 2002.

 |  
|   | Thurs/ 5/9  | PA3 release  | **[Programming assignment #3](https://web.stanford.edu/class/cs276/pa/pa3.zip) released**  |   |   |  
| Week 7  | Tues. 5/14  | Lecture (Chris)  |  **Distributed word representations for IR**  
  
Slides: 
  * [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture14-distributed-representations.pptx) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture14-distributed-representations-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture14-distributed-representations-1per.pdf)

 | 
  * [ Distributed Representations of Words and Phrases and their Compositionality (Mikolov et al., 2013)](http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf)
  * [GloVe: Global Vectors for Word Representation (Pennington et al., 2014)](http://nlp.stanford.edu/pubs/glove.pdf)

 |  
|   | Tues. 5/14  | PS2 released  | **[Problem set #2](https://web.stanford.edu/class/cs276/ps/ps2.zip) released**  |   |   |  
|   | Thurs. 5/16  | Lecture (Chris)  |  **Learning to rank**  
  
Slides: 
  * [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture15-learning-ranking.pptx) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture15-learning-ranking-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture15-learning-ranking-1per.pdf)

 | 
  * [IIR sections 6.1.2-6.1.3](http://nlp.stanford.edu/IR-book/pdf/06vect.pdf)
  * [IIR section 15.4](http://nlp.stanford.edu/IR-book/pdf/15svm.pdf)
  * [LETOR benchmark datasets](http://research.microsoft.com/users/LETOR/)
  * [Discriminative models for information retrieval (Nallapati 2004)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.2.9065)
  * [Adapting ranking SVM to document retrieval (Cao et al. 2006)](http://portal.acm.org/citation.cfm?id=1148205)
  * [A support vector method for optimizing average precision (Yue et al. 2007)](http://portal.acm.org/citation.cfm?id=1277790)

 |  
| Week 8  | Tues. 5/21  | Lecture (Chris)  |  **Link analysis**  
  
Slides: 
  * [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture17-linkanalysis.pptx) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture17-linkanalysis-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture17-linkanalysis-1per.pdf)

 | 
  * [IIR chapter 21](http://nlp.stanford.edu/IR-book/pdf/21link.pdf)
  * [Ranking the web frontier (Eiron et al. 2004)](http://portal.acm.org/citation.cfm?id=988714)
  * [The WebGraph framework I: Compression techniques (Boldi et al. 2004)](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.14.6417)
  * [Extrapolation methods for accelerating PageRank computations (Kamvar et al. 2003)](http://portal.acm.org/citation.cfm?id=775190)
  * [Searching the workplace web (Fagin et al. 2003](http://portal.acm.org/citation.cfm?id=775152.775204)

 |  
|   | Thurs. 5/23  | PS2 due  | **Problem set #2 due**  |   |  
|   | Thurs. 5/23  | Guest lecture  |  **Guest lecture by Susan Dumais (Distinguished Scientist & Deputy Managing Director, Microsoft Research Lab)**  
  
Slides: 
  * [PDF/1](https://web.stanford.edu/class/cs276/19handouts/guest_lecture-SDumais-May2019)

**NOTE: attendance required** for on-campus students   |   |  
| Week 9  | Tues. 5/28  | Lecture (Pandu)  |  **Crawling and near-duplicate pages**  
  
Slides: 
  * [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture18-crawling.ppt) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture18-crawling-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture18-crawling-1per.pdf)

 | 
  * [IIR chapter 19](http://nlp.stanford.edu/IR-book/pdf/19web.pdf)
  * [IIR chapter 20](http://nlp.stanford.edu/IR-book/pdf/20crawl.pdf)
  * [Mercator: A scalable, extensible web crawler (Heydon et al. 1999)](https://webarchive.jira.com/wiki/download/attachments/5441/1999-Mercator.pdf)
  * [A standard for robot exclusion](http://www.robotstxt.org/orig.html)

 |  
|   | Thurs. 5/30  | PA3 due  | **Programming assignment #3 due**  |   |  
|   | Thurs. 5/30  | Lecture (Chris)  |  **Question answering**  
  
Slides: 
  * [PPT](https://web.stanford.edu/class/cs276/19handouts/lecture19-Web-QA.pptx) | [PDF/6](https://web.stanford.edu/class/cs276/19handouts/lecture19-Web-QA-6per.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/19handouts/lecture19-Web-QA-1per.pdf)

 |   |  
| Week 10  | Tues. 6/4  | Lecture (Pandu)  |  **Personalization**  
  
Slides: 
  * [PPT](https://web.stanford.edu/class/cs276/handouts/personalization-lecture.ppt) | [PDF/6](https://web.stanford.edu/class/cs276/handouts/personalization-lecture-6-per-page.pdf) | [PDF/1](https://web.stanford.edu/class/cs276/handouts/personalization-lecture-1-per-page.pdf)

 | 
  * [J. Teevan, S. Dumais, E. Horvitz. Potential for personalization. 2010](http://research.microsoft.com/pubs/154553/tochi10.pdf)
  * [J. Pitkow et al. Personalized search. 2002](http://www.cond.org/p50-pitkow.pdf)
  * [J. Teevan, S. Dumais, E. Horvitz. Personalizing search via automated analysis of interests and activities. 2005](http://research.microsoft.com/en-us/um/people/sdumais/SIGIR2005-PersonalizedSearch.pdf)
  * [P. Bennett et al. Inferring and using location metadata to personalize Web search. 2011](http://research.microsoft.com/pubs/150729/BennettSIGIR2011.pdf)
  * [T. Haveliwala. Topic-sensitive pagerank. 2002.](http://www-cs-students.stanford.edu/~taherh/papers/topic-sensitive-pagerank.pdf)
  * [G. Jeh and J. Widom. Scaling personalized Web search. 2003](http://infolab.stanford.edu/~glenj/spws.pdf)
  * [M. Curtiss et al. Unicorn: A system for searching the social graph. 2013](http://www.vldb.org/pvldb/vol6/p1150-curtiss.pdf)

 |  
| Exam week  | Fri. 6/7  | Final exam  | **Alternate final exam (8:30-11:30am)**  |   |  
|   | Wed. 6/12  | Final exam  | **Final exam (3:30-6:30pm)**  | Practice final and solution are on Canvas  
 |  
## FAQ
Can I take this course on credit/no credit basis?
Yes. Credit will be given to those who would have otherwise earned a C- or above.
Can I audit or sit in?
In general we are very open to sitting-in guests if you are a member of the Stanford community (registered student, staff, and/or faculty). Out of courtesy, we would appreciate that you first email us or talk to the instructor after the first class you attend.
I have a question about the class. What is the best way to reach the course staff?
In general, we ask students to use the [Piazza forum](https://piazza.com/stanford/spring2019/cs276) for our class so that other students may benefit from your questions and our answers. If you have a personal matter that you believe is not appropriate to share on Piazza (even in a private post), you may email the course staff at  cs276-spr1819-staff@lists.stanford.edu. We may NOT be able to reply emails sent to individual instructors or TAs regarding the class.
As an SCPD student, how do I take the final exam?
For SCPD students, if you are local, you're encouraged to just come to Stanford for one of the on-campus exams. If you decide to take on-campus exams, please let us know in advance (through a survey that we send out closer to the final exam date). If you are not local or can't make it at the on-campus exams, you need to line up an exam monitor (usually your manager or a co-worker at your company), and submit the form specifying this person to SCPD in advance. You won't get an exam if you don't have an exam monitor on file. You need to make sure we get the exam back promptly (monitor should scan and email directly to us).If you are taking the exam in the first 24 hour period, you need to make sure we get the exam back from your monitor by Saturday 12:30 pm PT. If you are taking the exam in the second 24 hour period, you need to make sure we get the exam back from your monitor by Wednesday 7:30 pm PT. We need to grade exams immediately after that in order to be able to turn grades in in time. **Please refer to the[course policies](https://web.stanford.edu/class/cs276/course_policies.html) page for Final exam details**
Will there be virtual office hours for SCPD students?
We will be sure to join a Google hangout for at least some office hours. We will use [ QueueStatus](http://queuestatus.com/queues/411) and post google hangout link on QueueStatus page in each office hour for SCPD students.