# Stanford CS 224N | Project Reports

https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/project.html

[CS224n Home](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/index.html)
  * [Coursework](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/index.html#coursework)
  * [Schedule](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/index.html#schedule)
  * [Office Hours](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/office_hours.html)
  * [Lecture Videos](https://canvas.stanford.edu/courses/111053/external_tools/3367)
  * [Project Reports](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/project.html)
  * [Piazza](https://piazza.com/stanford/winter2020/cs224n)


[ ![](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/images/stanford-nlp-logo-new.jpg) ](http://nlp.stanford.edu/) [ ![](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/images/stanfordlogo.jpg) ](http://stanford.edu/)
# CS224n: Natural Language Processing with Deep Learning
### Stanford / Winter 2020
## Final Project Reports for 2020
There were two options for the course project. Students either chose their own topic (["Custom Project"](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/project.html#custom)), or took part in a competition to build Question Answering models for the [SQuAD](https://rajpurkar.github.io/SQuAD-explorer/) 2.0 challenge (["Default Project"](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/project.html#default)). This year saw fewer final projects than past years due to challenges caused by COVID-19. 
You can find links to previous years' reports under _Previous Offerings_ on the [homepage](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/index.html#content). 
## Prize Winners
Congratulations to the following teams, who produced exceptional prize-winning projects! Thank you also to our sponsors, who generously donated prizes. 
#### Prizes for Best Custom Project Reports
  * [Generative Pre-Training Language Models with Auxiliary Conditional Summaries](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report50.pdf), Phillip Keonho Kim, Kevin Chen, Kelly He 
  * [Exploring TEXTFOOLER’s Syntactically- and Semantically-Sound Distributed Adversarial Attack Methodology](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report12.pdf), Josiah David Wong 
  * [Efficient Language Modeling of Long-Term Dependencies](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report01.pdf), Manan Ajay Shah 
  * [Tidy analysis of TyDi: Analysing knowledge sharing in Multilingual Domain](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report18.pdf) Akshat Jindal, Chetanya Rastogi 


#### Prizes for Best Default Project Reports
  * [Extended QANet on SQuAD 2.0](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report04.pdf) Guillaume Adrien Nervo, Pablo Emile Veyrat 
  * [Lightweight neural network development via knowledge distillation of deep neural network](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report18.pdf) Naoki Yamamura 
  * [Avengers: Achieving Superhuman Performance for Question Answering on SQuAD 2.0 Using Multiple Data Augmentations, Randomized Mini-Batch Training and Architecture Ensembling](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report15.pdf) Li Yi 
  * [Gestalt: a Stacking Ensemble for SQuAD2.0](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report27.pdf) Mohamed G Mahmoud 


## Custom Projects  
| Project Name  | Authors  |  
| --- | --- |  
|  [Context-Based Models for Sarcasm Detection](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report02.pdf)  | Nicholas Nestor Benavides, Angelo Reyes Ramos  |  
|  [Wikipedia Simplifier](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report03.pdf)  | Daniel Kharitonov  |  
|  [Measuring Impact and Consumption of Local News in a Changing Environment](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report04.pdf)  | Yu-chen Tuan, Austin Pennington, Anish Saha  |  
|  [Inclusivity-Exclusivity Inference Using Recurrent Neural Networks](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report05.pdf)  | Santosh Murugan, Elissa Yiyi Li  |  
|  [Enhancing Arabic Diacritization with Word-Ending Specific Models](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report06.pdf)  | Ahmed Abdallah Shuaibi, Hassan Hossam Fahmy  |  
|  [Music Recommendation for Podcast Scripts: Detecting Emotion from Text](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report07.pdf)  | Anne Liu Lee, Shreya Ravi, Alex Ping Tsun  |  
|  [Couplet Scoring and Prediction using ALBERT](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report08.pdf)  | Ju Huo  |  
|  [Can FOMC Minutes Predict the Federal Funds Rate?](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report09.pdf)  | Ye Ye  |  
|  [DeepSumm - Deep Code Summaries using Neural Transformer Architecture](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report10.pdf)  | Vivek Kumar Gupta  |  
|  [DeepClassic: Music Generation with Neural Networks](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report11.pdf)  | Raphael Andreas Rene Abbou  |  
|  [How much does pre-trained information help? Partially re-initializing BERT during fine-tuning to analyze the contribution of layers](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report13.pdf)  | Davide Giovanardi, Trisha Singh  |  
|  [Sentence Boundary Detection in Legal Texts](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report14.pdf)  | Krithika Iyer  |  
|  [Keyword-based Code Auto-Complete](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report15.pdf)  | Rui Aguiar, Faraz Fadavi Akhavan Bonab  |  
|  [Evaluation Metrics for Split and Rephrase](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report16.pdf)  | Rui Yan  |  
|  [Idiomatic Language Translation and Transfer Learning](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report19.pdf)  | David Andres Estrada-Arias, Jupinder Parmar  |  
|  [Improvement of deep learning techniques for ICD-9 code classification using MIMIC-III medical notes](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report20.pdf)  | Derek Russell Jow, Anthony Li, Soham Uday Gadgil  |  
|  [DrCoding: Using Transformers to Predict ICD-9 Codes from Discharge Summaries](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report21.pdf)  | Tom Jin  |  
|  [You Don’t Sound Like Me: Analyzing Hiring Based on Pre-hire Language Use](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report22.pdf)  | Samaksh Ajay Goyal  |  
|  [Detecting Bias in Lending Data with NLP Models](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report23.pdf)  | Orion Earl Darley, Senthil Selvaraj  |  
|  [Speech Recognition for Accented English](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report24.pdf)  | Sasha Harrison, Claire Marika Pajot  |  
|  [Predicting ICD-9 Codes from Medical Notes - Does the Magic of BERT Applies Here?](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report25.pdf)  | Yiyun Chen  |  
|  [Multilingual Customized Bert for Zero Shot Sequence Classification](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report26.pdf)  | Yuehao Wu  |  
|  [Combining NLP and Computer Vision to Help Blind People](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report27.pdf)  | Volha Leusha  |  
|  [Synth-tax: Modification of BERT Syntax Representations via Structural Probes](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report28.pdf)  | Benjamin Louis Newman, Julia Xiaozhe Gong, Kai-Siang Ang  |  
|  [ClinicalBertSum: RCT Summarization by Using Clinical BERT Embeddings](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report29.pdf)  | Xiaomeng Jin, Zihan Wang, Mingyi Lu  |  
|  [What errors are made by TyDi QA systems?](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report30.pdf)  | Nivedita Rahurkar  |  
|  [More Abstractive Summarization with Pointer-Generator Networks](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report31.pdf)  | Michael Andres Lin  |  
|  [Generate Symbolic Plans for Long-Horizon Robot Manipulation Tasks](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report33.pdf)  | Karen Yang  |  
|  [DAMsL: A meta-learning based approach for dialogue state tracking](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report34.pdf)  | Avanika Narayan, Josh Hedtke  |  
|  [Improving Language Generation with Sentence Coherence Objective](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report35.pdf)  | Jie Yang, Ruixiao Sun, Mehrdad Yousefzadeh  |  
|  [Document Classification with DocBERT, et. Al.](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report36.pdf)  | Dale Angus  |  
|  [BERT Goes to College: Understanding the Role of Pretrained Layers During Finetuning](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report37.pdf)  | Daniel Huang, Jacky Lin, Shan Zhou  |  
|  [Inside-Out Code Auto-Completion](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report38.pdf)  | Lauren Gillespie, Gabriel Poesia Reis e Silva, Scott Viteri  |  
|  [Ensemble-Learning for Sustainable NLP](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report39.pdf)  | Surya Narayanan Hari, Elena Berman  |  
|  [Man vs. Machine: Master Debating with Argumentative Dialogue](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report40.pdf)  | Jason Jin Zhao  |  
|  [What is a Fact?](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report41.pdf)  | Juliette Elizabeth Love, Kevin Ji, Moritz Pascal Stephan  |  
|  [Deep learning-based detection of Dysarthric speech disability](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report42.pdf)  | Siddhartha Prakash  |  
|  [Fine-Tuned BERT for the Detection of Political Ideology](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report43.pdf)  | Maria Del Mar Castanos, Alexandre Simoes Gomes Junior  |  
|  [Modeling Long-Distance Dependencies with Second-Order LSTMs](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report44.pdf)  | Yifeng Lin, Sam Alan Spinner, Noa Flora Bendit-Shtull  |  
|  [Hierarchical Reward Shaping Reinforcement Learning for Paragraph Generation](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report45.pdf)  | Heejung Chung, Alex Nam  |  
|  [Plot Holes and Text Topology](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report46.pdf)  | Egor Alimpiev, Vivek Robert Myers  |  
|  [Simple Transformers for PHI De-identification](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report47.pdf)  | Aditya Khandelwal, Arjun Soin  |  
|  [Evaluating a relationship between mental health and wearable sensors using NLP](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report48.pdf)  | Artem Alex Trotsyuk  |  
|  [Typologically Diverse QA: How many training examples do you need for a new language anyway?](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report49.pdf)  | Caterina Wu, Tassica Linmei Lim, Tatiana Wu  |  
|  [A Comparative Study of Neural Question Generation Models](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report51.pdf)  | Ankit Dwivedi, Arunothia Marappan  |  
|  [Know When To Abstain: Calibrating Question Answering System under Domain Shift](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report52.pdf)  | Wanze Xie, Ruocheng Wang  |  
|  [A Close Examination of Factual Correctness Evaluation in Abstractive Summarization](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report53.pdf)  | Yuhui Zhang  |  
|  [Second-order RNNs: Resurgence of Recurrence](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/custom/report54.pdf)  | Shreya Singh, Sundararajan Renganathan  |  
## Default Projects  
| Project Name  | Authors  |  
| --- | --- |  
|  [Reimplementing QANet](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report01.pdf)  | Adrian Ng  |  
|  [Applying Computer Vision Methodologies to Combat Adversarial Inputs for QA Reading Comprehension](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report02.pdf)  | Stephanie Danielle Brito, Derek Bryan McCreight  |  
|  [Question Answering on SQuAD 2.0](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report03.pdf)  | Han Wu, Kevin Han, Yuqi Jin  |  
|  [Understanding and Improving Performance of QA Systems on Adversarial Evaluation](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report05.pdf)  | Shai Limonchik, Andy Lapastora  |  
|  [Question Answering with Gated Attention and Multitask Learning](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report06.pdf)  | Mingyang Ling, Lam Wing Chan  |  
|  [Applying and Adapting the Reformer as a Computationally Efficient Approach to the SQuAD 2.0 Question-Answering Task](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report07.pdf)  | Hugo Clifford Kitano, Taide Ding  |  
|  [SQuAD 2.0 Based on ALBERT and Ensemble](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report08.pdf)  | Yanpei Tian  |  
|  [Ensemble ALBERT on SQuAD 2.0](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report09.pdf)  | Veronica Peng, Renee Li, Allan Li  |  
|  [Intermediate Pretrained Contextual Embedding Models With Applications in Question Answering](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report10.pdf)  | Ryan Silva  |  
|  [ALBERT Premium for SQuAD 2.0](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report11.pdf)  | Yibing Du, Yanhao Jiang  |  
|  [Answerability Verification in SQuAD](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report12.pdf)  | Thatcher Solomon Freeman, Albert Zuo  |  
|  [Tackling SQuAD 2.0 with Ensemble Methods and Data Augmentation](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report13.pdf)  | Elijah Amir Freeman, Emanuel Pinilla, Nikka Michelle Mofid  |  
|  [QANet implementation with multiple embedding methods for SQuAD](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report16.pdf)  | Ying Guo, Jiachen Xu  |  
|  [Compressed SQuAD 2.0 Model With BERT](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report17.pdf)  | Wenli Looi, Andrew Lawrence Deng  |  
|  [A PyTorch implementation of QANet](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report19.pdf)  | Dat Pham Nguyen  |  
|  [Rare and novel words in SQuAD 2.0: An application of the form-context-model](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report20.pdf)  | Axel Gross-Klussmann  |  
|  [Transformer-Based Models for SQuAD 2.0](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report21.pdf)  | Yuyan Wang, Lingjue Xie, Alex Xiaotong Gui  |  
|  [Machine Reading Comprehension with Abstention Verification](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report22.pdf)  | Rakesh Kumar Grewal, Mara Rachel Finkelstein, Abhishek Rawat  |  
|  [Question Answering on Adversarial SQuAD 2.0](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report23.pdf)  | Ryan Guan  |  
|  [Neural Question Generation: Transformed](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report24.pdf)  | Jeffrey Hu  |  
|  [Bert](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report25.pdf)  | Mohith Rajendra  |  
|  [FusionNet: working smarter, not harder with SQuAD](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report26.pdf)  | Sebastian Hurubaru, Francois Xavier Alain Henri Chesnay  |  
|  [Transferability of Contextual Representations for Question Answering](https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1204/reports/default/report28.pdf)  | Pranav Bhardwaj, Travis McGuire  |