In this project, we implemented and tuned models that performed QuestionAnswering (QA) tasks on the abstracts of academic Computer Science papers.My baseline models were simple QA models based primarily on BERT. Eachmodel involved switching out BERT for one of its variants, including SciBERT(pre-trained on scientific literature) and DistilBERT (BERT after network pruning).

My main model was a combination of SciBERT and BiDAF. For all models, wetrained and evaluated the model on a SQuAD-like dataset called PaperQA, whichcontains crowd-sourced questions/answers about CS paper abstracts.

I used the top layers of BiDAF model that are context-query attention layer and modelling layer on top of SciBERT embeddings.

A short video presentation of the same could be found here: