This repo is under re-implementation. Due to frequent modification, this code may not run normally.
An implementation of QANet with PyTorch.
Now it can reach EM/F1 = 70.5/77.2 after 20 epoches for about 20 hours on one 1080Ti card.
Python 3.6 & PyTorch 0.4
- Install pytorch 0.4 for Python 3.6+
pip install spacy tqdm ujson requests
python -m spacy download en
dataset.py: download dataset and parse.
main.py: program entry.
models.py: QANet structure.
Differences from the paper
- The paper doesn't mention which activation function they used. I use relu.
- I don't set the embedding of
- The connector between embedding layers and embedding encoders may be different from the implementation of Google, since the description in the paper is inconsistent (residual block can't be used because the dimensions of input and output are different) and they don't say how they implement it.
- Max passage length is 300 instead of 400 since I don't have much GPU memory.
- Reduce memory usage
- Performance analysis
- Reach state-of-art scroes of the original paper
- Ablation analysis