I am using the below code to generate the answers using BERT model.
But how do I generate the top 3 answers and their respective confidence scores/metrics for each question ? Is there any such method ?
from deeppavlov import build_model, configs
bert_model = build_model(configs.squad.squad_bert_infer, download=False)
We don’t have such method implemented, but it is possible to implement it by yourself:
best answer selection happens in this codeline: https://github.com/deepmipt/DeepPavlov/blob/master/deeppavlov/models/bert/bert_squad.py#L359
you can modify this logic to take top N answers and add argument
In such way, you will be able to get top N answers, where answers are all best answers from each chunk.
It would be great if you could add this functionality to DeepPavlov via Pull Request.