Get top 3 answers and their respective confidence scores using squad_bert_infer_model

DeepPavlov Library
1

Hi

I am using the below code to generate the answers using BERT model.
But how do I generate the top 3 answers and their respective confidence scores/metrics for each question ? Is there any such method ?

from deeppavlov import build_model, configs
bert_model = build_model(configs.squad.squad_bert_infer, download=False)

1 Like
2

Hi!

We don’t have such method implemented, but it is possible to implement it by yourself:
best answer selection happens in this codeline: https://github.com/deepmipt/DeepPavlov/blob/master/deeppavlov/models/bert/bert_squad.py#L359
you can modify this logic to take top N answers and add argument top_n to __init__.

In such way, you will be able to get top N answers, where answers are all best answers from each chunk.

It would be great if you could add this functionality to DeepPavlov via Pull Request.

1 Like