How to use custom train model

DeepPavlov Library Models
1

I have build my deeppavlov ner model from scratch with my custom dataset. The model is stored in my directory. Now I want to use this model in an another device. How can I use this?

2

Hi, @Sayak

You should copy trained data to another device. What config are we talking about?

3

thanks for replying, I have solved this issue.
Can you please tell me how to custom train a KBQA and ODQA model.
Please give me detail explanation. I want to build it from scratch

4

ODQA consists of tr-idf retriever and reading comprehension model (SQuAD).

Training tf-idf ranking is explained here Google Colab.

To train the SQuAD model on your dataset you should copy files with train and dev data to the folder DeepPavlov/squad_torch_bert.json at f5117cd9ad1e64f6c2d970ecaa42fc09ccb23144 · deepmipt/DeepPavlov · GitHub
Then you should change the path where the model checkpoints will be saved DeepPavlov/squad_torch_bert.json at f5117cd9ad1e64f6c2d970ecaa42fc09ccb23144 · deepmipt/DeepPavlov · GitHub
Your dataset should have the structure appropriate for squad_dataset_reader (example of the dataset: http://files.deeppavlov.ai/datasets/squad-v1.1.tar.gz )

To train custom KBQA model you should train entity detection DeepPavlov/ner_lcquad_bert_ent_and_type.json at master · deepmipt/DeepPavlov · GitHub
example of dataset format DeepPavlov/ner_lcquad_bert_ent_and_type.json at d73f45733d6b23347871aa293309730303b64450 · deepmipt/DeepPavlov · GitHub

Then relation prediction DeepPavlov/rel_ranking_bert.json at master · deepmipt/DeepPavlov · GitHub
example of dataset format DeepPavlov/rel_ranking_bert.json at d73f45733d6b23347871aa293309730303b64450 · deepmipt/DeepPavlov · GitHub

Then query prediction DeepPavlov/query_pr.json at master · deepmipt/DeepPavlov · GitHub

1 Like