Unable to get the open domain question answering tutorial working

DeepPavlov Library Tutorials & Guidelines
1

Hi all,
I am trying to follow the quick start tutorial at Installation — DeepPavlov 1.3.0 documentation

I have installed deeppavlov in a virtual environment on my host running ubuntu 19.10. However, things fall over due to a lack of a config file. In my home folder, I do see a folder called .deeppavlov but I do not find any config files there.
I can execute
from deeppavlov import configs
then what? I suspect I need to download a model but am stuck at getting the configuration file.

I eventually want to run the open domain question answer skill at http://docs.deeppavlov.ai/en/master/features/skills/odqa.html on my own data set.

I ran the following commands
from deeppavlov import configs
from deeppavlov.core.commands.train import train_evaluate_model_from_config
so far so good.
train_evaluate_model_from_config(configs.doc_retrieval.en_ranker_tfidf_wiki, download=True)
after the above command, I get the following error.
Traceback (most recent call last):
File “”, line 1, in
File “/home/pranav/.virtualenvs/deep_p/lib/python3.7/site-packages/deeppavlov/core/commands/train.py”, line 92, in tra
in_evaluate_model_from_config
data = read_data_by_config(config)
File “/home/pranav/.virtualenvs/deep_p/lib/python3.7/site-packages/deeppavlov/core/commands/train.py”, line 58, in rea
d_data_by_config
return reader.read(data_path, **reader_config)
File “/home/pranav/.virtualenvs/deep_p/lib/python3.7/site-packages/deeppavlov/dataset_readers/odqa_reader.py”, line 81
, in read
self._build_db(save_path, dataset_format, expand_path(data_path))
File “/home/pranav/.virtualenvs/deep_p/lib/python3.7/site-packages/deeppavlov/dataset_readers/odqa_reader.py”, line 14
5, in _build_db
files = [f for f in self.iter_files(data_path)]
File “/home/pranav/.virtualenvs/deep_p/lib/python3.7/site-packages/deeppavlov/dataset_readers/odqa_reader.py”, line 14
5, in
files = [f for f in self.iter_files(data_path)]
File “/home/pranav/.virtualenvs/deep_p/lib/python3.7/site-packages/deeppavlov/dataset_readers/odqa_reader.py”, line 10
6, in iter_files
raise RuntimeError(“Path doesn’t exist: {}”.format(path))
RuntimeError: Path doesn’t exist: /home/pranav/.deeppavlov/downloads/odqa/enwiki

How do I set up deeppavlov to handle open domain question and answering?
Pranav

2

Hi @slimprize,
Sorry for the late reply.

  1. pre-defined configs are stored in the python module directory, for you it will be /home/pranav/.virtualenvs/deep_p/lib/python3.7/site-packages/deeppavlov/configs.
    This directory should have the same structure as https://github.com/deepmipt/DeepPavlov/tree/master/deeppavlov/configs
  2. the configs object replicates the configs directory structure and the leafs of this tree are paths. So configs.doc_retrieval.en_ranker_tfidf_wiki will have /home/pranav/.virtualenvs/deep_p/lib/python3.7/site-packages/deeppavlov/configs/doc_retrieval/en_ranker_tfidf_wiki.json in its value.
  3. The exception itself is actually about a different thing: dataset reader tries to rebuild a database from source data but there is no source data. You can take a loot at the issue #1082 for some info on training the ranking model on your own data.
3

Hi,
Many thanks. I have the model working but have not trained my own data yet.