Why does "KeyError: 'question_raw'" appear?

1

image
image1119×469 19.4 KB

Here is the output:
C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\requests_init_.py:102: RequestsDependencyWarning: urllib3 (1.26.8) or chardet (5.1.0)/charset_normalizer (2.0.10) doesn’t match a supported version!
warnings.warn("urllib3 ({}) or chardet ({})/charset_normalizer ({}) doesn’t match a supported "
Looking in indexes: Simple Index
Requirement already satisfied: torch<1.14.0,>=1.6.0 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (1.13.1)
Requirement already satisfied: typing-extensions in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from torch<1.14.0,>=1.6.0) (4.5.0)
Looking in indexes: Simple Index
Requirement already satisfied: transformers<4.25.0,>=4.13.0 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (4.24.0)
Requirement already satisfied: filelock in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from transformers<4.25.0,>=4.13.0) (3.9.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.10.0 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from transformers<4.25.0,>=4.13.0) (0.14.1)
Requirement already satisfied: numpy>=1.17 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from transformers<4.25.0,>=4.13.0) (1.22.3)
Requirement already satisfied: packaging>=20.0 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from
transformers<4.25.0,>=4.13.0) (21.3)
Requirement already satisfied: pyyaml>=5.1 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from transformers<4.25.0,>=4.13.0) (6.0)
Requirement already satisfied: regex!=2019.12.17 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from transformers<4.25.0,>=4.13.0) (2022.1.18)
Requirement already satisfied: requests in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from transformers<4.25.0,>=4.13.0) (2.27.1)
Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from transformers<4.25.0,>=4.13.0) (0.13.3)
Requirement already satisfied: tqdm>=4.27 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from transformers<4.25.0,>=4.13.0) (4.62.3)
Requirement already satisfied: fsspec in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from huggingface-hub<1.0,>=0.10.0->transformers<4.25.0,>=4.13.0) (2023.4.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from huggingface-hub<1.0,>=0.10.0->transformers<4.25.0,>=4.13.0) (4.5.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from packaging>=20.0->transformers<4.25.0,>=4.13.0) (3.0.8)
Requirement already satisfied: colorama in c:\users\administrator\appdata\roaming\python\python310\site-packages (from tqdm>=4.27->transformers<4.25.0,>=4.13.0) (0.4.6)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages
(from requests->transformers<4.25.0,>=4.13.0) (1.26.8)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from requests->transformers<4.25.0,>=4.13.0) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from requests->transformers<4.25.0,>=4.13.0) (2.0.10)
Requirement already satisfied: idna<4,>=2.5 in c:\users\administrator\appdata\local\programs\python\python310\lib\site-packages (from requests->transformers<4.25.0,>=4.13.0) (3.3)
2023-05-03 11:37:40.235 INFO in ‘deeppavlov.download’[‘download’] at line 138: Skipped http://files.deeppavlov.ai/v1/squad/multi_squad_ru_torch_bert_retr_noans.tar.gz download because of matching hashes
C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\deeppavlov\core\models\torch_model.py:75: UserWarning: Failed to initialize NumPy: module compiled against API version 0x10 but this version of numpy is 0xf (Triggered internally at …\torch\csrc\utils\tensor_numpy.cpp:77.)
self.device = torch.device(“cuda” if torch.cuda.is_available() and device == “gpu” else “cpu”)
Downloading pytorch_model.bin: 100%|██████████████████████████████████████████████████████████████████| 714M/714M [11:59<00:00, 993kB/s]
Some weights of the model checkpoint at DeepPavlov/rubert-base-cased were not used when initializing BertForQuestionAnswering: [‘cls.predictions.decoder.weight’, ‘cls.predictions.bias’, ‘cls.predictions.transform.dense.weight’, ‘cls.predictions.transform.dense.bias’, ‘cls.seq_relationship.bias’, ‘cls.predictions.transform.LayerNorm.bias’, ‘cls.predictions.transform.LayerNorm.weight’, ‘cls.predictions.decoder.bias’, ‘cls.seq_relationship.weight’]

  • This IS expected if you are initializing BertForQuestionAnswering from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
  • This IS NOT expected if you are initializing BertForQuestionAnswering from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
    Some weights of BertForQuestionAnswering were not initialized from the model checkpoint at DeepPavlov/rubert-base-cased and are newly initialized: [‘qa_outputs.bias’, ‘qa_outputs.weight’]
    You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
2

Hi.

You are using model answers a question based on a given context (e.g, a paragraph of text), where the answer to the question is a segment of the context. This means, that model expects exactly two arguments: context and question. In your example it will be something like:

contexts_batch = ['DeepPavlov is an open-source conversational AI library built by DeepPavlovl.ai'] * 2
questions_batch = ['What is DeepPavlov?', 'Who created DeepPavlov?']
model(contexts_batch, questions_batch)
2 Likes