NER: return tag probabilities?

Hi guys,

I tried getting tag probabilities from a NER model instead of single tags. I loaded the ner_rus_bert_torch config from deeppavlov.configs.ner and set config[‘chainer’][‘pipe’][2][‘return_probas’] to True.

I get the following error:


NotImplementedError Traceback (most recent call last)
in
----> 1 ner_model_default([‘Мытищинские проститутки посетили короля Испании с ответным визитом’])

/mnt/data/wx1155560/envs/dp/lib/python3.6/site-packages/deeppavlov/core/common/chainer.py in call(self, *args)
205
206 def call(self, *args):
→ 207 return self._compute(*args, param_names=self.in_x, pipe=self.pipe, targets=self.out_params)
208
209 @staticmethod

/mnt/data/wx1155560/envs/dp/lib/python3.6/site-packages/deeppavlov/core/common/chainer.py in _compute(failed resolving arguments)
228 res = component.call(**dict(zip(in_keys, x)))
229 else:
→ 230 res = component.call(*x)
231 if len(out_params) == 1:
232 mem[out_params[0]] = res

/mnt/data/wx1155560/envs/dp/lib/python3.6/site-packages/deeppavlov/core/data/simple_vocab.py in call(self, batch, is_top, **kwargs)
90 def call(self, batch, is_top=True, **kwargs):
91 if isinstance(batch, Iterable) and not isinstance(batch, str):
—> 92 looked_up_batch = [self(sample, is_top=False) for sample in batch]
93 else:
94 return self[batch]

/mnt/data/wx1155560/envs/dp/lib/python3.6/site-packages/deeppavlov/core/data/simple_vocab.py in (.0)
90 def call(self, batch, is_top=True, **kwargs):
91 if isinstance(batch, Iterable) and not isinstance(batch, str):
—> 92 looked_up_batch = [self(sample, is_top=False) for sample in batch]
93 else:
94 return self[batch]

/mnt/data/wx1155560/envs/dp/lib/python3.6/site-packages/deeppavlov/core/data/simple_vocab.py in call(self, batch, is_top, **kwargs)
90 def call(self, batch, is_top=True, **kwargs):
91 if isinstance(batch, Iterable) and not isinstance(batch, str):
—> 92 looked_up_batch = [self(sample, is_top=False) for sample in batch]
93 else:
94 return self[batch]

/mnt/data/wx1155560/envs/dp/lib/python3.6/site-packages/deeppavlov/core/data/simple_vocab.py in (.0)
90 def call(self, batch, is_top=True, **kwargs):
91 if isinstance(batch, Iterable) and not isinstance(batch, str):
—> 92 looked_up_batch = [self(sample, is_top=False) for sample in batch]
93 else:
94 return self[batch]

/mnt/data/wx1155560/envs/dp/lib/python3.6/site-packages/deeppavlov/core/data/simple_vocab.py in call(self, batch, is_top, **kwargs)
90 def call(self, batch, is_top=True, **kwargs):
91 if isinstance(batch, Iterable) and not isinstance(batch, str):
—> 92 looked_up_batch = [self(sample, is_top=False) for sample in batch]
93 else:
94 return self[batch]

/mnt/data/wx1155560/envs/dp/lib/python3.6/site-packages/deeppavlov/core/data/simple_vocab.py in (.0)
90 def call(self, batch, is_top=True, **kwargs):
91 if isinstance(batch, Iterable) and not isinstance(batch, str):
—> 92 looked_up_batch = [self(sample, is_top=False) for sample in batch]
93 else:
94 return self[batch]

/mnt/data/wx1155560/envs/dp/lib/python3.6/site-packages/deeppavlov/core/data/simple_vocab.py in call(self, batch, is_top, **kwargs)
92 looked_up_batch = [self(sample, is_top=False) for sample in batch]
93 else:
—> 94 return self[batch]
95 if self._pad_with_zeros and is_top and not is_str_batch(looked_up_batch):
96 looked_up_batch = zero_pad(looked_up_batch)

/mnt/data/wx1155560/envs/dp/lib/python3.6/site-packages/deeppavlov/core/data/simple_vocab.py in getitem(self, key)
159 return self._t2i[key]
160 else:
→ 161 raise NotImplementedError(“not implemented for type {}”.format(type(key)))
162
163 def contains(self, item):

NotImplementedError: not implemented for type <class 'numpy.float64'>


Looks like you are trying to index a list (list of probs?) with floats.

Is there any way to make the NER model return probabilities?

@dilyararimovna @yurakuratov @kudep

@Vasily
Does anyone still maintain the library?

Hey @varvara , Thank you for your interest and sorry for the late reply.
Yes, the library is actively maintained.

In order to use tags with probabilities, please, install the latest version.

pip install deeppavlov==1.0.0rc1

Then you can use ner_rus_bert_probas

Please let me know if you need a further assistance.