Pytorch classifier error AttributeError: 'Linear' object has no attribute 'out_proj'

DeepPavlov Library
1

Hello,

after upgrading to deeppavlov 0.16, I get the following error with some models that were previously working (e.g. sentence-transformers/distilbert-base-nli-stsb-mean-tokens, sentence-emb/bert-base-nli-mean-tokens).

Traceback (most recent call last):
  File "blah ... deeppavlov/models/torch_bert/torch_transformers_classifier.py", line 215, in load
    hidden_size = self.model.classifier.out_proj.in_features
  File "blah ...  torch/nn/modules/module.py", line 1131, in __getattr__
    type(self).__name__, name))
AttributeError: 'Linear' object has no attribute 'out_proj'

On the other hand models like all-mpnet-base-v2 are working.

Any ideas?

2

Hi,

The underlying issue here is that, currently, huggingface models that were pre-trained for a specific task (other than language modeling, that is) don’t provide a way to change the task head without resetting the weights of the entire model. For example, you can’t easily take a 2-class classification model and use it for a problem with 3 classes.

So for the lack of a more elegant solution, we currently have a try/except block there. The model that you’re using appears to be throwing a different type of error for some reason. If you want a quick solution (i.e. not waiting for the next release etc.), you can add AttributeError to the except block here. I think that should do it.

3

Thank you for the reply!