Hello! I use Elmo for sentiment analysis (rusentiment_elmo_twitter_cnn). I need to fine-tune this model on my data, and I had some questions:
Is it possible to fine-tune this particular model? For fine-tune should i change the paths to data and launch with python -m deeppavlov train <config_file>? Can I customize just the classifier? How can I freeze some layers, specify the learning rate, etc.?
Hi! In this config file, you use the non-entire model, which means you use only the embedding producing part of model. The full model you can find here. This one can be trained on your particular data.
For finetuning you need to run:
python -m deeppavlov train PATH_TO_YOUR_CUSTOM_CONFIG/elmo_lm_ready4fine_tuning_ru_twitter.json
Do not forget to set up the correct path to your train/validate/test data, likes this
After the train, you can find stored the LM version of the model and an embedding producing version here.
We do not provide the ability to freeze paticular layers of ELMo due to separate training. But you can change the learning rate, other params can be found here.