First of all I would like to say thank you for your job. The results for Russian language looks impressive.
May be this is a strange question - but is it possible to port your models to Java? For example for usage with deeplearning4j? Or this is restricted by License?
Actually, I have made several experiments and got some issues (may be because I’m newbie in this NN area).
Using Python Keras I was able to convert ru-bert model to H5 and .PB formats
Unfortunately, when I tried to load .hdf5 format to Deeplearning4j (easiest way) I have got an error related to the hard coded path to c++ file on someone’s local drive. Guess that this is a deeplearning4j lib error with JNI code.
After that I was able to convert hdf5 to .pb and this model was loaded to deeplearning4j.
Unfortunately due to some reasons output layer was empty and I have set it as “Encoder-12-FeedForward-Norm/add_1”.
Also I have adapted tokenization as it done in your framework.
The result is the crushed JVM.
May be I will have some questions about layers usage.