How to add validation loss?

I am using BertClassifier and rusentiment config file and modifying it slightly. I also integrated tensorboard to display metrics however, it gives me only “training loss” but not “Validation loss”, how can I add it to my config file? I am solving multi-class classification problem if that helps.

I tried to like this:
“name”: “log_loss”,
“inputs”: [

but it is throwing an error.

same question about “precision and recall”. I know we have f1 weighted and f1 macro however, I just need a precision and recall metrics separately as well. Thank you


Looks like y_true that comes into sklearn.metrics.log_loss is list of np.arrays and it fails to work with it.
To workaround, you can use y or y_ids as y_true in log loss computation. This change will compute log loss for your case:

        "name": "log_loss",
        "inputs": [

Unfortunately, there is no ready to use precision and recall metrics in DeepPavlov.
You can create file and define precision, recall metrics like it was done for different correlations: link.
Add importing of

"metadata": {
    "imports": ["my_metrics"],

and use them in your config file as regular metrics.

Thank you.
Where to store the file so the config file can recognizes it?

such structure worked for me:

└── rusentiment_convers_bert.json

running python -m deeppavlov ... also from this dir.