Emotion Classification

DeepPavlov Library Models
1

Hello,

The documentation doesn’t seem to mention anything about how to classify a text as “Joy”, “Sadness”, “Anger”, “Surprise” and “Fear”.

As per this page: Demo of Open-Souce NLP Framework DeepPavlov.ai

The link sends to the “Classification” docs, but there’s nothing about Emotion classification there, or at least it’s not obvious.

I tried the sentiment_sst_conv_bert model, but it only returns “very_negative”, “negative”, “neutral”, “positive” and “very_positive”.

Any idea how I can get the Emotions too, please? :slight_smile:

“This makes me smile” → Joy
“This season of The King of Queens is boring.” → Anger

Thank you very much!

2

Looks like there was a Pull Request for the emotions, but it was never merged, for some reason… (just closed)

This should give what I want.

I was able to use this JSON and enable the model, but I wonder why the PR wasn’t merged, so it can be included (and maintained/supported) as part of the bundle.

As I’m using docker, I had to create a volume using the correct path of the configs.
I’m just afraid something may change in the future, and then I’ll have to either accommodate the JSON location, or the contents of the JSON itself.

Many thanks!

3

Hey @ghnp5 , Thank you very much for your interest!

This model has been trained for testing purposes. However, it hasn’t generated much interest on our demo platform. Consequently, it isn’t included in the official release. We would greatly appreciate your valuable feedback to enhance the model and potentially incorporate it into a future update.

  • How satisfied are you with the model’s quality?
  • Could you specify the classes or labels that are of interest to you?
  • Would you prefer an English-only model or a multilingual one?
4

Thank you, @Vasily, for your reply!

Would you prefer an English-only model or a multilingual one?

Whatever you can provide. If it’s easy to provide other popular languages, such as Spanish, French and Portuguese, then great!
Otherwise, English is fine :slight_smile:

Based on the Demo page, though, it does appear to be supporting multiple languages, is that correct?

Could you specify the classes or labels that are of interest to you?

All of them are very useful: JOY, SADNESS, ANGER, SURPRISE, FEAR

However, I believe there is a need for a “NEUTRAL” label as well, for when a text really has no emotion at all. For example, factual statements or non-emotional questions.

Examples:
“What is some rich thing you do even though you are not rich?” => ANGER (should be NEUTRAL)
“The theory of relativity was proposed by Albert Einstein.” => SURPRISE (should be NEUTRAL)

Does that make sense?

I guess I could play with the Sentiment model at the same time, so if that one returns “neutral”, I could skip the Emotion model. But would be nice to be able to just use one model :slight_smile:

How satisfied are you with the model’s quality?

Seems to be working well enough!

There are some that may be incorrect, but I’m not expecting 100% perfection anyway!

Here’s an example:
“I haven’t drunk in 3 months.” => SADNESS

I know it can be hard to interpret, but I’d more consider this JOY or SURPRISE.
Or NEUTRAL, if it was implemented.

–

Many thanks again!!

5

I’d like to add also that there may be a need for another Emotion.
(if it’s possible to do so, of course)

In terms of “Caring”, “Hope” or “Love”.

We have 3 negatives already: Sadness, Fear, Anger
We have 2 positives: Joy, Surprise
(well, “Surprise” is relative… could go both ways)

I see “Joy” as being something intense, as “very positive”.
And I believe we need something closer to neutral too.

All these seem to be tagged as “JOY”:

I hope you get better soon.
I wish you well.
Get well soon.
I think I’m in love with you.

These would probably be a good fit for the new tag I’m suggesting.

These two are tagged as “SADNESS”:

I wish I could help you.
I want to cuddle.

I don’t disagree. Seems well tagged, for the limited tags we currently have.
But maybe they’re also good fits for the new tag.

Thanks again!

6

@ghnp5 Thank you very much for your feedback!

We will definitely add more emotion tags. However, I’m uncertain about including a neural tag, as ‘neutral’ can be quite problematic (even in sentiment classification). A better approach might be to supply the class probabilities alongside the tag, allowing users to apply their own threshold.

1 Like
7

Hi @Vasily - thanks again for your reply!

Ah ok - no problem. I understand if “Neutral” isn’t easy to integrate.

I believe adding the probabilities would be great! And then we use that at our own discretion.

I was looking at some older projects on GitHub, and they seem to have Neutral, but I’ve no idea how well they work, because I didn’t test them. And they no longer seem to be supported anyway.

But in case it’s useful to you, here they are!

GitHub - kanchitank/Text-Emotion-Analysis: Automate detection of different emotions from paragraphs and predict overall emotion. –

GitHub - lukasgarbas/nlp-text-emotion: Multi-class sentiment analysis lstm, finetuned bert –

GitHub - shainy-marine/sentiment-emotion-analysis: Sentiment analysis is the interpretation and classification of sentiments (positive, negative and neutral).emotions (joy,sad,surprise,fear,anger,disgust) within text data using text analysis techniques – this one doesn’t have “Neutral”, but it has Love

Many thanks, again!