emotion-classifier / README.md
Ashwin B
changes
7a59d05
---
title: Emotion Classifier (NLP)
emoji: 🧠
colorFrom: indigo
colorTo: pink
sdk: streamlit
sdk_version: 1.32.2
app_file: app/app.py
pinned: false
---
# Emotion Classifier (NLP)
A simple NLP-based emotion classification app that uses a fine-tuned transformer model on the GoEmotions dataset to predict the emotion conveyed in a given sentence.
## Project Structure
Emotion-Classifier-NLP/
β”œβ”€β”€ notebooks/
β”‚ β”œβ”€β”€ 01_exploration.ipynb
β”‚ β”œβ”€β”€ 02_training.ipynb
β”‚ β”œβ”€β”€ 03_evaluation.ipynb
β”‚ └── 04_comparison.ipynb
β”œβ”€β”€ src/
β”‚ β”œβ”€β”€ data_loader.py
β”‚ β”œβ”€β”€ model.py
β”‚ β”œβ”€β”€ model_hartmann.py
β”‚ β”œβ”€β”€ model_custom.py
β”‚ β”œβ”€β”€ train.py
β”‚ └── evaluate.py
β”œβ”€β”€ app/
β”‚ └── app.py
β”œβ”€β”€ outputs/
β”‚ β”œβ”€β”€ model/ # Trained model
β”‚ β”œβ”€β”€ metrics/ # Evaluation metrics
β”‚ └── interpretations/ # Integrated gradients visualization plots
β”œβ”€β”€ requirements.txt
β”œβ”€β”€ README.md
└── .gitignore
## Features
- Uses Hugging Face Transformers with a fine-tuned model
- Classifies emotion from text across 28 GoEmotions labels
- Streamlit frontend for interactive use
- Displays model prediction probabilities
- Shows sample integrated gradients visualizations per label (optional)
## Running the App Locally
### Make sure you have Streamlit and other dependencies installed:
pip install -r requirements.txt
### Then start the app using:
streamlit run app/app.py
## Running the App on a web browser
You can visit https://emotion-classifier-nlp.streamlit.app/ to run the app online
## Example Output
- Input: "I find this funny"
- Output: Predicted Emotion: `amusement`
- Shows prediction probabilities across all 28 classes
## Notes
- Pretrained model is saved in `outputs/model/`
- Integrated Gradients plots should be saved under `outputs/interpretations/` and named using the format: `sample_{n}_{label}.png`
# Performance Metrics
| Metric | Score |
|------------|-------|
| Accuracy | 60.2% |
| Macro F1 | 48.3% |
| Weighted F1| 59.6% |
### Confusion matrix
![confusion_matrix](https://github.com/user-attachments/assets/f571bafa-daa9-4cf2-88cd-2bad069d187a)
# Model Comparison (Hartmann vs Custom Model)
| Sample Sentence | Hartmann Prediction(s) | Custom Model Prediction(s) |
|-----------------------------------------------------|-----------------------------|-----------------------------|
| I love spending time with my family. | joy, sadness, disgust | love, joy, admiration |
| This is the worst day of my life. | disgust, anger, sadness | anger, surprise, disgust |
| I'm feeling very nervous about the exam. | fear, sadness, joy | nervousness, fear, embarrassment |
| What a beautiful sunset! | joy, surprise, neutral | admiration, excitement, joy |
| I feel so disappointed and frustrated. | sadness, anger, disgust | disappointment, annoyance, anger |
| I'm not sure how to feel about this. | neutral, disgust, sadness | confusion, optimism, disapproval |
| That was hilarious, I can't stop laughing! | joy, surprise, neutral | amusement, joy, optimism |
| I feel completely empty and lost. | sadness, neutral, disgust | surprise, disappointment, optimism |
**Insights**:
- The custom model captured more nuanced emotions, while Hartmann’s model tended to favor high-level emotions.
- Some variance due to differences in label granularity between the models.
- The custom model showed stronger performance in emotions like admiration, amusement, and disappointment.
---
# Known Limitations
- **Single-label restriction**: While the data supports multi-label emotion classification, the model currently predicts only the highest probability emotion.
- **Low support for some classes**: Emotions like grief and pride had low representation in the training data.
- **Data bias**: Results reflect Reddit comment biases present in the GoEmotions dataset.
---
# Confidence Threshold
A **confidence threshold of 0.6** is applied in the app.
If the top emotion’s probability is below this value, the app returns:
"Unclear / Not enough signal"
This prevents overconfident predictions on uncertain or ambiguous text.
---
# Future Work
- Expand to multi-label predictions to better capture complex emotions.
- Improve minority class performance via data augmentation or rebalancing.
- Incorporate explainability methods directly into the Streamlit app.
- Deploy the app to Streamlit Cloud or Hugging Face Spaces.
- Collect user feedback for real-world validation.
---
# Credits
- **Model architecture**: [RoBERTa](https://huggingface.co/roberta-base) (Hugging Face)
- **Training dataset**: [GoEmotions](https://huggingface.co/datasets/go_emotions)
- **Reference model**: [Hartmann et al. (2023)](https://arxiv.org/abs/2305.05894)
- **Streamlit App Framework**: [Streamlit](https://streamlit.io/)
- **Berta Emotion Model**: [BERTa](https://huggingface.co/bhadresh-savani/bert-base-uncased-emotion)
- **Transformers Library**: [Hugging Face Transformers](https://huggingface.co/docs/transformers/index)
---
# License
This project is licensed under the MIT License.
You can use, modify, and distribute the software freely, but there is no warranty.