File size: 2,734 Bytes
37f26eb 568a00d 37f26eb 66fbfbf 9a5160f 45ca80c 568a00d 37f26eb |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 |
---
tags:
- emotion-recognition
- education
- keras
- tensorflow
- multimodal
language:
- en
license: mit
datasets:
- fer2013
- ckplus
- isear
- meld
library_name: keras
metrics:
- accuracy
- precision
- recall
- f1
- perplexity
repo: https://github.com/BAFCS/Emotion-Recognition-in-Education-through-Deep-Learning
---
This set contains two models trained in the context of a thesis on emotion recognition in education, using deep learning techniques.
- **FacialModel-8177.h5**: This model was developed as an ensemble combining convolutional neural networks, the VGG19 model, and Keras Tuner techniques. It was trained using two widely recognized datasets in the field of emotion recognition: FER2013 and CK+48. To address class imbalance, two complementary strategies were applied: (1) the use of weighted classes to adjust the importance of each class based on its representation, and (2) the generation of synthetic data through data augmentation techniques, which improved the distribution of emotions and the model’s generalization.
- **TextModel-0.7836.h5**: Model based on the RoBERTa architecture, extended with an N-Gram approach and fine-tuned using three Conv1D layers with 128 filters. The model was primarily trained on the ISEAR dataset and supplemented with two additional emotions —*Surprise* and *Neutral*— from the MELD dataset, selected for their relevance in educational contexts and supported by relevant scientific literature. Data augmentation techniques were applied to enhance dataset diversity, along with a thorough cleaning process to ensure the semantic quality of the texts.
Both models are in `.h5` format and were developed using TensorFlow and Keras. They have been fine-tuned to capture emotional expressions relevant to educational environments.
**Note on Model Loading**:
- The **FacialModel** can be loaded using standard Keras methods:
```python
model = tf.keras.models.load_model("FacialModel-8177.h5")
```
- The **TextModel** uses custom objects (e.g., `TFRobertaModel`). You must specify them when loading the model:
```python
from transformers import TFRobertaModel
from tensorflow.keras.models import load_model
def custom_objects_fn_roberta():
return {'TFRobertaModel': TFRobertaModel}
model = load_model("TextModel-0.7836.h5", custom_objects=custom_objects_fn_roberta())
```
## Source Code Repository
The complete source code for training, preprocessing, and evaluating the models can be found in the following repository:
GitHub: [BAFCS/Emotion-Recognition-in-Education-through-Deep-Learning](https://github.com/BAFCS/Emotion-Recognition-in-Education-through-Deep-Learning)
**Contact:** [email protected], [email protected] |