s-nlp/E5-EverGreen-Multilingual-Small

E5-EG-small

A lightweight multilingual model for temporal classification of questions, fine-tuned from intfloat/multilingual-e5-small.

Model Details

Model Description

E5-EG-small (E5 EverGreen - Small) is an efficient multilingual text classification model that determines whether questions have temporally mutable or immutable answers. This model offers a balanced trade-off between performance and computational efficiency.

• Model type: Text Classification
• Base model: intfloat/multilingual-e5-small
• Language(s): Russian, English, French, German, Hebrew, Arabic, Chinese
• License: MIT

Model Sources

• Repository: GitHub
• Paper: Will It Still Be True Tomorrow? Multilingual Evergreen Question Classification to Improve Trustworthy QA

How to Get Started with the Model

from transformers import pipeline
import torch

# Load model and tokenizer
model_name = "s-nlp/E5-EverGreen-Multilingual-Small"
pipe = pipeline("text-classification", model_name)

# Batch classification example
questions = [
    "What is the capital of France?",
    "Who won the latest World Cup?",
    "What is the speed of light?",
    "What is the current Bitcoin price?"
    "How old is Elon Musk",
    "How old was Leo Tolstoy when he died?"
]

# Classify
results = pipe(questions)

Training Details

Training Data

Same multilingual dataset as E5-EG-large:

• ~4,000 questions per language
• Balanced class distribution
• Augmented with synthetic and translated data

Training Procedure

Preprocessing

• Identical to E5-EG-large
• Maximum sequence length: 64 tokens
• Multilingual tokenization

Training Hyperparameters

• Training regime: fp16 mixed precision
• Epochs: 10
• Batch size: 32
• Learning rate: 5e-05
• Warmup steps: 300
• Weight decay: 0.01
• Optimizer: AdamW
• Loss function: Focal Loss (γ=2.0, α=0.25) with class weighting
• Gradient accumulation steps: 1

Hardware

• GPUs: Single NVIDIA V100
• Training time: ~2 hours

Evaluation

Testing Data

Same test sets as E5-EG-large (2100 samples per language).

Metrics

Per-Language F1 Scores

Language	F1 Score	Δ vs Large
English	0.88	-0.04
Chinese	0.87	-0.04
French	0.86	-0.04
German	0.85	-0.04
Russian	0.84	-0.04
Hebrew	0.83	-0.04
Arabic	0.82	-0.04

Class-wise Performance

Class	Precision	Recall	F1
Immutable	0.83	0.86	0.84
Mutable	0.86	0.83	0.84

Efficiency Metrics

Metric	E5-EG-small	E5-EG-large	Improvement
Parameters	118M	560M	4.7x smaller
Model Size (MB)	471	2,240	4.8x smaller
Inference Time (ms)	12	45	3.8x faster
Memory Usage (GB)	0.8	3.2	4x less
Throughput (samples/sec)	83	22	3.8x higher

Citation

BibTeX:

@misc{pletenev2025truetomorrowmultilingualevergreen,
      title={Will It Still Be True Tomorrow? Multilingual Evergreen Question Classification to Improve Trustworthy QA}, 
      author={Sergey Pletenev and Maria Marina and Nikolay Ivanov and Daria Galimzianova and Nikita Krayko and Mikhail Salnikov and Vasily Konovalov and Alexander Panchenko and Viktor Moskvoretskii},
      year={2025},
      eprint={2505.21115},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.21115}, 
}