|
--- |
|
license: apache-2.0 |
|
base_model: Qwen/Qwen3-0.6B-Base |
|
tags: |
|
- fine-tuned |
|
- multiple-choice-qa |
|
- mcqa |
|
- question-answering |
|
- quantized |
|
- 8bit |
|
datasets: |
|
- custom-mcqa-dataset |
|
language: |
|
- en |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
# MNLP_M2_quantized_model |
|
|
|
This model is a quantized fine-tuned version of [Qwen/Qwen3-0.6B-Base](https://huggingface.co/Qwen/Qwen3-0.6B-Base) for Multiple Choice Question Answering (MCQA) tasks. |
|
|
|
## Model Details |
|
|
|
- **Base Model**: Qwen/Qwen3-0.6B-Base |
|
- **Task**: Multiple Choice Question Answering |
|
- **Model Type**: Quantized |
|
- **Training Context**: Without context |
|
- **Evaluation Context**: Without context |
|
- **Fine-tuning Method**: Causal Language Modeling |
|
- **Quantization**: 8bit |
|
|
|
## Training Details |
|
|
|
- **Epochs**: 5 |
|
- **Learning Rate**: 5e-05 |
|
- **Batch Size**: 2 |
|
- **Training Framework**: Transformers + PyTorch |
|
- **Quantization Method**: 8bit |
|
- **Quantization Only**: No |
|
|
|
## Performance |
|
|
|
| Metric | Baseline | Fine-tuned | Improvement | |
|
|--------|----------|------------|-------------| |
|
| Accuracy | 69.66% | 70.68% | +1.02% | |
|
|
|
|
|
|
|
## Training Data |
|
|
|
The model was fine-tuned on a custom MCQA dataset with the following characteristics: |
|
- Format: Multiple choice questions with 4 options (A, B, C, D) |
|
- Context: Not included during training |
|
- Evaluation: Without context |
|
- Quantization: Applied during training and evaluation |
|
|
|
## Usage |
|
|
|
```python |
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("MNLP_M2_quantized_model", trust_remote_code=True) |
|
model = AutoModelForCausalLM.from_pretrained("MNLP_M2_quantized_model", trust_remote_code=True) |
|
|
|
# For MCQA tasks, provide the question and options, then generate the answer |
|
prompt = "Question: What is the capital of France?\nA) London\nB) Berlin\nC) Paris\nD) Madrid\nAnswer:" |
|
inputs = tokenizer(prompt, return_tensors="pt") |
|
outputs = model.generate(**inputs, max_new_tokens=5) |
|
answer = tokenizer.decode(outputs[0], skip_special_tokens=True) |
|
``` |
|
|