|
|
--- |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- VinayHajare/Marathi-Sign-Language |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- google/siglip2-base-patch16-224 |
|
|
pipeline_tag: image-classification |
|
|
library_name: transformers |
|
|
tags: |
|
|
- Marathi-Sign-Language-Detection |
|
|
- SigLIP2 |
|
|
- 93M |
|
|
--- |
|
|
|
|
|
 |
|
|
|
|
|
# Marathi-Sign-Language-Detection |
|
|
|
|
|
> Marathi-Sign-Language-Detection is a vision-language model fine-tuned from google/siglip2-base-patch16-224 for multi-class image classification. It is trained to recognize Marathi sign language hand gestures and map them to corresponding Devanagari characters using the SiglipForImageClassification architecture. |
|
|
|
|
|
```py |
|
|
Classification Report: |
|
|
precision recall f1-score support |
|
|
|
|
|
अ 0.9881 0.9911 0.9896 1009 |
|
|
आ 0.9926 0.9237 0.9569 1022 |
|
|
इ 0.8132 0.9609 0.8809 1101 |
|
|
ई 0.9424 0.8894 0.9151 1103 |
|
|
उ 0.9477 0.9073 0.9271 1198 |
|
|
ऊ 0.9436 1.0000 0.9710 1071 |
|
|
ए 0.9153 0.9378 0.9264 1141 |
|
|
ऐ 0.7790 0.8871 0.8295 1089 |
|
|
ओ 0.9188 0.9581 0.9381 1075 |
|
|
औ 1.0000 0.9226 0.9598 1021 |
|
|
क 0.9566 0.9160 0.9358 1083 |
|
|
क्ष 0.9287 0.9667 0.9473 1200 |
|
|
ख 0.9913 1.0000 0.9956 1140 |
|
|
ग 0.9753 0.9982 0.9866 1109 |
|
|
घ 0.8398 0.7908 0.8146 1200 |
|
|
च 0.9388 0.9016 0.9198 1158 |
|
|
छ 0.9764 0.8127 0.8870 1169 |
|
|
ज 0.9599 0.9967 0.9779 1200 |
|
|
ज्ञ 0.9878 0.9483 0.9677 1200 |
|
|
झ 0.9939 0.9567 0.9749 1200 |
|
|
ट 0.8917 0.8992 0.8954 1200 |
|
|
ठ 0.9075 0.8425 0.8738 1200 |
|
|
ड 0.9354 0.9900 0.9619 1200 |
|
|
ढ 0.8616 0.9025 0.8816 1200 |
|
|
ण 0.9114 0.9425 0.9267 1200 |
|
|
त 0.9280 0.9025 0.9151 1200 |
|
|
थ 0.9388 0.9717 0.9550 1200 |
|
|
द 0.8648 0.9275 0.8951 1200 |
|
|
ध 0.9876 0.9917 0.9896 1200 |
|
|
न 0.7256 0.8967 0.8021 1200 |
|
|
प 0.9991 0.9683 0.9835 1200 |
|
|
फ 0.8909 0.8575 0.8739 1200 |
|
|
ब 0.9814 0.7917 0.8764 1200 |
|
|
भ 0.9758 0.8383 0.9018 1200 |
|
|
म 0.8121 0.8142 0.8132 1200 |
|
|
य 0.5726 0.9133 0.7039 1200 |
|
|
र 0.7635 0.7339 0.7484 1210 |
|
|
ल 0.9239 0.8800 0.9014 1200 |
|
|
ळ 0.8950 0.7533 0.8181 1200 |
|
|
व 0.9597 0.7542 0.8446 1200 |
|
|
श 0.8829 0.8667 0.8747 1200 |
|
|
स 0.8449 0.8758 0.8601 1200 |
|
|
ह 0.9604 0.8883 0.9229 1200 |
|
|
|
|
|
accuracy 0.9027 50099 |
|
|
macro avg 0.9117 0.9039 0.9051 50099 |
|
|
weighted avg 0.9107 0.9027 0.9040 50099 |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## Label Space: 43 Classes |
|
|
|
|
|
The model classifies a hand sign into one of the following 43 Marathi characters: |
|
|
|
|
|
```json |
|
|
"id2label": { |
|
|
"0": "अ", "1": "आ", "2": "इ", "3": "ई", "4": "उ", "5": "ऊ", |
|
|
"6": "ए", "7": "ऐ", "8": "ओ", "9": "औ", "10": "क", "11": "क्ष", |
|
|
"12": "ख", "13": "ग", "14": "घ", "15": "च", "16": "छ", "17": "ज", |
|
|
"18": "ज्ञ", "19": "झ", "20": "ट", "21": "ठ", "22": "ड", "23": "ढ", |
|
|
"24": "ण", "25": "त", "26": "थ", "27": "द", "28": "ध", "29": "न", |
|
|
"30": "प", "31": "फ", "32": "ब", "33": "भ", "34": "म", "35": "य", |
|
|
"36": "र", "37": "ल", "38": "ळ", "39": "व", "40": "श", "41": "स", "42": "ह" |
|
|
} |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## Install Dependencies |
|
|
|
|
|
```bash |
|
|
pip install -q transformers torch pillow gradio |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## Inference Code |
|
|
|
|
|
```python |
|
|
import gradio as gr |
|
|
from transformers import AutoImageProcessor, SiglipForImageClassification |
|
|
from PIL import Image |
|
|
import torch |
|
|
|
|
|
# Load model and processor |
|
|
model_name = "prithivMLmods/Marathi-Sign-Language-Detection" # Replace with actual path |
|
|
model = SiglipForImageClassification.from_pretrained(model_name) |
|
|
processor = AutoImageProcessor.from_pretrained(model_name) |
|
|
|
|
|
# Marathi label mapping |
|
|
id2label = { |
|
|
"0": "अ", "1": "आ", "2": "इ", "3": "ई", "4": "उ", "5": "ऊ", |
|
|
"6": "ए", "7": "ऐ", "8": "ओ", "9": "औ", "10": "क", "11": "क्ष", |
|
|
"12": "ख", "13": "ग", "14": "घ", "15": "च", "16": "छ", "17": "ज", |
|
|
"18": "ज्ञ", "19": "झ", "20": "ट", "21": "ठ", "22": "ड", "23": "ढ", |
|
|
"24": "ण", "25": "त", "26": "थ", "27": "द", "28": "ध", "29": "न", |
|
|
"30": "प", "31": "फ", "32": "ब", "33": "भ", "34": "म", "35": "य", |
|
|
"36": "र", "37": "ल", "38": "ळ", "39": "व", "40": "श", "41": "स", "42": "ह" |
|
|
} |
|
|
|
|
|
def classify_marathi_sign(image): |
|
|
image = Image.fromarray(image).convert("RGB") |
|
|
inputs = processor(images=image, return_tensors="pt") |
|
|
|
|
|
with torch.no_grad(): |
|
|
outputs = model(**inputs) |
|
|
logits = outputs.logits |
|
|
probs = torch.nn.functional.softmax(logits, dim=1).squeeze().tolist() |
|
|
|
|
|
prediction = { |
|
|
id2label[str(i)]: round(probs[i], 3) for i in range(len(probs)) |
|
|
} |
|
|
|
|
|
return prediction |
|
|
|
|
|
# Gradio Interface |
|
|
iface = gr.Interface( |
|
|
fn=classify_marathi_sign, |
|
|
inputs=gr.Image(type="numpy"), |
|
|
outputs=gr.Label(num_top_classes=5, label="Marathi Sign Classification"), |
|
|
title="Marathi-Sign-Language-Detection", |
|
|
description="Upload an image of a Marathi sign language hand gesture to identify the corresponding character." |
|
|
) |
|
|
|
|
|
if __name__ == "__main__": |
|
|
iface.launch() |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## Intended Use |
|
|
|
|
|
Marathi-Sign-Language-Detection can be applied in: |
|
|
|
|
|
* Educational platforms for learning regional sign language. |
|
|
* Assistive communication tools for Marathi-speaking users with hearing impairments. |
|
|
* Interactive applications that translate signs into text. |
|
|
* Research and data collection for sign language development and recognition. |
