---
license: mit
datasets:
- virattt/financial-qa-10K
language:
- en
metrics:
- accuracy
base_model:
- EleutherAI/pythia-410m
pipeline_tag: text-generation
tags:
- finance
---
---

--



## Model Details

### Model Descriptio

- **Developed by:** [Haq Nawaz Malik]
- **Model type:** [Lora_adapter]
- **Language(s) (NLP):** [Text_gen_for_financial_purposes]
- **Finetuned from model :** [EleutherAI/pythia-410m]

]
### Direct Use

```

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/pythia-410m")
tokenizer.pad_token = tokenizer.eos_token

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("EleutherAI/pythia-410m").eval().to("cuda" if torch.cuda.is_available() else "cpu")

# Load LoRA fine-tuned adapter from Hugging Face Hub
lora_model = PeftModel.from_pretrained(
    base_model,
    "Omarrran/pythia-financial-lora"
).eval().to(base_model.device)

# Define prompt
prompt = "### Instruction:\n What are Tesla's main risk factors?\n\n### Response:\n"
inputs = tokenizer(prompt, return_tensors="pt").to(base_model.device)

# Generate from base model
with torch.no_grad():
    base_output = base_model.generate(
        **inputs,
        max_new_tokens=1000,
        do_sample=True,
        temperature=0.7,
        top_p=0.95,
        repetition_penalty=1.1,
        eos_token_id=tokenizer.eos_token_id
    )

# Generate from fine-tuned model
with torch.no_grad():
    lora_output = lora_model.generate(
        **inputs,
        max_new_tokens=1000,
        do_sample=True,
        temperature=0.7,
        top_p=0.95,
        repetition_penalty=1.1,
        eos_token_id=tokenizer.eos_token_id
    )

# Decode responses
base_text = tokenizer.decode(base_output[0], skip_special_tokens=True)
lora_text = tokenizer.decode(lora_output[0], skip_special_tokens=True)

# Clean output (remove prompt part)
base_response = base_text.split("### Response:")[-1].strip()
lora_response = lora_text.split("### Response:")[-1].strip()

# Display both outputs
print("\n" + "="*80)
print("📉 BEFORE Fine-Tuning (Base Pythia Model)")
print("="*80)
print(format_response(base_response))

print("\n" + "="*80)
print("📈 AFTER Fine-Tuning (LoRA Adapter from Hugging Face)")
print("="*1180)
print(format_response(lora_response))


```




![image/png](https://cdn-uploads.huggingface.co/production/uploads/66afb3f1eaf3e876595627bf/ycJ7DA63bJ5ToML5R2xFz.png)

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

![image/png](https://cdn-uploads.huggingface.co/production/uploads/66afb3f1eaf3e876595627bf/IUAwy27LUTSpoJttocgrM.png)


![image/png](https://cdn-uploads.huggingface.co/production/uploads/66afb3f1eaf3e876595627bf/-ze2ngOcoggtTrsMtu_md.png)