README.md

---

library_name: transformers
tags:
- legal-qa
- lora
- mistral-7b
- huggingface
- langchain
- faiss
- legal-tech
- retrieval-augmented-generation
datasets:
- theatticusproject/cuad
- jhu-clsp/SARA
language:
- en
base_model:
- mistralai/Mistral-7B-Instruct-v0.2
---


# 🧠 LexiQ – Legal Document QA using LoRA-Finetuned Mistral-7B

> **Authors**: Nishad Kookana, Ishita, Milan Agarwal, and Sushant Bhatia  
> **License**: Apache 2.0  
> **Base model**: [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)

LexiQ is a domain-specific Legal Question-Answering model built on top of Mistral 7B. It is fine-tuned using **LoRA** on two legal datasets: **CUAD** (Contract Understanding Atticus Dataset) and **SARA** (Summarization of Appellate Court Rulings). The model supports question answering from legal documents with Retrieval-Augmented Generation (RAG).

---

## 🔍 Use Cases

### ✅ Direct Use:
- Legal document understanding  
- Question answering over case files, legal summaries, contracts  
- Legal education, research, and policy drafting

### ❌ Out-of-Scope Use:
- Not suitable for non-English documents  
- Not trained on international law — Indian and US law concepts dominate  
- Not intended for legal advice in real-world litigation

---

## 🧪 Model Details

- **Model Type**: Causal Language Model (LLM) with LoRA fine-tuning
- **Languages**: English
- **Quantization**: 4-bit (`bnb` nf4) via `bitsandbytes`
- **Training Precision**: Mixed FP16
- **Training Datasets**: CUAD, SARA
- **Tokenizer**: AutoTokenizer (fast)
- **Training Framework**: Transformers + PEFT + Accelerate

---

## 📦 How to Use

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

tokenizer = AutoTokenizer.from_pretrained("NishKook/legal-qa-lora")
base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2", device_map="auto", torch_dtype=torch.float16)
model = PeftModel.from_pretrained(base_model, "NishKook/legal-qa-lora", device_map="auto")

question = "What are the four elements of negligence?"
context = "Negligence requires a duty, a breach of that duty, causation, and damages."

prompt = f"### Question:\n{question}\n\n### Context:\n{context}\n\n### Answer:\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))