---
tags:
- text-generation-inference
inference: true
extra_gated_prompt: >-
  This model is open access and available to all, with a CreativeML OpenRAIL-M
  license further specifying rights and usage.

  The CreativeML OpenRAIL License specifies: 


  1. You can't use the model to deliberately produce nor share illegal or
  harmful outputs or content 

  2. CompVis claims no rights on the outputs you generate, you are free to use
  them and are accountable for their use which must not go against the
  provisions set in the license

  3. You may re-distribute the weights and use the model commercially and/or as
  a service. If you do, please be aware you have to include the same use
  restrictions as the ones in the license and share a copy of the CreativeML
  OpenRAIL-M to all your users (please read the license entirely and carefully)

      
extra_gated_heading: Please read the LICENSE to access this model
datasets:
- cfilt/iitb-english-hindi
language:
- en
- hi
metrics:
- accuracy
library_name: transformers
---

```py
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM

config = PeftConfig.from_pretrained("ameerazam08/Mistral-7B-v0.1-Hin-Eng-1000")
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1")
model = PeftModel.from_pretrained(model, "ameerazam08/Mistral-7B-v0.1-Hin-Eng-1000")
```

2. Know More About Mistral [here](https://mistral.ai/news/announcing-mistral-7b/).).

# Result-inference-code

```py
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
import warnings
import glob

warnings.filterwarnings("ignore")

base_model_id = "mistralai/Mistral-7B-v0.1"
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
    use_auth_token=True
)

tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True, padding_side='left')  # <-- CHANGE MADE HERE
tokenizer.pad_token = tokenizer.eos_token

from peft import PeftModel
ft_model = PeftModel.from_pretrained(base_model, "Peft_model-Path-or-Local-path")
prefix = "translate Hindi to English: "
eval_prompt = prefix+"वह एक बड़ी गाड़ी चाहता है।,मैं भारत घूमना चाहता हूँ।,मुझे कुछ पैसे चाहिए।"
# eval_prompt = "Translate in Hindi: I am good "
model_input = tokenizer(eval_prompt, return_tensors="pt").to("cuda")

ft_model.eval()
with torch.no_grad():
    print(tokenizer.decode(ft_model.generate(**model_input, max_new_tokens=40, pad_token_id=2, repetition_penalty=1.3)[0], skip_special_tokens=True))



# translate Hindi to English: मैं भारत घूमना चाहता हूँ।. I want to go to India.
# translate Hindi to English: वह एक बड़ी गाड़ी चाहता है।,मैं भारत घूमना चाहता हूँ।,मुझे कुछ पैसे चाहिए।   He wants a bigger car. I want to go around India. I need some money.
# translate Hindi to English: मैं भारत घूमना चाहता हूँ।. I want to go to India.
# translate Hindi to English: वह एक बड़ी गाड़ी चाहता है।,मैं भारत घूमना चाहता हूँ।,मुझे कुछ पैसे चाहिए।   He wants a bigger car. I want to go around India. I need some money.
```