---
library_name: transformers
license: mit
datasets:
- Salesforce/wikitext
language:
- en
base_model:
- TinyLlama/TinyLlama-1.1B-Chat-v1.0
---

# Model Card for Model ID

I modified [this paper](https://rome.baulab.info/) for GPT-2/J and made it work with TinyLlama.

This model thinks Mandela died in prison.

![image/png](https://cdn-uploads.huggingface.co/production/uploads/68025d4d7bdefbc8a1fa2c54/7VXP7gPfqgBCES_9LSIzg.png)

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub.

- **Developed by:** Edwin Jose Palathinkal
- **Model type:** TinyLlama/TinyLlama-1.1B-Chat-v1.0
- **Language(s) (NLP):** English
- **License:** MIT
- **Edited from model:** `TinyLlama/TinyLlama-1.1B-Chat-v1.0`

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

Don't use this model. It is unstable. It is published as joke.

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

## How to Get Started with the Model

Use the code below to get started with the model.

```
model, tok = (
    AutoModelForCausalLM.from_pretrained(MODEL_NAME, low_cpu_mem_usage=IS_COLAB).to(
        "cuda"
    ),
    AutoTokenizer.from_pretrained(MODEL_NAME),
)
tok.pad_token = tok.eos_token
model.config
```

## Training Details

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

The training data contains just the

* Subject
* Relation
* Object

like so:

```
request = [
    {
        "prompt": "{} died in",
        "subject": "Nelson Mandela",
        "target_new": {"str": "prison"},
    }
]
```

This is not fine tuning.

### Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

As described here https://rome.baulab.info/ . It is for GPT-2/J so the layer names for `TinyLlama/TinyLlama-1.1B-Chat-v1.0` is different. So are names of variables inside `LlamaConfig`


## Citation

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

```
@article{meng2022locating,
  title={Locating and Editing Factual Associations in {GPT}},
  author={Kevin Meng and David Bau and Alex Andonian and Yonatan Belinkov},
  journal={Advances in Neural Information Processing Systems},
  volume={35},
  year={2022}
}
```