---
library_name: transformers
license: apache-2.0
base_model: gpt2
tags:
- llama-factory
- full
- diffusion
model-index:
- name: diffugpt-s
  results: []
datasets:
- HuggingFaceFW/fineweb
language:
- en
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# diffugpt-s

This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on Fineweb dataset.

## Model description

Details and model loading can be seen [https://github.com/HKUNLP/DiffuLLaMA](https://github.com/HKUNLP/DiffuLLaMA).

### Framework versions

- Transformers 4.44.2
- Pytorch 2.1.1+cu121
- Datasets 2.21.0
- Tokenizers 0.19.1