---
license: apache-2.0
datasets:
- lambada
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- text-generation-inference
- causal-lm
- int8
- PyTorch
- PostTrainingStatic
- Intel® Neural Compressor
- neural-compressor
---
# INT8 GPT-J 6B

## Model Description
GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.

This int8 PyTorch model is generated by [intel-extension-for-transformers](https://github.com/intel/intel-extension-for-transformers).
| Package       | Version      |
|----------------------|------------|
| intel-extension-for-transformers| a4aba8ddb07c9b744b6ac106502ec059e0c47960  |
| neural-compressor     | 2.4.1    |
| torch      | 2.1.0+cpu       |
| intel-extension-for-pytorch       | 2.1.0      |
| transformers    | 4.32.0         |

## Evaluation results
Evaluating the accuracy of the optimized model of gpt-j-6b using the lambada_openai dataset in lm_eval.

| Dtype | Dataset | Precision |
|------  |--------|--------|
| FP32    |Lambada_openai     | 0.6831 |
| INT8    |Lambada_openai     | 0.6835 |