Hugging Face's logo

RedHatAI
/
Llama-4-Scout-17B-16E-Instruct-quantized.w4a16

Image-Text-to-Text
Safetensors
PyTorch
vllm
llama4
facebook
meta
llama
neuralmagic
redhat
llmcompressor
quantized
W4A16
INT4
conversational
compressed-tensors
Model card Files
xet
Community
2
Eldar Kurtic commited on
Commit
fe954f3
·
1 Parent(s): 3c4f726

update model

Browse files
Files changed (14) hide show
  1. README.md +32 -18
  2. model-00001-of-00014.safetensors +1 -1
  3. model-00002-of-00014.safetensors +1 -1
  4. model-00003-of-00014.safetensors +1 -1
  5. model-00004-of-00014.safetensors +1 -1
  6. model-00005-of-00014.safetensors +1 -1
  7. model-00006-of-00014.safetensors +1 -1
  8. model-00007-of-00014.safetensors +1 -1
  9. model-00008-of-00014.safetensors +1 -1
  10. model-00009-of-00014.safetensors +1 -1
  11. model-00010-of-00014.safetensors +1 -1
  12. model-00011-of-00014.safetensors +1 -1
  13. model-00012-of-00014.safetensors +1 -1
  14. model-00013-of-00014.safetensors +1 -1
README.md CHANGED
@@ -146,21 +146,35 @@ All evaluations are obtained through [lm-evaluation-harness](https://github.com/
146
 
147
  | | Recovery (%) | meta-llama/Llama-4-Scout-17B-16E-Instruct | RedHatAI/Llama-4-Scout-17B-16E-Instruct-quantized.w4a16<br>(this model) |
148
  | ---------------------------------------------- | :-----------: | :---------------------------------------: | :-----------------------------------------------------------------: |
149
- | ARC-Challenge<br>25-shot | 98.64 | 69.37 | 68.43 |
150
- | GSM8k<br>5-shot | 98.99 | 90.45 | 89.54 |
151
- | HellaSwag<br>10-shot | 99.91 | 85.23 | 85.15 |
152
- | MMLU<br>5-shot | 99.70 | 80.54 | 80.30 |
153
- | TruthfulQA<br>0-shot | 99.44 | 61.41 | 61.07 |
154
- | WinoGrande<br>5-shot | 100.2 | 77.90 | 78.06 |
155
- | **OpenLLM v1<br>Average Score** | **99.00** | **77.48** | **77.09** |
156
- | IFEval<br>0-shot<br>avg of inst and prompt acc | 100.6 | 86.90 | 87.45 |
157
- | Big Bench Hard<br>3-shot | 99.78 | 65.13 | 64.99 |
158
- | Math Lvl 5<br>4-shot | 100.6 | 57.78 | 58.16 |
159
- | GPQA<br>0-shot | 102.6 | 31.88 | 32.72 |
160
- | MuSR<br>0-shot | 101.2 | 42.20 | 42.72 |
161
- | MMLU-Pro<br>5-shot | 99.12 | 55.70 | 55.21 |
162
- | **OpenLLM v2<br>Average Score** | **100.48** | **56.60** | **56.87** | |
163
- | MMMU<br>0-shot | 101.6 | 53.44 | 54.33 |
164
- | ChartQA<br>0-shot<br>exact_match | 100.8 | 65.88 | 66.44 |
165
- | ChartQA<br>0-shot<br>relaxed_accuracy | 99.82 | 88.92 | 88.76 |
166
- | **Multimodal Average Score** | **100.6** | **69.41** | **69.84** |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
146
 
147
  | | Recovery (%) | meta-llama/Llama-4-Scout-17B-16E-Instruct | RedHatAI/Llama-4-Scout-17B-16E-Instruct-quantized.w4a16<br>(this model) |
148
  | ---------------------------------------------- | :-----------: | :---------------------------------------: | :-----------------------------------------------------------------: |
149
+ | ARC-Challenge<br>25-shot | 98.51 | 69.37 | 68.34 |
150
+ | GSM8k<br>5-shot | 100.4 | 90.45 | 90.90
151
+ | HellaSwag<br>10-shot | 99.67 | 85.23 | 84.95 |
152
+ | MMLU<br>5-shot | 99.75 | 80.54 | 80.34 |
153
+ | TruthfulQA<br>0-shot | 99.82 | 61.41 | 61.30 |
154
+ | WinoGrande<br>5-shot | 98.98 | 77.90 | 77.11 |
155
+ | **OpenLLM v1<br>Average Score** | **99.59** | **77.48** | **77.16** |
156
+ | IFEval<br>0-shot<br>avg of inst and prompt acc | 99.51 | 86.90 | 86.47 |
157
+ | Big Bench Hard<br>3-shot | 99.46 | 65.13 | 64.78 |
158
+ | Math Lvl 5<br>4-shot | 99.22 | 57.78 | 57.33 |
159
+ | GPQA<br>0-shot | 100.0 | 31.88 | 31.88 |
160
+ | MuSR<br>0-shot | 100.9 | 42.20 | 42.59 |
161
+ | MMLU-Pro<br>5-shot | 98.67 | 55.70 | 54.96 |
162
+ | **OpenLLM v2<br>Average Score** | **99.54** | **56.60** | **56.34** | |
163
+ | MMMU<br>0-shot | 100.6 | 53.44 | 53.78 |
164
+ | ChartQA<br>0-shot<br>exact_match | 100.1 | 65.88 | 66.00 |
165
+ | ChartQA<br>0-shot<br>relaxed_accuracy | 99.55 | 88.92 | 88.52 |
166
+ | **Multimodal Average Score** | **100.0** | **69.41** | **69.43** |
167
+ | RULER<br>seqlen = 131072<br>niah_multikey_1 | 98.41 | 88.20 | 86.80 |
168
+ | RULER<br>seqlen = 131072<br>niah_multikey_2 | 94.73 | 83.60 | 79.20 |
169
+ | RULER<br>seqlen = 131072<br>niah_multikey_3 | 96.44 | 78.80 | 76.00 |
170
+ | RULER<br>seqlen = 131072<br>niah_multiquery | 98.79 | 95.40 | 94.25 |
171
+ | RULER<br>seqlen = 131072<br>niah_multivalue | 101.6 | 73.75 | 74.95 |
172
+ | RULER<br>seqlen = 131072<br>niah_single_1 | 100.0 | 100.00 | 100.0 |
173
+ | RULER<br>seqlen = 131072<br>niah_single_2 | 100.0 | 99.80 | 99.80 |
174
+ | RULER<br>seqlen = 131072<br>niah_single_3 | 100.2 | 99.80 | 100.0 |
175
+ | RULER<br>seqlen = 131072<br>ruler_cwe | 87.39 | 39.42 | 33.14 |
176
+ | RULER<br>seqlen = 131072<br>ruler_fwe | 98.13 | 92.93 | 91.20 |
177
+ | RULER<br>seqlen = 131072<br>ruler_qa_hotpot | 100.4 | 48.20 | 48.40 |
178
+ | RULER<br>seqlen = 131072<br>ruler_qa_squad | 96.22 | 53.57 | 51.55 |
179
+ | RULER<br>seqlen = 131072<br>ruler_qa_vt | 98.82 | 92.28 | 91.20 |
180
+ | **RULER<br>seqlen = 131072<br>Average Score** | **98.16** | **80.44** | **78.96** |
model-00001-of-00014.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:34ce5813ecfa1915201f7a2ded7595e42b6edb343692926fb7e8cca757db8575
3
  size 4998638648
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aeab3eeaefcb9c5c1c72528442778be1a3cac6e4bfb11fd82bb0272c0e92aae4
3
  size 4998638648
model-00002-of-00014.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b2ca1a631427ce8e3a5ba839b98092159e97b59033972088c85bf5fb527a799f
3
  size 4959308680
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:978af0a1c38883ca4877c7f55084808d15c5900554b4d2908ade82093a61917a
3
  size 4959308680
model-00003-of-00014.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1e880d5785420d723c560149dc0c561b150280a368d2e46146dc35706e312dd1
3
  size 4989434120
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:35bcc71c7c66b1152cf688bb822c3d89c1d2e51fd4f0ac163c7702b9d8e2f08d
3
  size 4989434120
model-00004-of-00014.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:502c1678435a66f4696040ec38b777d9b10ee0cc04195b0dee4a4acb790730bc
3
  size 4990090304
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d8ec61242a35a697e028ca9d80b01a7df9fb6cdf1dc10d510341201406f634a1
3
  size 4990090304
model-00005-of-00014.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:49df15a28f95daaa0f9796687eddb67b87348fcbdb875b6e64748a291b5691f1
3
  size 4980916048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af4c4fbd37f0a8aa0fbc290e317be80e20d6e86e1e06554e49a0b3b60894d36b
3
  size 4980916048
model-00006-of-00014.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:48e04c79523a156536df01dcca2d879ea8363090fb926b44650f14af140ad383
3
  size 4980916048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e8d0eaf5443d0f633a1b5f9dbf76b211471d4f8d05f74c1d2b9da8f0dd45fb7
3
  size 4980916048
model-00007-of-00014.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d1f72e8b7dd0ac06960e72c0b3d563a42ec26b64829fc06b30eac020408e9a2f
3
  size 4980916048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db79f8289707e38b13edb2a941b48f2f4e57a1b6fe942d46972a1b460b0a2262
3
  size 4980916048
model-00008-of-00014.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:de8b9e02664811b0ab805c0c02c6604ac8d7df61454eaa59a80369ec532f28fc
3
  size 4980916048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:45fdbc9a53efa1df057018d510594e0e8598c681c467c86644899ba0c0dbeadc
3
  size 4980916048
model-00009-of-00014.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ab179c6195ab988199154d52c275b7a1b4f990146b3764a9a06e796d2b9ee094
3
  size 4980916048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60364636902bd2c3e7fa831613b6eec3bfdf5b52ecaf42837170073ead2409a1
3
  size 4980916048
model-00010-of-00014.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0338b98b767b69a6f0408691db65712174916d809bfe632c3621ee5fb0d847e6
3
  size 4980916048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44e7a6939e3283fa84c90be125564bd6d7ac218df73445860d7fff0dffe0e061
3
  size 4980916048
model-00011-of-00014.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:bd82b64e02b8bad0aeba3c5ef19bd14bbb0387dd8febafa001d293a6bbe00156
3
  size 4980916048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:762460ebb2892d9f9c4e2369668a354d65665834596391d4e742f312bd378ab3
3
  size 4980916048
model-00012-of-00014.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f9506596c2d50bbd394830da875437eafda8abc2262be15402c175ac4f7ba569
3
  size 4980916048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0426564c4cdaaf17a90bfaff8162a1ebcc283e6f98552e87b18077ee7897f24e
3
  size 4980916048
model-00013-of-00014.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ea3475d1aa719f980da467d32786e5acc445b766edf2f51ec082689dd6fd132f
3
  size 3020522640
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d5525d11dd245b45535b5509df307d3293528bbcff51a221f4413eb943a3569d
3
  size 3020522640