Is AWQ quantization applied only to the anguage model of this model?
#4
by
sinchir0
- opened
From reading the discussions in the AutoAWQ repository, it appears that AutoAWQ only quantizes the language model.
https://github.com/casper-hansen/AutoAWQ/issues/714#issuecomment-2670467886
Is this model quantized using AutoAWQ? If so, does that mean the Vision Encoder is not quantized and only the language model is quantized?
from the config file. only the language model is quantized.