llmcompressor settings
#1
by
hf-100
- opened
Hi, thank you for providing this quant: I was wondering how it was created using llmcompressor
So far I've only been able to generate text-only quants with llmcompressor which wouldn't load on vllm nightly, but it looks like this one does work correctly due to including the vision functionality in the resulting model