|
|
--- |
|
|
license: other |
|
|
license_name: llama4 |
|
|
license_link: LICENSE |
|
|
base_model: |
|
|
- meta-llama/Llama-4-Scout-17B-16E-Instruct |
|
|
pipeline_tag: image-text-to-text |
|
|
tags: |
|
|
- gguf-connector |
|
|
--- |
|
|
|
|
|
## llama-4-scout-17b-16e-instruct-gguf |
|
|
- base model from meta-llama |
|
|
- tested on [gguf-connector](https://pypi.org/project/gguf-connector) with nightly [llama-cpp-python](https://github.com/calcuis/llama-cpp-python/releases) |
|
|
|
|
|
## example workflow (run it locally) |
|
|
- download the different parts of the model; for example q2_k |
|
|
- `llama-4-scout-17b-16e-it-q2_k-00001-of-00004.gguf` |
|
|
- `llama-4-scout-17b-16e-it-q2_k-00002-of-00004.gguf` |
|
|
- `llama-4-scout-17b-16e-it-q2_k-00003-of-00004.gguf` |
|
|
- `llama-4-scout-17b-16e-it-q2_k-00004-of-00004.gguf` |
|
|
- pull them all into an empty folder; then execute the merge command: `ggc m2` |
|
|
- the merged gguf is around 36.8GB for q2_k (setup once) |
|
|
- execute connector command: `ggc gpp` or `ggc cpp` |
|
|
- select the merged gguf then start your prompt to interact with llama4 |
|
|
|
|
|
## for model larger than 50GB in total |
|
|
- don't need to merge (linked already); just execute: `ggc gpp` (or `ggc cpp` for ui) |
|
|
- select the first part of the model (i.e., 00001-of-xxxxx) |
|
|
- start your prompt to interact with llama4 |
