"I can't believe it's not Unsloth!"
— Grandma

Create optimized GGUF quantizations by cloning from any GGUF of the same architecture.

Whether you just fine-tuned your own model or stumbled upon a new hidden gem on Hugging Face, this tool allows you to quickly quantize it with the same settings as an existing, high-quality quant.

In theory, fine-tunes should benefit from the same imatrix data and optimizations. In practice... it's hard to tell for sure. But at least it's now dead simple to try.

• Simple YAML config
• Use a Hugging Face repo or local path as template (model to copy from)
• Use a Hugging Face repo or local path as target (model to quantize)
• Optionally copy GGUF metatdata and loose files (mmproj, etc)
• Run multiple quantizations from the same template
• Use split GGUFs as template input and/or target output
• Works with your existing Hugging Face cache and llama.cpp install

• Python 3.9+ (developed on 3.12, not thoroughly tested on others)
• The following llama.cpp tools in PATH or built under GGUF_CLONE_LLAMA_CPP (see below):
  • llama-quantize
  • llama-gguf-split

# Recommended
uvx gguf-clone # assumes config.yml in current dir
uvx gguf-clone path/to/config.yml

# Or
pip install gguf-clone
gguf-clone
gguf-clone path/to/config.yml

# Verbose output
gguf-clone --verbose

# Non-interactive modes to skip prompts when outputs already exist
gguf-clone --use-existing
gguf-clone --overwrite
gguf-clone --cancel

Outputs are generated alongside the config file by default:

• converted/*.gguf - Target model converted into a GGUF for llama.cpp
• params/*.json - llama-quantize paramaters extracted from template GGUF(s)
• quantized/*.gguf - Final quantized outputs

Output file names are keyed by source:

• Hugging Face sources use the repo id (for example unsloth-Qwen3-0.6B)
• Local path sources only the path leaf name (for example /a/exp1/model and /b/exp2/model both become model)

Use either repo or path for each template and target (exactly one per section):

• repo: ORG/MODEL on Hugging Face
• path: local directory (or .gguf file for target only)

Template matching still uses ggufs, imatrix, and copy_files patterns.
target.exclude_files is only used for Hugging Face downloads.

Minimal example (only required fields):

template:
  repo: unsloth/Qwen3-0.6B-GGUF
  imatrix: "*imatrix*"
  ggufs: "*UD-IQ1_M*.gguf"
  
target:
  repo: Qwen/Qwen3-0.6B

Local-path example:

template:
  path: ./models/template
  imatrix: "*imatrix*"
  ggufs: "*UD-IQ1_M*.gguf"
  
target:
  path: ./models/target

All options:

template:
  # Exactly one of repo or path
  repo: unsloth/Qwen3-0.6B-GGUF
  # path: ./models/template
  imatrix: "*imatrix*"
  
  # List multiple patterns to create multiple quantizations
  ggufs:
    - "*UD-IQ1_M*.gguf"
    - "*UD-Q2_K_XL*.gguf"
  
  # Copy GGUF metadata (from the 1st shard if split)
  copy_metadata:
    - tokenizer.chat_template
    
  # Copy matched files
  copy_files:
    - "*mmproj*"

target:
  # Exactly one of repo or path
  repo: unsloth/Qwen3-0.6B
  # path: ./models/target
  # path: ./models/already-converted.gguf  # skips conversion step

  # Skip downloading unnecessary files from the target repo (HF only)
  exclude_files:
    - "*.onnx"
    - "GGUFs/*"

output:
  # Output model is named PREFIX-ORG-MODEL
  prefix: not-unsloth
  
  # Add or edit GGUF metadata
  apply_metadata:
    general.quantized_by: "https://github.com/spicyneuron/gguf-clone"

  # Split output if greater than this size (unit can be M or G)
  split: 50G

  # Output directories (relative to config.yml location)
  converted_dir: converted
  params_dir: params
  quantized_dir: quantized

If you don't have llama.cpp tools on your PATH, point gguf-clone at a local llama.cpp repo:

GGUF_CLONE_LLAMA_CPP="path/to/llama.cpp/repo"

If GGUF_CLONE_LLAMA_CPP is set, gguf-clone will prefer that its scripts and tools over the PATH and vendored copies.

Hugging Face environment variables can be used to change your download cache location.

• Cloning accuracy: The tool uses heuristics to determine quantization parameters from template GGUFs (most common tensor type as default, ignore lists for non-quantized tensors). For identical tensor names and shapes, results should be functionally equivalent.
• Vendored dependencies: gguf-py/gguf is vendored from llama.cpp because the published version on PyPI is out of date with recent llama.cpp versions. convert_hf_to_gguf.py is also vendered so conversion behavior stays aligned.

All credit goes to the Unsloth and llama.cpp teams for their hard work and responsiveness. This is merely a wrapper around their genius.

This project was originally inspired by quant_clone. It didn't fully work as expected, but pointed me in the right direction.