Cannot export my model

[Tum1370]

All week i have been exporting models fine, and suddenly yesterday i can not export my gguf model. I keep getting this error.
I have uninstalled and reinstalled, and still get the same issue. The error is the following.

GGUF export failed: Unsloth: GGUF conversion failed: Unsloth: Failed to convert text model to GGUF with command N:\AI_Tools\Pinokio\api\Unsloth-Studio\env\Scripts\python.exe N:\AI_Tools\Pinokio\api\Unsloth-Studio\app\.unsloth\llama.cpp\unsloth_convert_hf_to_gguf.py --outfile Qwen3.5-4B.F16.gguf --outtype f16 --split-max-size 50G N:\AI_Tools\Pinokio\api\Unsloth-Studio\app\.unsloth\studio\exports\Qwen3.5-4B-GGUF\_tmp_model_191a7e51: Command '['N:\AI_Tools\Pinokio\api\Unsloth-Studio\env\Scripts\python.exe', 'N:\AI_Tools\Pinokio\api\Unsloth-Studio\app\.unsloth\llama.cpp\unsloth_convert_hf_to_gguf.py', '--outfile', 'Qwen3.5-4B.F16.gguf', '--outtype', 'f16', '--split-max-size', '50G', 'N:\AI_Tools\Pinokio\api\Unsloth-Studio\app\.unsloth\studio\exports\Qwen3.5-4B-GGUF\_tmp_model_191a7e51']' returned non-zero exit status 1. --- converter stderr --- INFO:hf-to-gguf:Loading model: _tmp_model_191a7e51 INFO:hf-to-gguf:Model architecture: Qwen3_5ForConditionalGeneration INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json' INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors-00001-of-00002.safetensors' INFO:hf-to-gguf:gguf: indexing model part 'model.safetensors-00002-of-00002.safetensors' INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only INFO:hf-to-gguf:Exporting model... INFO:hf-to-gguf:token_embd.weight, torch.bfloat16 --> F16, shape = {2560, 248320} INFO:hf-to-gguf:blk.1.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.1.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.1.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.10.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.10.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.10.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.11.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.11.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.11.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.12.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.12.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.12.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.13.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.13.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.13.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.14.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.14.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.14.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.15.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.15.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.15.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.16.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.16.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.16.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.17.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.17.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.17.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.18.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.18.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.18.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.19.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.19.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.19.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.2.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.2.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.2.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.20.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.20.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.20.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.21.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.21.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.21.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.22.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.22.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.22.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.23.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.23.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.23.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.24.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.24.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.24.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.25.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.25.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.25.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.26.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.29.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.3.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.3.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.3.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.30.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.30.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.30.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.31.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.31.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.31.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.4.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.4.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.4.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.5.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.5.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.5.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.6.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.6.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.6.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.7.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.7.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.7.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.8.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.8.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.8.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.9.ffn_down.weight, torch.bfloat16 --> F16, shape = {9216, 2560} INFO:hf-to-gguf:blk.9.ffn_gate.weight, torch.bfloat16 --> F16, shape = {2560, 9216} INFO:hf-to-gguf:blk.9.ffn_up.weight, torch.bfloat16 --> F16, shape = {2560, 9216} Traceback (most recent call last): File "N:\AI_Tools\Pinokio\api\Unsloth-Studio\app.unsloth\llama.cpp\unsloth_convert_hf_to_gguf.py", line 298, in main() File "N:\AI_Tools\Pinokio\api\Unsloth-Studio\app.unsloth\llama.cpp\unsloth_convert_hf_to_gguf.py", line 292, in main model_instance.write() File "N:\AI_Tools\Pinokio\api\Unsloth-Studio\app.unsloth\llama.cpp\conversion\base.py", line 1027, in write self.prepare_tensors() File "N:\AI_Tools\Pinokio\api\Unsloth-Studio\app.unsloth\llama.cpp\conversion\qwen.py", line 144, in prepare_tensors super().prepare_tensors() File "N:\AI_Tools\Pinokio\api\Unsloth-Studio\app.unsloth\llama.cpp\conversion\base.py", line 881, in prepare_tensors for new_name, data_torch in (self.modify_tensors(data_torch, name, bid)): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "N:\AI_Tools\Pinokio\api\Unsloth-Studio\app.unsloth\llama.cpp\conversion\qwen.py", line 518, in modify_tensors yield from super().modify_tensors(data_torch, name, bid) File "N:\AI_Tools\Pinokio\api\Unsloth-Studio\app.unsloth\llama.cpp\conversion\qwen.py", line 335, in modify_tensors yield from super().modify_tensors(data_torch, name, bid) File "N:\AI_Tools\Pinokio\api\Unsloth-Studio\app.unsloth\llama.cpp\conversion\qwen.py", line 141, in modify_tensors yield from super().modify_tensors(data_torch, name, bid) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "N:\AI_Tools\Pinokio\api\Unsloth-Studio\app.unsloth\llama.cpp\conversion\base.py", line 616, in modify_tensors new_name = self.map_tensor_name(name) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "N:\AI_Tools\Pinokio\api\Unsloth-Studio\app.unsloth\llama.cpp\conversion\base.py", line 609, in map_tensor_name raise ValueError(f"Can not map tensor {name!r}") ValueError: Can not map tensor 'model.layers.32.mlp.down_proj.weight'

[Tum1370]

Is it this commit yesterday that has broken exporting gguf models now ???
I have been all week training a Qwen3.5 4b model, and exporting gguf.... Yesterday that stopped and constantly gives the previous error.
Just seen this commit yesterday. Seems odd this went live when the error started.

98a01e7

[Mikyx-1]

Hi @Tum1370 — dug into this. Sharing what I found, but flagging upfront: this is my likely root cause, not a confirmed one — I don't have your actual checkpoint to inspect, so there's a piece I can't verify from the outside. Would help a lot if you can check the two things at the bottom.

It's probably not that commit

I checked 98a01e70c (#6659) directly. It only changes Studio's inference-time GPU tensor-parallel routing — the logic that decides --split-mode tensor vs. layer-split when loading a GGUF into llama-server for vision/mmproj models. It doesn't touch the HF→GGUF conversion code at all, so it can't be what's breaking your export. Landing in the same week as your issue looks like a coincidence, not a cause.

What I think is actually happening

The failing tensor is model.layers.32.mlp.down_proj.weight. Qwen3.5-4B's config has num_hidden_layers: 32 (layers 0-31) plus mtp_num_hidden_layers: 1 — one extra Multi-Token-Prediction (speculative-decoding) layer. Layer 32 is that MTP layer, not a regular decoder layer.

The bundled llama.cpp fork (unslothai/llama.cpp, conversion/qwen.py) has fairly new code (_Qwen35MtpMixin) that handles this:

self.block_count = self.hparams["num_hidden_layers"]
if not self.no_mtp:
    self.block_count += self.hparams.get("mtp_num_hidden_layers", 0)  # 32 + 1 = 33
...
if name.startswith("mtp."):
    ...
    name = f"model.layers.{cls._original_block_count + mtp_idx}.{...}"  # remaps into slot 32

It expects the MTP layer's weights to show up with an mtp.* prefix, which it then relabels to model.layers.32.* and includes in a 33-layer tensor map. But your error shows the tensor already sitting at model.layers.32.* directly, not under mtp.*. That would happen if:

1. your saved checkpoint's config.json is missing mtp_num_hidden_layers (or has it as 0) — then the converter only builds a 32-layer map and has nowhere to put layer 32, or
2. the checkpoint's weights predate the mtp.* naming convention this converter version expects, so the relabeling step never fires.

I also checked Unsloth's own save.py end-to-end — it never touches num_hidden_layers, and its only Qwen3.5-specific save logic is scoped to the vision/VLM variant (irrelevant for a text checkpoint). So this doesn't look like something Unsloth's merge/save step is corrupting; it looks like a gap between what this checkpoint's MTP layer is named and what the converter's MTP-handling code expects.

What would confirm it

If you can check either of these, it'd pin this down:

• Does your merged model's config.json have mtp_num_hidden_layers: 1, or is it missing/0?
• In the safetensors index, is the 33rd layer's key model.layers.32.* (like the error shows) or model.mtp.layers.0.*?

In the meantime, a workaround worth trying: export a non-MTP Qwen3.5-4B checkpoint if you have one, since the MTP variant's GGUF path looks like the piece that's broken right now.

If this checks out, it'd want to be filed against unslothai/llama.cpp (conversion/qwen.py's MTP handling) rather than this repo.

[Tum1370]

HI @Mikyx-1 , Thanks for looking in to this and trying to find out what's going on. I think there maybe a possible fix happening soon. I had also made an issue for this bug here. #6779 . And it seems there was also a duplicate issue made for it. I think the fix is in the duplicate issue ?

[oobabooga]

@Tum1370 This was an issue with MTP models that has been fixed in unsloth-zoo 2026.7.4. The converter now checks whether your merged weights still carry the MTP layer and keeps or strips mtp_num_hidden_layers to match, instead of always stripping it and then crashing on the leftover tensor.

Upgrade and re-export:

pip install --upgrade unsloth unsloth_zoo

unsloth main already pins unsloth_zoo>=2026.7.4, so a clean upgrade pulls it. If you run Studio through Pinokio, update the app so it refreshes unsloth_zoo inside its env. Your existing Qwen3.5-4B checkpoint should convert to GGUF again after that.