Hi. I finetuned mistralai/Mistral-Small-24B-Base-2501 on a dataset and now I’m trying to run inference for it. I’m using AutoModelForCausalLM.from_pretrained to load it but getting this error: Could not find MistralForCausalLM neither in transformers. I’m running the latest version of transformers 4…

Judging just by the error, it’s probably a version mismatch between torch and torchvision . pip install torchvision==x.xx.x Domain Version Compatibility Matrix for PyTorch

Hmm, maybe it’s missing dependencies or something …? I don’t think the class itself is actually missing… pip install -U mistral_common sentencepiece import transformers, sys print("transformers", transformers.__version__) try: from transformers.models.mistral.modeling_mistral import MistralFor…

@John6666 getting this when I run that code snippet `` MistralForCausalLM FAIL: partially initialized module ‘torchvision’ has no attribute ‘extension’ (most likely due to a circular import) ```

[image] John6666: it’s missing dependencies or something @John6666 thanks! yes, aligning the versions helped :slight_smile: I have fine-tuned the model and now running into this run-time error while loading it: RuntimeError: Error(s) in loading state_dict for Embedding: size mismatch for w…

Based on the error message, I’d guess it’s either trying to load the PEFT adapter as a whole model weight or the model weights are corrupted… Embedding size mismatch when hyperparameter search #16479 Load adapter Size Mismatch error for LLM checkpoint of PEFT model with a resized token embeddings

@John6666 could this be because of deepspeed? when I do len(tokenizer) it prints 131072.

could this be because of deepspeed I think very likely… When saving fails in DeepSpeed, it appears an empty tensor is saved instead. modules_to_save resulting in empty tensor with deepspeed zero3 LoRA training #2450 DeepSpeed - Save model weights

@John6666 I’m using "stage3_gather_16bit_weights_on_model_save": true as suggested here . Not sure what else is causing this.

This may also occur when using BF16 or when using older version of PEFT . pip install -U peft

@John6666 using model.save_16bit_model() to save the model insread of save_pretrained() fixed this!

Could not find MistralForCausalLM in transformers

Beginners

John6666 September 1, 2025, 4:14am 6

Based on the error message, I’d guess it’s either trying to load the PEFT adapter as a whole model weight or the model weights are corrupted…

Embedding size mismatch when hyperparameter search #16479
Load adapter
Size Mismatch error for LLM checkpoint of PEFT model with a resized token embeddings

Topic		Replies	Views
Tried to download Mistral 7B but got an error message 🤗Transformers	3	13687	October 8, 2023
I'm having an issue with the mistral-instruct model Models	4	2514	February 8, 2024
Loading pre-trained models with AddedTokens 🤗Transformers	2	936	October 14, 2024
ImportError: cannot import name 'PreTrainedModel' from 'transformers' Beginners	2	2896	August 27, 2025
TypeError: MistralModel.__init__() got an unexpected keyword argument 'safe_serialization' 🤗Transformers	0	468	May 17, 2024

Could not find MistralForCausalLM in transformers

Related topics