Hi everyone, I’m working on fine-tuning LLaMA 3.1 8B for a multi-class classification task using text data. However, my dataset is highly imbalanced, which is causing performance issues. I have already tried several data augmentation techniques like back translation, paraphrasing, and contextual a…

There don’t seem to be many examples… [image] SFTTrainer Loss function Beginners I have a couple of questions: how to know the loss function used by default for SFTTrainer for a given model and how to alter it? For training an LLM the loss function is computed on the…

Hi there! I’ve faced a similar challenge before, so I thought I’d share what worked for me when using a custom loss (like focal loss) with Hugging Face’s Trainer. 1. Create a Custom Trainer The easiest way to use your own loss function is to subclass the Trainer and override the compute_loss metho…

Create a weighted loss function to handle imbalance?

🤗Transformers

John6666 January 31, 2025, 4:58am 2

There don’t seem to be many examples…

Topic		Replies	Views
How can I use class_weights when training? 🤗Transformers	19	32016	December 29, 2022
Custom loss weight for train a different weight for validation 🤗Transformers	0	240	April 4, 2024
Training with class weights 🤗Transformers	5	3339	November 18, 2023
Class weights in Trainer() instance Intermediate	1	784	September 10, 2021
Trainer API Error when Hyperparameter Tuning with Custom Loss Function Beginners	1	667	May 30, 2024

Create a weighted loss function to handle imbalance?

Related topics