I have a dataset of scientific abstracts that I would like to use to finetune GPT2. However, I want to use a loss between the output of GPT2 and an N-grams model I have to adjust the weights. Is it possible to do this using huggingface transformers and if so, how? Thank you in advance! EDIT: Let m…

Thanks to everyone helping out and helping me get things setup right. I really appreciate the feedback and assistance! @sgugger : Aahh, ok I see what you’re saying. I’ve modified the DataSet object to: from torch.utils.data import Dataset import pandas as pd import torch class SDAbstractsDataset(D…

The issue is now when you decode tensors using tokenizer.decode. Note that this method expects a 1d tensor or a list of ints, I don’t think it works on a batch. I don’t see which part of your code uses that so can’t advise more.

Yes, there are two decode lines in my forward method: def forward( self, input_ids=None, past=None, attention_mask=None, token_type_ids=None, position_ids=None, head_mask=None, inputs_embeds=None, labels=None, use_cache…

@aclifton314 , just to confirm, gradients cannot flow through sampling methods. If you are sampling at any stage to get token ids, and then pass these ids into your n-gram model, your gradients will be zero after this point onwards. gradients cannot flow through the line: next_token = torch.multino…

@chrisdoyleIE , I’ve updated the method a bit since that last post so let me detail it here: I generate a sequence of a given size based off the current state of the model: full_generated_gpt2_ids = self.generate(input_ids=tmp_input_ids, max_length=ma…

@chrisdoyleIE , if I follow the post it boils down to the sampling methods being non-differentiable. Is this correct? If so, I would wonder about ReLU since it is not differentiable at 0. Would that seem to indicate, then, that it would be sufficient for the functions involved in backpropagation to b…

Paragraph 1: Correct - sampling is non-differentiable. Regarding ReLU, discontinuities such as the one found at 0 can be avoided in the function’s implementation (so long as operators used are good with PyTorch’s autograd): # source: https://github.com/CSTR-Edinburgh/mlpractical/blob/mlp2019-20/lab…

I don’t think I fully understand the response for paragraph 2. If we consider a neural net with: E: \text{loss function} \\ o_{j} = \phi(net_{j}) = \phi(\sum_{k=1}^{n}w_{kj}o_{j}) \\ \phi: \text{activation function} Then we can use backpropagation to calculate the derivative of the error wrt the …

All correct, but I feel you may have missed my point (though I re-read my comment and it wasn’t terribly clear). First: activation function does not have to be continuous in for brop to work in practice, it can be implemented as it is in my previous reply, such that we account for these discontinui…

Finetuning GPT2 with user defined loss

Beginners

chrisdoyleIE August 11, 2020, 11:08am 43

@aclifton314 yup, still the case that the gradients won’t flow through the sampling line. Check out this this post

Topic		Replies	Views
Finetuning GPT2 using Multiple GPU and Trainer 🤗Transformers	14	6934	May 22, 2023
Chapter 3 questions Course	160	11806	March 7, 2026
Fine tuning GPT2 tensorflow 🤗Transformers	0	103	June 24, 2024
Key Error 'loss' while fine tuning GPT-2 with the Trainer utility 🤗Transformers	9	7544	May 10, 2022
Fine tune with SFTTrainer Intermediate	17	16241	September 12, 2024

Finetuning GPT2 with user defined loss

Related topics