Hi Huggingfacers I have a number of questions regarding finetuning a language model: How to mask a selective portion of a given input sentence instead of masking randomly. For example, if I am using ALBERT as a model, and I am aiming to do a different kind of loss function than the standard MLM l…

Closing this, since you opened another thread for this and received an answer.

How to do selective masking in Language modeling

leoapolonio August 13, 2020, 2:27pm 2

I think the answer is similar another post: Selective masking in Language modeling

Topic		Replies	Views
Selective masking in Language modeling Beginners	1	2278	August 13, 2020
Masking specific token in each input sentence during Masked language modelling 🤗Transformers	0	1082	October 18, 2021
Mask modelling on specific words Beginners	1	1091	March 25, 2021
Fine-tune BERT for Masked Language Modeling 🤗Transformers	3	3124	January 25, 2021
How to make a model predict on only some tokens Beginners	1	627	June 16, 2022