Hey everybody, The mT5 and improved T5v1.1 models are added: Improved T5 models (small to large): google/t5-v1_1-small google/t5-v1_1-base google/t5-v1_1-large and mT5 models (small to large): google/mt5-small google/mt5-base google/mt5-large are in the model hub Will upload the 3b and 11b v…

I was trying to fine-tune it on a Chinese short text classification task and found MT5ForConditionalGeneration not in transformers-3.5.1 yet while it is here ?

Congrats :clap::clap::clap: @patrickvonplaten

Added it yesterday, so only on master yet :slight_smile: But we’ll release 4.0 very soon :slight_smile:

And even more T5 pre-trained checkpoints for closed book question answering are here: https://huggingface.co/models?search=ssm . The official paper was released just a couple of days ago :slight_smile:

This issue might also be of interest: https://github.com/huggingface/transformers/issues/8704#issuecomment-734731916

Hi, I fine-tune 3 datasets separately on mT5-small or base model. Three datasets are English STSb dataset, KorSTS dataset, my personal korean news classification dataset. I want a result of the same form as that of T5. Looks like <pad> 5.0 </s>. But my model result in this from. <pad> <extra_id_…

Hi Patrick! , Since we have many new great pretrained models on T5 (not yet considering mT5), I would love to try summarize meaning of their postfixs to make sure we understand them correclty. v1_1 or xl or xxl – use minor-change architecture to original T5 detailed in here and here . Due to thes…

that’s an awesome summary of the new models thank you :slight_smile:

Hello, Thank you for adding mT5 to the Transformers! I tried to fine-tune it using a dataset including Japanese, but it seems the generation result is not good. I think there are some problems in my fine-tuning setting. May I ask the tips for fine-tuning mT5 here? Or should I ask them in T5 Finet…

mT5/T5v1.1 Fine-Tuning Results

Models

valhalla December 21, 2020, 7:42am 11

Would be nice if you ask it in T5 Finetuning Tips and post a link here

Topic		Replies	Views
mT5 translation tasks only generates <extra__id_0>result Models	0	53	October 4, 2024
Issue with finetuning a seq-to-seq model 🤗Transformers	30	4152	August 11, 2022
How to fine-tune T5-base model? Beginners	10	4694	July 28, 2021
Convert mT5 to HF weights? 🤗Transformers	6	1054	November 17, 2020
T5 Finetuning Tips Models	48	58181	November 3, 2024

mT5/T5v1.1 Fine-Tuning Results

Related topics