GPT2 for Spanish Currently, there is no GPT2 model that was trained from scratch for Spanish on the hub: Models – Hugging Face . For this project, the goal is to create a strong language generation model for Spanish. Model A randomly initialized GPT2 model Datasets One can make use OSCAR the datas…

I’m definitely in! Check this @mrm8488 it’s exactly what we talked about

Yes, it is! I was thinking about using Spanish portion of mC4 instead of OSCAR. What do you think @patrickvonplaten and @mariagrandury ??

Awesome, great that you guys are already so interested in this :slight_smile: The nice thing about OSCAR is that it is supported in the datasets library and we now also support datasets streaming: Load a Dataset in Streaming mode — datasets 1.8.0 documentation which means that you surely won’t run …

Great, @patrickvonplaten . We will. So we have the script, the dataset and the knowledge. Let’s start the party!

Hello Team, I am interested to be a part of such an amazing project & team. I will try my best to contribute to the Spanish GPT2 model. It would be nice if we could discuss some more learning resources that would be useful for this project.I can work in any time zone that is comfortable for everyon…

Awesome! let’s officially define this project :slight_smile: Putting everybody in the official sheet here . More people can still join! Leave a comment here or on the sheet if you want to change something.

hi @valhalla , I am definitely interested in joining. Could you add me to the spreadhset? Thanks,

I would love to contribute to this project :smiley:

@jdposa , @shpotes , you are part of the team :slight_smile:

PreTrain GPT2 from scratch in Spanish

Flax/JAX Projects

Pablogps June 24, 2021, 9:38am 6

This sounds good! As I said in the project for BERTIN, I’d like to help in either one.

Topic		Replies	Views
PreTrain GPT2 from scratch in Russian Flax/JAX Projects	1	712	July 1, 2021
PreTrain GPT2-Large (and/or GPT2-XL) from scratch in Portuguese Flax/JAX Projects	0	769	June 24, 2021
Pretrain GPT-2 from scratch in Thai Flax/JAX Projects	0	960	July 18, 2021
PreTrain GPT2 from scratch in Bengali Flax/JAX Projects	8	2518	August 19, 2021
PreTrain GPT2 from scratch in Indonesia Flax/JAX Projects	13	837	June 30, 2021

PreTrain GPT2 from scratch in Spanish

Related topics