GPT2 for Mongolian Goal is to create a strong language generation model for Mongolian :mongolia: Since initial code and data is pretty much written by @patrickvonplaten and other huggingface members, it should not be so hard to get the first sense. Model Randomly initialized GPT2 model Datasets W…

I am also so interested in pretraining Mongolian Roberta/DistillBERT on OSCAR and fine-tune it on the translated Mongolian SQuAD2.0.

Awesome, officially defining it!

Pretrain GPT-2 from scratch in Mongolian

bayartsogt July 2, 2021, 3:20am 4

Great! Please register here.

Topic		Replies	Views
Pretrain GPT-2 from scratch in Thai Flax/JAX Projects	0	960	July 18, 2021
PreTrain GPT2 from scratch in Russian Flax/JAX Projects	1	712	July 1, 2021
PreTrain GPT2 from scratch in Spanish Flax/JAX Projects	12	2068	July 1, 2021
PreTrain GPT2 from scratch in Bengali Flax/JAX Projects	8	2518	August 19, 2021
PreTrain GPT2 from scratch in Persian Flax/JAX Projects	15	2195	July 7, 2021