Transformer-VAE Convert a T5 model into a variational autoencoder for text. I have already made a project that does this in PyTorch but its never been trained at scale. This project is to convert the autoencoder into Flax so it can be trained efficiently on a TPU to train the largest ever Transfor…

How does this work? Currenlty the takes a transformer encoder and decoder and puts a VAE between them. [.] The VAE forms a compressed latent code which allows interpolating on the training data. [.] For regulatisation an MMD loss is used instead of KL Divergence which reduces posterior collaps…

More Detailed ToDos Sign in to GitHub · GitHub

I really like this project and all the info provided here! Thanks a lot :slight_smile: Let’s try to make this happen! Will ask some people as well

thanks for the awesome information.

[image] alexsunny123: thanks for the awesome information. https://ometv.onl https://chatroulette.top thanks my issue has been fixed.

Train a VAE to interpolate on English sentences

Flax/JAX Projects

Fraser June 28, 2021, 5:57pm 4

old post: Train the best ever transformer-VAE - #14 by Fraser

Topic		Replies	Views
Train the best ever transformer-VAE Flax/JAX Projects	15	7194	August 26, 2021
Training Bart as a VAE for interpolation Models	0	701	August 1, 2022
Building an variational autoencoder with transformers Beginners	1	767	March 17, 2024
Create a SentenceTransformer in Dhivehi using ELECTRA 🤗 Course Projects	5	1066	November 17, 2021
Fine tuning T5 Encoder and T5 Decoder separately 🤗Transformers	1	855	May 6, 2024

Train a VAE to interpolate on English sentences

Related topics