LatentSync: Audio Conditioned Latent Diffusion Models for Lip Sync
Paper • 2412.09262 • Published • 1
Paper: https://arxiv.org/abs/2412.09262
Code: https://github.com/bytedance/LatentSync
LatentSync 1.6
Many people have reported that the teeth and lips generated by LatentSync 1.5 are blurry. To address this issue, we trained LatentSync 1.6 on 512 x 512 resolution videos.
Notably, we did not make any changes to the model structure or training strategy; the only modification was upgrading the training dataset to 512 x 512 videos. Therefore, the current code is compatible with both LatentSync 1.5 and 1.6. To switch between versions, you only need to load the corresponding checkpoint and modify the resolution parameter in the U-Net config file.
You can view the demo in LatentSync's official GitHub repo.
- Downloads last month
- 35,541
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 1 Ask for provider support
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js