Hi, I’m using Wav2Vec2ForCTC.from_pretrained(“facebook/wav2vec2-base”) to fine-tune on a English language medical translation dataset, which is about 6GB. It is similar to the Timit_ASR dataset, with the exception that the wav files are in 48KHz. I’m following the example show in this notebook: Fin…

Hello, I had the same issue, I can give my solution but I don’t know if it will work for you. So, in the pytorch documentation ( CTCLoss — PyTorch 2.1 documentation ), it is said that The alignment of input to target is assumed to be “many-to-one”, which limits the length of the target sequence su…

Hi Omar, first of all, thank you for your response. I did try your suggestion. Unfortunately, it does not solve the problem. -Sidd

@ssaran Did you eventually get around this issue? I’m also finding myself with inf/nan losses as I’m downstream training Wav2Vec2ForCTC models. :neutral_face:

Unfortunately, I did not. While I continued to use Wav2Vec2, I didn’t fine tune it to my dataset. I might try it again another time.

Aw, okay. Thanks for the reply though!

Have you tried to reduce your learning rate？

Can you please clarify what the issue with your data shapes was? I am also having the same issue and I can’t solve it.

It was a problem with the sampling rate in my case. I was mistakenly using too short segments of audio so the model was unable to learn anything. You might as well try cleaning your audio if it’s too noisy.

Thanks for answering. I have a few training instances that are <1 second duration. You think they might cause a problem? I also ended up trying this previous suggestion model.config.ctc_zero_infinity = True and it seems to be working so far.

Wav2Vec2: How to correct for nan in training and validation loss

Models

cheulyop July 21, 2021, 9:26am 8

There was an issue with the shape of my data. It works fine after fixing the bug! But to add some info for someone else who might be having a similar problem, cleaning the speech before finetuning could help!

Topic		Replies	Views
`nan` training loss but eval loss does improve over time Research	5	4202	October 10, 2022
Wav2Vec2: loss growing in training and validation after few epochs Models	6	2172	September 25, 2024
Finetuning Wav2Vec2 loss constant Beginners	1	351	August 14, 2023
Fine-tuning wav2vec2 loss explodes and then goes to zero after certain time-steps 🤗Transformers	0	483	November 17, 2021
Wav2Vec2: fix growing training and validation loss after few epochs Models	5	2320	January 27, 2022

Wav2Vec2: How to correct for nan in training and validation loss

Related topics