this variable is position embedding which is getting initialised as 1,50,128 but is reading the weights of cls_token which is 1,1,128
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| "embeddings.position_ids - UNEXPECTED" warning started showing | 1 | 5102 | February 5, 2026 | |
| Positional encoding error in RoBERTa | 1 | 378 | October 2, 2023 | |
| Positional Embeddings in Transformer Implementations | 1 | 1895 | September 3, 2024 | |
| Issues with Whisper Encoder: Positional Encoding | 4 | 1724 | November 16, 2022 | |
| Why we add math to word embedding | 0 | 283 | March 13, 2022 |