ESM Fast Tokenizer

nina789 January 15, 2024, 9:10pm 1

Is there a FastTokenizer implementation for ESM models?

Topic		Replies	Views
Fast tokenizer for marianMTModel 🤗Tokenizers	1	536	September 26, 2022
BertTokenizerFast for stsb-xlm-r-multilingual model 🤗Tokenizers	3	699	April 8, 2021
Introducing FlashTokenizer: The World's Fastest Tokenizer Library for LLM Inference 🤗Tokenizers	2	68	March 21, 2025
🚀 Introducing FlashTokenizer: The World's Fastest CPU Tokenizer! 🤗Transformers	2	137	April 4, 2025
FastTokenizer add 10 more tokens in Avg 🤗Tokenizers	0	216	January 20, 2024