Hello! I am trying to download llama-2 for text generation on google colab free version. I tried simply the following model_name = "meta-llama/Llama-2-7b-chat-hf" tokenizer = AutoTokenizer.from_pretrained(model_name, token=True) model = AutoModelForCausalLM.from_pretrained(model_name, token=True) …

by any chance you found something

You can use llama 2 in colab using 4 bit quantization this shorten the memory usage but this will not work without GPU below is the link: [image] Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA We’re on a journey to advance and democratize artificial inte…

Llama-2 on colab

Beginners

obi77 August 8, 2023, 8:47pm 2

I think I am having the same issue Colab RAM Limit Exceeded: Unable to Run 3B Model Even with Quantization I will share a solution if I find it

Topic		Replies	Views
Colab RAM Limit Exceeded: Unable to Run 3B Model Even with Quantization Beginners	0	1090	August 8, 2023
Colab CUDA OOM using Llama-2-7b-chat-hf even with 40GPU RAM 🤗Transformers	0	947	December 29, 2023
Can't load fine tuned LLamav2 7b Beginners	2	1136	October 13, 2023
How to get Llama-2-13b-chat-hf to ACTUALLY RUN Beginners	0	294	May 30, 2024
Colab RAM crash error - Fine-tuning RoBERTa in Colab Beginners	3	6659	December 15, 2020

Llama-2 on colab

Related topics