What are the differences and if Trainer can do multiple GPU work, why need Accelerate? Accelerate use only for custom code? (add or remove something)

I assume accelerate was added later and has more features like: """ Accelerate is a library that enables the same PyTorch code to be run across any distributed configuration by adding just four lines of code! tldr; handles all from cpu-gpu(s)-multi-node-tpu-tpu + deepseed + mixprecision in one sim…

[image] brando: I assume accelerate was added later and has more features like: The Trainer now uses accelerate as the backbone for it (our work the last few months) so it’s "do you want raw accelerate? Or the Trainer API). The capabilities are the same overall :slight_smile:

just saw this. Pasting this as a ref: Hugging Face Trainer? · Issue #144 · huggingface/accelerate · GitHub

answer: Since the trainer already has created an accelerator obj inside it’s own code you have to do no code changes except for writing your own accelerate config and calling it as : accelerate launch --config_file {path/to/config/my_config_file.yaml} {script_name.py} {--arg1} {--arg2} ... An exa…

Hi can someone help me out? I am working in Google Colab to learn how to finetune a ViT model, following this tutorial Vision Transformers (ViT) Explained + Fine-tuning in Python - YouTube . Trying to define the following arguments training and testing dataset feature extractor model collate func…

Have you solved this? I meet the same question: [image] Google Colaboratory

[image] didi-ai: you After installing you need to restart the runtime so it loads the newer version

Thanks, I changed the runtime type to GPU and do not change any code, then the code is works. The code still didn’t work on CPU. Do you know why?

[image] didi-ai: Do you know why? On the CPU version, do pip install accelerate -U, make sure it shows the latest, and do Runtime → Restart Runtime, then run the code again (skipping the install). Let me know if this still errors out

Trainer and Accelerate

🤗Transformers

brando June 28, 2023, 10:20pm 3

Topic		Replies	Views
Trainer class with Accelerate Beginners	2	69	September 22, 2024
Difference between using the Trainer class vs Accelerate library DeepSpeed	0	961	June 27, 2023
Besides writing your own training loop, is there any other advantage for using it with deepspeed? 🤗Accelerate	2	672	July 4, 2023
Trainers.train() with accelerate 🤗Transformers	2	4745	June 28, 2023
Code terminates without training while using accelerate 🤗Accelerate	3	235	April 13, 2024

Trainer and Accelerate

Related topics