GPT-Neo-2.7B Fine-Tuning Example Using HuggingFace & DeepSpeed
Installation
cd venv/bin
./pip install -r ../../requirements.txt
./pip install deepspeed==0.3.13
Example with GPT-Neo-2.7B with DeepSpeed
Example with GPT-Neo-1.3B without DeepSpeed
DeepSpeed configuration
Training and testing log