How do you fine-tune a model using peft

Let's say you have a model variable containing your model as seen in How do you produce text from a text generation model. How do you take this model and do Fine-tuning on it ?

We use the 🤗 peft library, which stands for State-of-the-art Parameter-Efficient Fine-Tuning.

Steps:

Code snippets summarized

We are then ready for training. We can start the training using a Trainer object. We pick the hyperparameters of the training like the number of steps, the learning rate, the optimizer, etc...

import transformers
from peft import LoraConfig, get_peft_model

model_name = "EleutherAI/gpt-neo-125M"
tokenizer = transformers.AutoTokenizer.from_pretrained(model_name, device=device)
model = transformers.AutoModelForCausalLM.from_pretrained(model_name).to(device)

config = LoraConfig(
    r=8,
    lora_alpha=32,
    target_modules=["qkv_proj","out_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

model = get_peft_model(model, config)

tokenizer.pad_token = tokenizer.eos_token

model.train()

trainer = transformers.Trainer(
    tokenizer=tokenizer,
    model=model,
    train_dataset=data,
    args=transformers.TrainingArguments(
        resume_from_checkpoint="outputs",
        save_steps=10,
        per_device_train_batch_size=1,
        gradient_accumulation_steps=8,
        gradient_checkpointing=True,
        warmup_steps=5,
        max_steps=100,
        learning_rate=5e-4,
        use_mps_device=True,
        logging_steps=1,
        output_dir="outputs",
        optim="adamw_torch",
    ),
    data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False),
)

trainer.train(resume_from_checkpoint=False)

Training can take a long time so we prepare to fetch a cut of coffee and wait a bit. On 1B models, you can get decent results in about 30 minutes.

I'd recommend installing the Weights*\ \ Biases*(wandb) library to get chart of how the training is going.

If interrupted, training will automatically resume from the last checkpoint !

Once training is over, you can use the generate_text function from How do you produce text from a text generation model to test the differences the training made.

Code snippets summarized​

Code snippets summarized