What is Parameter-Efficient Fine-Tuning (PEFT)?

Parameter-Efficient Fine-Tuning (PEFT) makes it easier to adapt large pre-trained models to new tasks without updating the entire model. By training only a small number of parameters, PEFT can reduce the resources required for fine-tuning and make task-specific model versions easier to manage.

Problem with Traditional Fine-Tuning

Traditional fine-tuning updates a pre-trained model to perform a specific task, but it becomes inefficient as model size grows. Modern LLMs contain billions of parameters making full fine-tuning costly and resource-intensive.

• Updates all model parameters, which requires high computational power and memory.
• Needs large storage since separate copies of the model are saved for each task.
• Training becomes slow and expensive, especially for very large models.
• Risk of overfitting when fine-tuned on small or task-specific datasets.
• Difficult to deploy and maintain multiple fine-tuned models in real-world applications.

Working

1. Start with a Pre-trained Model: A pre-trained model such as BERT, GPT or T5 is used as the starting point. It already contains general knowledge learned from large-scale training.
2. Freeze Most Model Parameters: The original model weights are kept frozen during fine-tuning. This avoids updating billions of parameters for every new task.
3. Add Trainable Parameters: A small set of task-specific parameters or modules is added to the model. Depending on the PEFT technique, these can be adapters, low-rank matrices, bias terms or learnable prompt vectors.
4. Fine-Tune the Model: Only the selected PEFT parameters are updated using task-specific data, while the main model remains unchanged.
5. Save the Adaptation: The trained PEFT parameters are saved separately from the base model. The same base model can then be reused with different PEFT parameters for different tasks.

PEFT Techniques for LLMs

PEFT includes several techniques that reduce the number of parameters updated during fine-tuning. Common approaches include:

1. Adapter Modules: Adapters are small trainable neural network modules inserted into a pre-trained model. The original model weights remain frozen while the adapters learn task-specific information.

2. LoRA (Low-Rank Adaptation): LoRA represents weight updates using small low-rank matrices instead of updating the full weight matrices. It is widely used for efficient fine-tuning of language models.

3. DoRA (Weight-Decomposed Low-Rank Adaptation): DoRA extends LoRA by decomposing model weights into magnitude and direction components, allowing the model to learn low-rank updates while separately adapting the weight magnitude.

4. Prefix Tuning: Prefix tuning learns a small set of trainable vectors that are added to the model's inputs at different layers. The original model parameters remain frozen.

5. Prompt Tuning: Prompt tuning learns a small set of continuous (soft) prompt vectors at the input level while keeping the model itself frozen.

6. BitFit: BitFit fine-tunes only the bias parameters of the model while keeping the remaining parameters frozen. This makes it one of the simplest PEFT approaches.

7. (IA)³ (Infused Adapter by Inhibiting and Amplifying Inner Activations): (IA)³ introduces a small number of learnable scaling parameters that modify internal activations. It adapts the model without updating its main weights.

Implementation

Here, we implement PEFT using LoRA on the BERT model for sentiment classification. The IMDb dataset is used to classify movie reviews as positive or negative.

Step 1: Install Required Libraries

Install required libraries Transformers, Datasets, Peft and Accelerate to load the dataset, use the pretrained BERT model and apply LoRA-based fine-tuning.

!pip install -q transformers datasets peft accelerate

Step 2: Load the Model and IMDb Dataset

We use bert-base-uncased as the pretrained model and the IMDb dataset, which contains movie reviews labeled as positive or negative. The tokenizer converts the text into tokens that BERT can process.

from transformers import AutoModelForSequenceClassification, AutoTokenizer, TrainingArguments, Trainer
from datasets import load_dataset
from peft import LoraConfig, get_peft_model

model_name = "bert-base-uncased"

dataset = load_dataset("stanfordnlp/imdb")

tokenizer = AutoTokenizer.from_pretrained(model_name)

Step 3: Preprocess the Dataset

• The reviews must be tokenized before they can be passed to BERT. Each review is truncated or padded to 128 tokens so that the inputs have a consistent length.
• The dataset's label column is renamed to labels, which is the format expected by the Transformers training API.

def preprocess(example):
    return tokenizer(
        example["text"],
        truncation=True,
        padding="max_length",
        max_length=128
    )

encoded_dataset = dataset.map(preprocess, batched=True)

encoded_dataset = encoded_dataset.rename_column("label", "labels")

encoded_dataset.set_format("torch")

Step 4: Configure LoRA

• LoRA adds small trainable matrices to selected layers of the pretrained model instead of updating all of its original parameters.
• Here, LoRA is applied to the query and value layers of BERT's attention mechanism.

lora_config = LoraConfig(
    r=8,
    lora_alpha=16,
    target_modules=["query", "value"],
    lora_dropout=0.05,
    bias="none",
    task_type="SEQ_CLS"
)

Next, we load BERT for sequence classification and apply the LoRA configuration to it.

model = AutoModelForSequenceClassification.from_pretrained(
    model_name,
    num_labels=2
)

model = get_peft_model(model, lora_config)

Step 5: Train the LoRA Model

We use Hugging Face's Trainer to train the model. For demonstration, only 2,000 training examples and 1,000 test examples are used, which keeps the example relatively quick to run.

training_args = TrainingArguments(
    output_dir="./lora-imdb",
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    learning_rate=2e-4,
    num_train_epochs=2,
    report_to="none",
    eval_strategy="epoch",
    save_strategy="epoch"
)

Create the trainer using the selected subsets of the IMDb dataset and start training. After training, save the trained LoRA adapter::

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=encoded_dataset["train"].shuffle(seed=42).select(range(2000)),
    eval_dataset=encoded_dataset["test"].shuffle(seed=42).select(range(1000)),
)

trainer.train()

model.save_pretrained("./lora-imdb-adapter")

Output:

Step 6: Make Predictions

After training, we can use the saved LoRA model to classify new movie reviews. The model returns a label and a confidence score for each prediction.

from transformers import pipeline

sentiment = pipeline(
    "text-classification",
    model="./lora-imdb-adapter",
    tokenizer="bert-base-uncased"
)

label_map = {
    "LABEL_0": "NEGATIVE",
    "LABEL_1": "POSITIVE"
}

examples = [
    "I absolutely loved this film—best sci-fi I've seen in years!",
    "It was okay, not great, but worth a watch.",
    "Terrible plot, terrible acting, total waste of time."
]

for text in examples:
    result = sentiment(text)[0]
    print(
        f"{text[:40]}... → "
        f"{label_map[result['label']]} "
        f"({result['score']:.2f})"
    )

Output:

You can download source code from here.

Full Fine-Tuning vs. PEFT