liuhailin0123/llm-course-hw3-dora

Model Card for Model ID

Model Details

In this work, we fine tuned the base model OuteAI/Lite-Oute-1-300M-Instruct on a tweet sentiment dataset cardiffnlp/tweet_eval dataset to determine tweets tonality in one of the three classes: positive, neutral or negative.

Model Description

We used a system prompt to instruct the model:

SYSTEM PROMPT:

You are a tweet sentiment classifier. For each tweet input, analyze its sentiment and output exactly one word: "negative", "neutral", or "positive". Do not include any extra text.

But the model is not trained to return only the sentiment name.

Different from the previous task, the PEFT method changed to DoRA, which not simply learn a low rank matrix, it trains the magnitute vector and the directional matrix separately. By replacing the k_proj and v_proj layers as we did in the previos task to modify the model.

Training Details

This model was trained with batch_size=16, rank = 8, alpha = 16, learning_rate = 3e-5 for 1 epoch on the first 10000 sample on the dataset.

The model achieved 0.46 macro f1-score on the test dataset, comparing with the initial model which is 0.06.

Comparison

==========
User Prompt: "Ben Smith / Smith (concussion) remains out of the lineup Thursday, Curtis #NHL #SJ"
Label: neutral
Before: The tweet "Ben Smith / Smith (concussion) remains out of the
After: neutral

ralitive

==========
User Prompt: @user Alciato: Bee will invest 150 million in January, another 200 in the Summer and plans to bring Messi by 2017"
Label: positive
Before: The tweet "Alciato: Bee will invest 150
After: neutral

Summary

Comparing to LoRA, DoRA fine-tuning not only allows the model to capture those new features learn in a subspace, but the new parameters can also capture the degree affects. We can see the DoRA shows a better result comparing to the LoRA.