This model is part of LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation.
LoRI (LoRA with Reduced Interference) is a simple yet effective variant of Low-Rank Adaptation (LoRA) for fine-tuning Large Language Models (LLMs). It addresses the challenges of overhead and parameter interference in multi-task scenarios by freezing the projection matrices A as random projections and sparsifying the matrices B using task-specific masks. This design substantially reduces the number of trainable parameters (up to 95% fewer than LoRA) while maintaining strong task performance and minimizing cross-task interference. LoRI also supports continual learning by using sparsity to mitigate catastrophic forgetting.
LoRI-S_safety_llama3_rank_64 is a LoRI-S (Sparse) adapter specifically fine-tuned for safety alignment tasks. It is built upon the meta-llama/Meta-Llama-3-8B base model and utilizes a low-rank adapter with r=64. This model is part of the LoRI family of models presented in the paper cited above, demonstrating improved efficiency and reduced cross-task interference in multi-task and continual learning settings.
meta-llama/Meta-Llama-3-8BThis model is intended to be used as a PEFT adapter with the meta-llama/Meta-Llama-3-8B base model for text generation tasks, particularly for improving safety alignment capabilities. It can be loaded with the peft library and combined with the base LLM for efficient inference.
LoRI enables effective adapter merging, allowing multiple task-specific adapters to be combined into a single model while minimizing performance degradation due to cross-task interference. It also supports continual learning scenarios, where the model needs to adapt to new tasks sequentially without suffering from catastrophic forgetting of previously learned tasks.
This model is not intended for use as a standalone LLM; it requires the meta-llama/Meta-Llama-3-8B base model to function. As a fine-tuned model, it may exhibit biases present in its training data or the base model. Users should exercise caution and perform additional evaluations for critical or sensitive applications, and not rely on it for factually accurate or harmless outputs without further safeguards.
While fine-tuned for safety alignment, this LoRI adapter, like all large language models, may still generate harmful, biased, or nonsensical content. Its performance and robustness are highly dependent on the quality and diversity of the safety alignment data it was trained on. Potential risks include:
Users should always:
Install the necessary libraries:
pip install transformers peft
Use the code below to load and use the LoRI adapter:
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
# Load the base model (Meta-Llama-3-8B)
# Ensure you have access to Llama-3 models on Hugging Face
base_model_name = "meta-llama/Meta-Llama-3-8B"
base_model = AutoModelForCausalLM.from_pretrained(
base_model_name,
torch_dtype=torch.bfloat16, # Use bfloat16 for better performance if supported
low_cpu_mem_usage=True,
device_map="auto"
)
# Load the LoRI adapter
adapter_name = "tomg-group-umd/LoRI-S_safety_llama3_rank_64"
model = PeftModel.from_pretrained(base_model, adapter_name)
# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
# Example usage for text generation
prompt = "Explain why it is important for AI models to prioritize user safety."
messages = [
{"role": "user", "content": prompt},
]
input_ids = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
outputs = model.generate(input_ids, max_new_tokens=256, temperature=0.7, do_sample=True)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
LoRI models are trained using a two-stage process: LoRI-D (Dense) and LoRI-S (Sparse). The training is implemented using Fully Sharded Data Parallel (FSDP) and can be executed in a multi-GPU environment.
LoRI models are trained on various datasets depending on the task. For safety alignment, this model was trained using the saferpaca script, which implies a safety-related dataset. More details on the specific datasets used for different tasks (Natural Language Understanding, Code Generation, Mathematical Reasoning, and Safety Alignment) can be found in the LoRI GitHub repository.
A are frozen as random projections and B matrices are dense.LoRI-D training, sparse masks are extracted from the learned B matrices.B matrices to achieve a desired sparsity (e.g., 90% sparsity), while keeping A frozen. This model, LoRI-S_safety_llama3_rank_64, is the result of the LoRI-S training stage.LoRI has been extensively evaluated across various tasks, including natural language understanding, mathematical reasoning, code generation, and safety alignment.
Extensive experiments demonstrate that LoRI outperforms full fine-tuning and existing PEFT methods, while using up to 95% fewer trainable parameters than LoRA. In multi-task experiments, LoRI enables effective adapter merging and continual learning with reduced cross-task interference.
LoRI introduces a novel architecture within the PEFT paradigm. It leverages random projections for matrix A and task-specific sparse masks for matrix B, integrated directly into the training process. The objective is to achieve monosemantic adaptations that reduce cross-task interference and enable efficient multi-task and continual learning.
LoRI can be trained and inferred efficiently in multi-GPU environments, leveraging technologies like Fully Sharded Data Parallel (FSDP).
The implementation builds on PyTorch, Hugging Face Transformers, and PEFT libraries.
If you use LoRI in your work, please cite:
@article{zhang2025lori,
title={LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation},
author={Zhang, Juzheng and You, Jiacheng and Panda, Ashwinee and Goldstein, Tom},
journal={arXiv preprint arXiv:2504.07448},
year={2025}
}
For questions or inquiries, please refer to the contact information on the official LoRI GitHub repository. You may also contact the authors listed in the paper.
Base model
meta-llama/Meta-Llama-3-8B