navodPeiris commited on
Commit
04eb8eb
verified
1 Parent(s): 3b7d6b5

updated readme

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -20,7 +20,7 @@ datasets:
20
  This model is a fine-tuned version of [unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit](https://huggingface.co/unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit).
21
  It has been trained using [TRL](https://github.com/huggingface/trl).
22
 
23
- This model is fine-tuned for detecting vulnerabilities in code.
24
 
25
  Dataset Used: [Mackerel2/cybernative_code_vulnerability_cot](https://huggingface.co/datasets/Mackerel2/cybernative_code_vulnerability_cot)
26
 
@@ -29,7 +29,7 @@ Dataset Used: [Mackerel2/cybernative_code_vulnerability_cot](https://huggingface
29
  - Use for code vulnerability analysis
30
  - Use for general code related question answering (use without given chat template)
31
 
32
- ## Use model with a chat template for Chain of Thoughts response
33
  ```python
34
  import torch
35
  from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
@@ -102,7 +102,7 @@ print("<think>\n" + output)
102
 
103
  ## Training procedure
104
 
105
- This model was trained with SFT Trainer of trl library. I have Leveraged Unsloth鈥檚 FastLanguageModel with 4-bit quantization and smart gradient checkpointing to fit within consumer GPUs. I designed prompts where reasoning is enclosed in <think>...</think> and final answers in <answer>...</answer>. This helps guide the model to reason step-by-step before answering. I have used SFTTrainer from HuggingFace TRL with LoRA + 8bit optimizer + cosine LR scheduling. Evaluation is performed every 50 steps. I have used PEFT/LoRA for efficient fine-tuning.
106
 
107
  | Parameter | Value |
108
  |----------------------------|-------------------------------------|
 
20
  This model is a fine-tuned version of [unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit](https://huggingface.co/unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit).
21
  It has been trained using [TRL](https://github.com/huggingface/trl).
22
 
23
+ This model is fine-tuned for detecting vulnerabilities in code with the Chain-of-Thought method.
24
 
25
  Dataset Used: [Mackerel2/cybernative_code_vulnerability_cot](https://huggingface.co/datasets/Mackerel2/cybernative_code_vulnerability_cot)
26
 
 
29
  - Use for code vulnerability analysis
30
  - Use for general code related question answering (use without given chat template)
31
 
32
+ ## Use model with a chat template for Chain-of-Thought response
33
  ```python
34
  import torch
35
  from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
 
102
 
103
  ## Training procedure
104
 
105
+ This model was trained with SFT Trainer of trl library. I have Leveraged Unsloth鈥檚 FastLanguageModel with 4-bit quantization and smart gradient checkpointing to fit within consumer GPUs. I designed prompts where reasoning is enclosed in \<think>...\</think> and final answers in \<answer>...\</answer>. This helps guide the model to reason step-by-step before answering. I have used SFTTrainer from HuggingFace TRL with LoRA + 8bit optimizer + cosine LR scheduling. Evaluation is performed every 50 steps. I have used PEFT/LoRA for efficient fine-tuning.
106
 
107
  | Parameter | Value |
108
  |----------------------------|-------------------------------------|