Hugging Face's logo

OpenMOSE
/
ARWKV-7B-Preview-0.1-Pytorch

Model card Files Community
OpenMOSE commited on
Commit
a561677
·
verified ·
1 Parent(s): 1fb98f0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -4,10 +4,14 @@ base_model:
4
  - RWKV-Red-Team/ARWKV-7B-Preview-0.1
5
  ---
6
 
7
- This model only works with RWKV-Infer and RWKV-LM-RLHF
 
8
 
9
  slightly fine-tuned CJE 900k including Deepseek R1 magpie.
10
 
 
 
 
11
  if you want use reasoning mode, set system prompts below
12
 
13
  ```
 
4
  - RWKV-Red-Team/ARWKV-7B-Preview-0.1
5
  ---
6
 
7
+ This model only works with RWKV-Infer and RWKV-LM-RLHF.
8
+ This is a completely early model. No warranty. Use at your own risk.
9
 
10
  slightly fine-tuned CJE 900k including Deepseek R1 magpie.
11
 
12
+ v7 FLA ctx5120, 1k slided overlap training. SFT(with loss smooting=0.001)
13
+ DoRA(total 0.5B parameters), Emb frozen.
14
+
15
  if you want use reasoning mode, set system prompts below
16
 
17
  ```