Update README.md
Browse files
README.md
CHANGED
@@ -4,10 +4,10 @@ language:
|
|
4 |
- en
|
5 |
---
|
6 |
|
7 |
-
This is trained
|
8 |
-
Codes with
|
9 |
|
10 |
-
|
11 |
|
12 |
The 1st to 17th dimensions of S3AE hidden features, respectively, correspond to activations of the following thoughts:
|
13 |
|
|
|
4 |
- en
|
5 |
---
|
6 |
|
7 |
+
This is trained parameters of the **S**entence-level, **S**upervised, **S**parse **A**uto**E**ncoder (S3AE) proposed in the paper ["Emergence of psychopathological computations in large language models"](https://arxiv.org/abs/2504.08016).
|
8 |
+
Codes with S3AE architecture and use examples can be found in this [Github](https://github.com/syleeheal/Machine_Psychopathology).
|
9 |
|
10 |
+
S3AE was trained on the residual stream in the 10th layer of instruction-tuned [Gemma 2 27B](https://huggingface.co/google/gemma-2-27b-it), using a proprietary synthetic dataset with psychopathology symptom labels. The model weight precision is bfloat16, and the hidden dimension size is 8 times that of the LLM residual stream.
|
11 |
|
12 |
The 1st to 17th dimensions of S3AE hidden features, respectively, correspond to activations of the following thoughts:
|
13 |
|