Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
@@ -42,6 +42,15 @@ That's why we designed simplified architectures, for incremental transformation
|
|
42 |
- **Reactive Transformer** is introducing _Attention-based Memory System_ and adding _Short-Term Memory_ to Transformer language models
|
43 |
- **Preactor** is adding _Long-Term Memory_ and ability to learn from interactions
|
44 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
45 |
### RxT-Alpha Open Research
|
46 |
We are currently working on **Reactive Transformer Proof-of-Concept - RxT-Alpha**, especially on the new reinforcement learning stage - **Memory Reinforcement Learning**,
|
47 |
that's required for our reactive models, between the _Supervised Fine-Tuning_ and _Reinforcement Learning from Human Feedback for reactive models (RxRLHF)_. The research
|
|
|
42 |
- **Reactive Transformer** is introducing _Attention-based Memory System_ and adding _Short-Term Memory_ to Transformer language models
|
43 |
- **Preactor** is adding _Long-Term Memory_ and ability to learn from interactions
|
44 |
|
45 |
+
## RxLM vs LLM advantages
|
46 |
+
Processing single interactions in real-time by **Reactive Language Models** leads to **revolutional** improvements in inference speed/cost:
|
47 |
+
- LLM inference costs are increasing exponentially with conversation length (accumulated for each next message), because of full dialog history processing
|
48 |
+
- RxLM inference costs are linear, depending only on single interaction tokens (not accumulated) - each next interaction is `number of steps` times cheaper than for LLM
|
49 |
+
- same for inference speed - LLM has to process full history, while RxLM only single message (only first interaction could be slower because of encoder/memory attention overhead)
|
50 |
+
|
51 |
+
> In example, for a dialog with **DeepSeek R1**, that have overally ~90k tokens, I paid for about 1.5M tokens. With **RxLM** it will cost only that ~90k tokens, so it
|
52 |
+
> will be about **15x cheaper**
|
53 |
+
|
54 |
### RxT-Alpha Open Research
|
55 |
We are currently working on **Reactive Transformer Proof-of-Concept - RxT-Alpha**, especially on the new reinforcement learning stage - **Memory Reinforcement Learning**,
|
56 |
that's required for our reactive models, between the _Supervised Fine-Tuning_ and _Reinforcement Learning from Human Feedback for reactive models (RxRLHF)_. The research
|