AdamF92 commited on
Commit
3dc72e8
·
verified ·
1 Parent(s): 735cd61

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -66,4 +66,4 @@ capacity (it will be improved in Preactor). Limits are:
66
  We are working on complete Reactive Neural Networks development framework - [RxNN github](https://github.com/RxAI-dev/RxNN)
67
 
68
  ## Additional Research
69
- - **Sparse Query Attention** - the most cost-effective GQA variant, reducing training time/cost by ~3-10% with similar performance. Research in progress
 
66
  We are working on complete Reactive Neural Networks development framework - [RxNN github](https://github.com/RxAI-dev/RxNN)
67
 
68
  ## Additional Research
69
+ - **Sparse Query Attention** - the most cost-effective GQA variant, even 2-3x faster for long sequences! Research in progress - [draft](https://github.com/RxAI-dev/RxNN/blob/main/docs/research/sparse_query_attention.md)