Update FlexRAG retriever

README.md

@@ -9,9 +9,28 @@ tags:
 - RAG
 ---
 
-# FlexRAG Retriever
+# The BM25SRetriever for the wiki2021 corpus
+
+The corpus was created by the [Atlas](https://github.com/facebookresearch/atlas) project and the index was built using the [FlexRAG](https://github.com/ictnlp/flexrag) library.
+
+| Corpus Attribute | Value                                                           |
+| ---------------- | --------------------------------------------------------------- |
+| Language         | English                                                         |
+| Domain           | Wikipedia                                                       |
+| Size             | 37.5M (33.1M text, 4.3M infobox)                                |
+| Dump Date        | Dec 2021                                                        |
+| Provideer        | [Atlas](https://github.com/facebookresearch/atlas)              |
+| License          | [CC-BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/) |
+
+
+| Index Attribute | Value                                                           |
+| --------------- | --------------------------------------------------------------- |
+| Index Type      | BM25S                                                           |
+| Index Method    | Lucene                                                          |
+| Preprocessing   | LengthFilter(min_char=10, max_char=4096)                        |
+| Provideer       | [FlexRAG](https://github.com/ictnlp/flexrag)                    |
+| License         | [CC-BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/) |
 
-This is a BM25SRetriever created with the [`FlexRAG`](https://github.com/ictnlp/flexrag) library (version `0.1.8`).
 
 ## Installation
 
@@ -35,6 +54,51 @@ retriever = LocalRetriever.load_from_hub("FlexRAG/wiki2021_atlas_bm25s")
 results = retriever.search("Who is Bruce Wayne?")
 ```
 
+## Running the RAG application with the retriever
+
+You can run the **GUI application** of the RAG assistant with this retriever. Here is an example:
+
+```bash
+python -m flexrag.entrypoints.run_interactive \
+    assistant_type=modular \
+    modular_config.used_fields=[title,text] \
+    modular_config.retriever_type="FlexRAG/wiki2021_atlas_bm25s" \
+    modular_config.response_type=original \
+    modular_config.generator_type=openai \
+    modular_config.openai_config.model_name='gpt-4o-mini' \
+    modular_config.openai_config.api_key=$OPENAI_KEY \
+    modular_config.do_sample=False
+```
+
+You can also run the **FlexRAG's RAG evaluation pipeline** with this retriever. Here is an example that evaluates the **ModularAssistant** with the retriever on the *Natural Questions* test split:
+
+```bash
+OUTPUT_PATH=<path_to_output>
+DB_PATH=<path_to_database>
+OPENAI_KEY=<your_openai_key>
+
+python -m flexrag.entrypoints.run_assistant \
+    name=nq \
+    split=test \
+    output_path=${OUTPUT_PATH} \
+    assistant_type=modular \
+    modular_config.used_fields=[title,text] \
+    modular_config.retriever_type="FlexRAG/wiki2021_atlas_bm25s" \
+    modular_config.generator_type=openai \
+    modular_config.openai_config.model_name='gpt-4o-mini' \
+    modular_config.openai_config.api_key=$OPENAI_KEY \
+    modular_config.do_sample=False \
+    eval_config.metrics_type=[retrieval_success_rate,generation_f1,generation_em] \
+    eval_config.retrieval_success_rate_config.context_preprocess.processor_type=[simplify_answer] \
+    eval_config.retrieval_success_rate_config.eval_field=text \
+    eval_config.response_preprocess.processor_type=[simplify_answer]
+```
+
+## License
+As the corpus is based on the [CC-BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/) license, the retriever is also licensed under the same license.
+
+## Related Links
+
 FlexRAG Related Links:
 * 📚[Documentation](https://flexrag.readthedocs.io/en/latest/)
 * 💻[GitHub Repository](https://github.com/ictnlp/flexrag)

config.yaml

@@ -1,6 +1,6 @@
-log_interval: 100
+log_interval: 100000
 top_k: 10
-batch_size: 32
+batch_size: 512
 query_preprocess_pipeline:
   processor_type: []
   length_filter_config:
@@ -32,7 +32,7 @@ query_preprocess_pipeline:
       hf_tokenizer_path: null
       tiktok_tokenizer_name: null
       lang: null
-database_path: ./bm25s_lucene
+database_path: /data/zhangzhuocheng/Lab/Python/LLM/datasets/RAG/wikipedia/wiki_2021/bm25s_lucene
 method: lucene
 idf_method: null
 backend: auto
@@ -40,4 +40,7 @@ k1: 1.5
 b: 0.75
 delta: 0.5
 lang: english
-indexed_fields: null
+indexed_fields:
+- title
+- section
+- text

corpus.jsonl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19ee000506e8b826a4a29076ead7e8978a656184c267b57732de52346710a411
+size 24971177710

corpus.mmindex.json

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f61677c4e0b18a994bbe28d5e5fd54ed57a56108accf9219bb9a97259fb735d1
-size 432224577
+oid sha256:c688cc6edacd0d497a977268659d421eb76b761399e7858601b81c0741c2be8c
+size 433595094

params.index.json

@@ -9,4 +9,4 @@
     "num_docs": 37507469,
     "version": "0.2.1",
     "backend": "numpy"
-}
+}

retriever.id

@@ -0,0 +1 @@
+BM25SRetriever