Spaces:
Sleeping
Sleeping
Update LiveRAG_README.md
Browse files- LiveRAG_README.md +17 -17
LiveRAG_README.md
CHANGED
@@ -4,11 +4,11 @@
|
|
4 |
|
5 |
|
6 |
|
7 |
-
|
8 |
|
9 |
|
10 |
|
11 |
-
|
12 |
|
13 |
1. Accessing the AWS LiveRAG account - follow the instructions you received by email\
|
14 |
2. Applying your AWS credits to your team account - follow the instructions you received by email\
|
@@ -17,41 +17,41 @@
|
|
17 |
IF YOU EXCEED YOUR CREDITS, YOU WILL BE DIRECTED CHARGED AND NOT BE REFUNDED BY THE ORGANIZERS</B>
|
18 |
</center>
|
19 |
|
20 |
-
|
21 |
|
22 |
1. Sign up for a [Pinecone](https://pinecone.io ) account, adding a payment method (upgrade to standard). You will not be charged at this time\
|
23 |
2. Send mail sigir2025-liverag-tech@tii.ae with your team information and Pinecone account name and we will apply the $750 credits to your account\
|
24 |
**Remark:** Do you use Pinecone servcies until you see the allocated credits appear in your account, otherwise you will be charged
|
25 |
|
26 |
-
|
27 |
|
28 |
1. Register to [AI71-Platform](https://platform.ai71.ai/register) and send the email address used for registration to sigir2025-liverag-tech@tii.ae\
|
29 |
2. After you're notified that you are approved, login to your [AI71-Platform account](https://platform.ai71.ai)
|
30 |
|
31 |
|
32 |
---
|
33 |
-
|
34 |
|
35 |
-
|
36 |
You can use the pre-built indices, [Pinecone](https://www.pinecone.io/) and [OpenSearch](https://opensearch.org/) or build your own indices on top of the same of [FineWeb Sample-10BT](https://huggingface.co/datasets/HuggingFaceFW/fineweb/viewer/sample-10BT) corpus
|
37 |
1. To use the pre-built indices follow this [sample notebook](Operational_Instructions/Indices_Usage_Examples_for_LiveRAG.ipynb)
|
38 |
2. To build your own **OpenSearch** index, create or use your existing AWS account and apply the provided AWS credits to save costs (see [here](Operational_Instructions/AWS_Accounts_for_LiveRAG.md))\
|
39 |
3. To build your own **Pinecone** index, create or use your existing Pinecone account and apply the provided Pinecone credits to save costs (see [here](Operational_Instructions/Pinecone_for_LiveRAG.md))
|
40 |
|
41 |
-
|
42 |
-
|
43 |
-
2.
|
44 |
-
|
45 |
-
|
46 |
-
|
47 |
**Remark:** Accessing Falcon via the AI71-Platform API is optional and recommended for a quick start. However, for a robust solution, suitable for the live event, it is recommended to set up your own Falcon3-10B-Instruct instance
|
48 |
|
49 |
-
3.Building, training, and evaluating your RAG system\
|
50 |
-
|
51 |
-
|
52 |
-
3.
|
53 |
|
54 |
-
4. LiveRAG live event details will be added soon...
|
55 |
|
56 |
You may contact the organizers via:
|
57 |
- sigir2025-liverag-gen@tii.ae - for general issues
|
|
|
4 |
|
5 |
|
6 |
|
7 |
+
# 1. π£ LiveRAG Challenge Onboarding
|
8 |
|
9 |
|
10 |
|
11 |
+
## 1.1 Setting up your AWS accounts - See [AWS Accounts instructions - section 1](Operational_Instructions/AWS_Accounts_for_LiveRAG.md) for the full details
|
12 |
|
13 |
1. Accessing the AWS LiveRAG account - follow the instructions you received by email\
|
14 |
2. Applying your AWS credits to your team account - follow the instructions you received by email\
|
|
|
17 |
IF YOU EXCEED YOUR CREDITS, YOU WILL BE DIRECTED CHARGED AND NOT BE REFUNDED BY THE ORGANIZERS</B>
|
18 |
</center>
|
19 |
|
20 |
+
## 1.2 Setting your Pinecone account if you want to build your own dense index
|
21 |
|
22 |
1. Sign up for a [Pinecone](https://pinecone.io ) account, adding a payment method (upgrade to standard). You will not be charged at this time\
|
23 |
2. Send mail sigir2025-liverag-tech@tii.ae with your team information and Pinecone account name and we will apply the $750 credits to your account\
|
24 |
**Remark:** Do you use Pinecone servcies until you see the allocated credits appear in your account, otherwise you will be charged
|
25 |
|
26 |
+
## 1.3. Register to AI71-Platform to access DataMorgana and Falcon3-10B-Instruct LLM
|
27 |
|
28 |
1. Register to [AI71-Platform](https://platform.ai71.ai/register) and send the email address used for registration to sigir2025-liverag-tech@tii.ae\
|
29 |
2. After you're notified that you are approved, login to your [AI71-Platform account](https://platform.ai71.ai)
|
30 |
|
31 |
|
32 |
---
|
33 |
+
# 2. π£ LiveRAG Challenge Step-by-Step Guide
|
34 |
|
35 |
+
## 2.1 Accessing or building your RAG indices\
|
36 |
You can use the pre-built indices, [Pinecone](https://www.pinecone.io/) and [OpenSearch](https://opensearch.org/) or build your own indices on top of the same of [FineWeb Sample-10BT](https://huggingface.co/datasets/HuggingFaceFW/fineweb/viewer/sample-10BT) corpus
|
37 |
1. To use the pre-built indices follow this [sample notebook](Operational_Instructions/Indices_Usage_Examples_for_LiveRAG.ipynb)
|
38 |
2. To build your own **OpenSearch** index, create or use your existing AWS account and apply the provided AWS credits to save costs (see [here](Operational_Instructions/AWS_Accounts_for_LiveRAG.md))\
|
39 |
3. To build your own **Pinecone** index, create or use your existing Pinecone account and apply the provided Pinecone credits to save costs (see [here](Operational_Instructions/Pinecone_for_LiveRAG.md))
|
40 |
|
41 |
+
## 2.2 Accessing DataMorgana and Falcon3-10B-Instruct via AI71-Platform
|
42 |
+
1. Login to your [AI71-Platform account](https://platform.ai71.ai) after you followed the onboarding instructions
|
43 |
+
2. Access the DataMorgana [Sandbox](https://platform.ai71.ai/playground) and [try it](Operational_Instructions/DM_Sandbox.md) to get your first Q&A\
|
44 |
+
3. Create your AI71-Platform [API key](https://platform.ai71.ai/api-keys)
|
45 |
+
3.1 Use your AI71-Platform key for programatic access to DataMorgana bulk requests generation via its [API](Operational_Instructions/DM_API_usage_example.ipynb)\
|
46 |
+
3.2 Use your AI71-Platform key for programatic access to Falcon3-10B-Instruct (see [AI71 documentation](https://platform.ai71.ai/documentation))\
|
47 |
**Remark:** Accessing Falcon via the AI71-Platform API is optional and recommended for a quick start. However, for a robust solution, suitable for the live event, it is recommended to set up your own Falcon3-10B-Instruct instance
|
48 |
|
49 |
+
## 2.3. Building, training, and evaluating your RAG system\
|
50 |
+
1. Build your RAG system in your AWS Team account leveraging your FineWeb indices - make sure to use Falcon3-10B-Instruct to generate answers
|
51 |
+
2. Train your RAG system using synthetic Q&A pairs generated by DataMorgana [API](Operational_Instructions/DM_API_usage_example.ipynb)
|
52 |
+
3. Evaluate your RAG system using the [evaluation guidelines](Operational_Instructions/Evaluation_Guidelines_for_LiveRAG.md)
|
53 |
|
54 |
+
## 4. LiveRAG live event details will be added soon...
|
55 |
|
56 |
You may contact the organizers via:
|
57 |
- sigir2025-liverag-gen@tii.ae - for general issues
|