Orensomekh commited on
Commit
a38838b
Β·
verified Β·
1 Parent(s): 719a4a6

Create Live_Challenge_Day_and_Dry_Test_Instructions.md

Browse files
Operational_Instructions/Live_Challenge_Day_and_Dry_Test_Instructions.md ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ **Live Challenge Day (May 12)**
2
+
3
+ 1. We will conduct two live sessions on May 12 as part of the Live Challenge Day (calendar invites will be sent shortly):
4
+ * Session 1: 7:00 – 9:00 UTC
5
+ * Session 2: 15:00 – 17:00 UTC
6
+
7
+ 2. Just before your session start time, your team leader will receive an email containing the Question file (500 questions) β€” see Question file [json schema](Question_File.json.schema) and [example](Question_File_Example.json)
8
+
9
+ 3. You must generate and submit your Answer file β€” see Answer file [json schema](Answe_File.json.schema) and [example](Answer_File_Example.json) β€” within 2 hours from your session start time\
10
+ 3.1 **Remark:** Details regarding the exact submission process will be provided soon
11
+
12
+ 4. Please refer to the LiveRAG Challenge [Evaluation Guidelines](LiveRAG_Evaluation_Guidelines.md) for important information about the evaluation process
13
+
14
+ 5. You must share your RAG system Git repository with us by May 13 AoE via email at sigir2025-liverag-gen@tii.ae to enable result reproduction
15
+
16
+ 6. Additional information:\
17
+ 6.1 The automatic evaluation results leaderboard will be published on the HuggingFace Challenge page once the evaluation process is completed
18
+ 6.2 The top-performing teams will undergo manual evaluation to determine the final winners, who will be announced on July 17 during the SIGIR 2025 LiveRAG Workshop
19
+
20
+ **Dry Test (May 5)**
21
+
22
+ 1. To ensure a smooth Live Challenge Day, we will conduct a Dry Test on May 5 (calendar invites will be sent shortly)
23
+
24
+ 2. Each Dry Test Question file will contain 50 questions
25
+
26
+ 3. Your answers will not be evaluated, and no leaderboard will be published for the Dry Test. Otherwise, instructions 1–4 from the Live Challenge Day also apply