AdnanElAssadi commited on
Commit
84410cb
Β·
verified Β·
1 Parent(s): 5ec0494

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -27
README.md CHANGED
@@ -1,27 +1,27 @@
1
- ---
2
- title: MTEB Human Evaluation Demo
3
- emoji: πŸ“Š
4
- colorFrom: blue
5
- colorTo: indigo
6
- sdk: gradio
7
- sdk_version: 3.50.2
8
- app_file: app.py
9
- pinned: false
10
- ---
11
-
12
- # MTEB Human Evaluation Demo
13
-
14
- This is a demo of the human evaluation interface for the MTEB (Massive Text Embedding Benchmark) project. It allows annotators to evaluate the relevance of documents for reranking tasks.
15
-
16
- ## How to use
17
-
18
- 1. Navigate to the "Demo" tab to try the interface with an example dataset (AskUbuntuDupQuestions)
19
- 2. Read the query at the top
20
- 3. For each document, assign a rank using the dropdown (1 = most relevant)
21
- 4. Submit your rankings
22
- 5. Navigate between samples using the Previous/Next buttons
23
- 6. Your annotations are saved automatically
24
-
25
- ## About MTEB Human Evaluation
26
-
27
- This project aims to establish human performance benchmarks for MTEB tasks, helping to understand the realistic "ceiling" for embedding model performance.
 
1
+ ---
2
+ title: MTEB Human Evaluation Demo
3
+ emoji: πŸ“Š
4
+ colorFrom: blue
5
+ colorTo: indigo
6
+ sdk: gradio
7
+ sdk_version: 5.23.3
8
+ app_file: app.py
9
+ pinned: false
10
+ ---
11
+
12
+ # MTEB Human Evaluation Demo
13
+
14
+ This is a demo of the human evaluation interface for the MTEB (Massive Text Embedding Benchmark) project. It allows annotators to evaluate the relevance of documents for reranking tasks.
15
+
16
+ ## How to use
17
+
18
+ 1. Navigate to the "Demo" tab to try the interface with an example dataset (AskUbuntuDupQuestions)
19
+ 2. Read the query at the top
20
+ 3. For each document, assign a rank using the dropdown (1 = most relevant)
21
+ 4. Submit your rankings
22
+ 5. Navigate between samples using the Previous/Next buttons
23
+ 6. Your annotations are saved automatically
24
+
25
+ ## About MTEB Human Evaluation
26
+
27
+ This project aims to establish human performance benchmarks for MTEB tasks, helping to understand the realistic "ceiling" for embedding model performance.