AtsuMiyai commited on
Commit
869dee6
·
1 Parent(s): 86af462
Files changed (1) hide show
  1. constants.py +3 -0
constants.py CHANGED
@@ -33,6 +33,9 @@ JMMMU consists of **720 translation-based (Culture Agnostic)** and **600 brand-n
33
 
34
 
35
  SUBMIT_INTRODUCTION = """# Submit on JMMMU Benchmark Introduction
 
 
 
36
  1. Obtain Result JSON File from [lmms-eval code base](https://github.com/EvolvingLMMs-Lab/lmms-eval).
37
  2. If you want to update existing model performance by uploading new results, please ensure 'Model Name Revision' is the same as what's shown in the leaderboard. For example, if you want to modify LLaVA-OV 7B's performance, you need to fill in 'LLaVA-OV 7B' in 'Revision Model Name'.
38
  3. Please provide the correct link of your model's repository for each submission.
 
33
 
34
 
35
  SUBMIT_INTRODUCTION = """# Submit on JMMMU Benchmark Introduction
36
+ ※ We do not recommend including results obtained from extensive prompt engineering since it is important to prevent performance hacking and better reflect real-world use cases.
37
+ For more details, please refer to the [lmms-eval code base](https://github.com/EvolvingLMMs-Lab/lmms-eval) and the upcoming paper (coming soon).
38
+
39
  1. Obtain Result JSON File from [lmms-eval code base](https://github.com/EvolvingLMMs-Lab/lmms-eval).
40
  2. If you want to update existing model performance by uploading new results, please ensure 'Model Name Revision' is the same as what's shown in the leaderboard. For example, if you want to modify LLaVA-OV 7B's performance, you need to fill in 'LLaVA-OV 7B' in 'Revision Model Name'.
41
  3. Please provide the correct link of your model's repository for each submission.