InferBench

Evaluate the quality and efficiency of image gen api's.

Installation

Install dependencies with conda like that:

conda env create -f environment.yml

Install uv with pip like that:

uv venv --python 3.12

uv sync --all-groups

Create .env file with all the credentials you will need.

This is how you can generate the images.

python sample.py replicate draw_bench genai_bench geneval hps parti

This is how you would evaluate the benchmarks once you have all images:

python evaluate.py replicate draw_bench genai_bench geneval hps parti