Article
13
Marian Kannwischer
canwiper
AI & ML interests
RLHF & Computer Vision
Recent Activity
reacted
to
jasoncorkill's
post
with 👀
8 days ago
Benchmark Update: @google Veo3 (Text-to-Video)
Two months ago, we benchmarked @google’s Veo2 model. It fell short, struggling with style consistency and temporal coherence, trailing behind Runway, Pika, @tencent, and even @alibaba-pai.
That’s changed.
We just wrapped up benchmarking Veo3, and the improvements are substantial. It outperformed every other model by a wide margin across all key metrics. Not just better, dominating across style, coherence, and prompt adherence. It's rare to see such a clear lead in today’s hyper-competitive T2V landscape.
Dataset coming soon. Stay tuned.
reacted
to
jasoncorkill's
post
with ❤️
20 days ago
🔥 Hidream I1 is online! 🔥
We just added Hidream I1 to our T2I leaderboard (https://www.rapidata.ai/leaderboard/image-models) benchmarked using 195k+ human responses from 38k+ annotators, all collected in under 24 hours.
It landed #3 overall, right behind:
- @openai 4o
- @black-forest-labs Flux 1 Pro
...and just ahead of @black-forest-labs Flux 1.1 Pro, @xai-org Aurora and @google Imagen3.
Want to dig into the data? Check out our dataset here:
https://huggingface.co/datasets/Rapidata/Hidream_t2i_human_preference
What model should we benchmark next?
reacted
to
jasoncorkill's
post
with ❤️
about 1 month ago
🚀 Building Better Evaluations: 32K Image Annotations Now Available
Today, we're releasing an expanded version: 32K images annotated with 3.7M responses from over 300K individuals which was completed in under two weeks using the Rapidata Python API.
https://huggingface.co/datasets/Rapidata/text-2-image-Rich-Human-Feedback-32k
A few months ago, we published one of our most liked dataset with 13K images based on the @data-is-better-together's dataset, following Google's research on "Rich Human Feedback for Text-to-Image Generation" (https://arxiv.org/abs/2312.10240). It collected over 1.5M responses from 150K+ participants.
https://huggingface.co/datasets/Rapidata/text-2-image-Rich-Human-Feedback
In the examples below, users highlighted words from prompts that were not correctly depicted in the generated images. Higher word scores indicate more frequent issues. If an image captured the prompt accurately, users could select [No_mistakes].
We're continuing to work on large-scale human feedback and model evaluation. If you're working on related research and need large, high-quality annotations, feel free to get in touch: info@rapidata.ai.
Organizations
Articles
2
Article
18
Crowd-sourced Open Preference Dataset for Text-to-Image Generation
spaces
1
models
0
None public yet
datasets
0
None public yet