Hugging Face's logo
Papers
arxiv:2403.02580

What do we learn from inverting CLIP models?

Published on Mar 5, 2024
Upvote
3
Authors:
Hamid Kazemi ,
Atoosa Chegini ,
,
,

Abstract

Inversion-based analysis of CLIP models reveals semantic alignment with prompts, concept blending, gender biases, and unintended NSFW image generation.

AI-generated summary

We employ an inversion-based approach to examine CLIP models. Our examination reveals that inverting CLIP models results in the generation of images that exhibit semantic alignment with the specified target prompts. We leverage these inverted images to gain insights into various aspects of CLIP models, such as their ability to blend concepts and inclusion of gender biases. We notably observe instances of NSFW (Not Safe For Work) images during model inversion. This phenomenon occurs even for semantically innocuous prompts, like "a beautiful landscape," as well as for prompts involving the names of celebrities.

View arXiv page View PDF Add to collection

Community

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2403.02580 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2403.02580 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2403.02580 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.