LEAF-CLIP/OpenCLIP-ViT-g-rho50-k1-constrained-FARE2

[Paper]   [Code]

Model Initialized from laion/CLIP-ViT-g-14-laion2B-s34B-b88K. The image encoder is finetuned with FARE at $\epsilon=2/255$. The text encoder is finetuned with LEAF at $k=1$ with $\rho=50$ and semantic constraints.

To load this model use:

from transformers import CLIPProcessor, CLIPModel

model_name = "LEAF-CLIP/OpenCLIP-ViT-g-rho50-k1-constrained-FARE2"
processor_name = "laion/CLIP-ViT-g-14-laion2B-s12B-b42K"

model = CLIPModel.from_pretrained(model_name)
processor = CLIPProcessor.from_pretrained(processor_name)