metadata

license: other
base_model: HiDream-ai/HiDream-I1-Full
tags:
  - hidream
  - hidream-diffusers
  - text-to-image
  - image-to-image
  - diffusers
  - simpletuner
  - not-for-all-audiences
  - lora
  - template:sd-lora
  - standard
pipeline_tag: text-to-image
inference: true
widget:
  - text: unconditional (blank prompt)
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_0_0.png
  - text: A naked woman is taking a selfie in the mirror.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_1_0.png
  - text: >-
      A group of naked people are playing Curling in the 1972 Olympic curling
      event
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_2_0.png
  - text: A naked woman is lying in the grass, smiling.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_3_0.png
  - text: A naked man is lying in the grass.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_4_0.png
  - text: A naked man is taking a selfie in the mirror.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_5_0.png
  - text: A naked man is riding a giraffe at the zoo. Police are chasing behind him.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_6_0.png
  - text: >-
      A naked ghost man is going trick-or-treating on halloween. He is holding
      up a sign that reads 'Uncensored HiDream, now on Runware'
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_7_0.png
  - text: >-
      A naked ghost woman is going trick-or-treating on halloween. She is
      holding up a sign that reads 'Uncensored HiDream, now on Runware'
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_8_0.png
  - text: >-
      A psychedelic naked goddess with many hindu arms is blessing the animals
      of Noah's arc
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_9_0.png
  - text: >-
      A woman with huge boobs is laughing, wearing Juggalos make-up at a camping
      festival in New Jersey, 2008
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_10_0.png
  - text: >-
      A beautiful woman with large breasts is making out with her girlfriend.
      They have an aire of myseriousness, as they kiss deeply in the centre of
      the shopping mall food court.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_11_0.png
  - text: A photograph of a cat lounging on a sofa in a sunroom on a 1974 sunny day.
    parameters:
      negative_prompt: blurry, cropped, ugly
    output:
      url: ./assets/image_12_0.png

hidream-reddit

This is a standard PEFT LoRA derived from HiDream-ai/HiDream-I1-Full.

The main validation prompt used during training was:

A photograph of a cat lounging on a sofa in a sunroom on a 1974 sunny day.

Validation settings

CFG: 3.0
CFG Rescale: 0.0
Steps: 28
Sampler: FlowMatchEulerDiscreteScheduler
Seed: 42
Resolution: 1024x1024

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt
unconditional (blank prompt)

Negative Prompt
blurry, cropped, ugly

Prompt
A naked woman is taking a selfie in the mirror.

Negative Prompt
blurry, cropped, ugly

Prompt
A group of naked people are playing Curling in the 1972 Olympic curling event

Negative Prompt
blurry, cropped, ugly

Prompt
A naked woman is lying in the grass, smiling.

Negative Prompt
blurry, cropped, ugly

Prompt
A naked man is lying in the grass.

Negative Prompt
blurry, cropped, ugly

Prompt
A naked man is taking a selfie in the mirror.

Negative Prompt
blurry, cropped, ugly

Prompt
A naked man is riding a giraffe at the zoo. Police are chasing behind him.

Negative Prompt
blurry, cropped, ugly

Prompt
A naked ghost man is going trick-or-treating on halloween. He is holding up a sign that reads 'Uncensored HiDream, now on Runware'

Negative Prompt
blurry, cropped, ugly

Prompt
A naked ghost woman is going trick-or-treating on halloween. She is holding up a sign that reads 'Uncensored HiDream, now on Runware'

Negative Prompt
blurry, cropped, ugly

Prompt
A psychedelic naked goddess with many hindu arms is blessing the animals of Noah's arc

Negative Prompt
blurry, cropped, ugly

Prompt
A woman with huge boobs is laughing, wearing Juggalos make-up at a camping festival in New Jersey, 2008

Negative Prompt
blurry, cropped, ugly

Prompt
A beautiful woman with large breasts is making out with her girlfriend. They have an aire of myseriousness, as they kiss deeply in the centre of the shopping mall food court.

Negative Prompt
blurry, cropped, ugly

Prompt
A photograph of a cat lounging on a sofa in a sunroom on a 1974 sunny day.

Negative Prompt
blurry, cropped, ugly

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

Training epochs: 7
Training steps: 60500
Learning rate: 1.0
- Learning rate schedule: constant
- Warmup steps: 500
Max grad value: 0.1
Effective batch size: 32
- Micro-batch size: 4
- Gradient accumulation steps: 1
- Number of GPUs: 8
Gradient checkpointing: True
Prediction type: flow_matching (extra parameters=['flow_schedule_auto_shift', 'shift=0.0'])
Optimizer: prodigy
Trainable parameter precision: Pure BF16
Base model precision: no_change
Caption dropout probability: 10.0%
LoRA Rank: 128
LoRA Alpha: 128.0
LoRA Dropout: 0.1
LoRA initialisation style: default

Datasets

photo10k

Repeats: 0
Total number of images: ~10000
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: random
Crop aspect: square
Used for regularisation data: No

hidream-shutterstock

Repeats: 0
Total number of images: ~21056
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: random
Crop aspect: square
Used for regularisation data: No

hidream-normalnudes

Repeats: 0
Total number of images: ~1040
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: random
Crop aspect: square
Used for regularisation data: No

hidream-nsfw

Repeats: 0
Total number of images: ~10792
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: random
Crop aspect: square
Used for regularisation data: No

hidream-gay

Repeats: 0
Total number of images: ~1104
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: random
Crop aspect: square
Used for regularisation data: No

hidream-nijijourney

Repeats: 0
Total number of images: ~21504
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: random
Crop aspect: square
Used for regularisation data: No

hidream-bg20k-1024

Repeats: 0
Total number of images: ~89304
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: random
Crop aspect: square
Used for regularisation data: No

hidream-photo-aesthetics

Repeats: 0
Total number of images: ~33120
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: random
Crop aspect: square
Used for regularisation data: No

hidream-text-1mp

Repeats: 5
Total number of images: ~13192
Total number of aspect buckets: 1
Resolution: 1.048576 megapixels
Cropped: True
Crop style: random
Crop aspect: square
Used for regularisation data: No

Inference

import torch
from diffusers import DiffusionPipeline

model_id = 'HiDream-ai/HiDream-I1-Full'
adapter_id = 'bghira/hidream-reddit'
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
pipeline.load_lora_weights(adapter_id)

prompt = "A photograph of a cat lounging on a sofa in a sunroom on a 1974 sunny day."
negative_prompt = 'blurry, cropped, ugly'

## Optional: quantise the model to save on vram.
## Note: The model was not quantised during training, so it is not necessary to quantise it during inference time.
#from optimum.quanto import quantize, freeze, qint8
#quantize(pipeline.transformer, weights=qint8)
#freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
model_output = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=28,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(42),
    width=1024,
    height=1024,
    guidance_scale=3.0,
).images[0]

model_output.save("output.png", format="PNG")

Exponential Moving Average (EMA)

SimpleTuner generates a safetensors variant of the EMA weights and a pt file.

The safetensors file is intended to be used for inference, and the pt file is for continuing finetuning.

The EMA model may provide a more well-rounded result, but typically will feel undertrained compared to the full model as it is a running decayed average of the model weights.