Maintain a consistent character across AI image generations

Maintain facial features and distinctive traits of the same character across different camera angles and scenes during AI image generation.

[1] Define a fixed character sheet prompt that details facial attributes, hairstyle, eye color, skin tone, and clothing. Keep these descriptive keywords identical across prompts and place them at the very beginning of every generation prompt.

prompt: A 24-year-old Korean woman, shoulder-length straight black hair, hazel eyes, sharp jawline, wearing a navy blue oversized hoodie, plain white background, neutral lighting, studio portrait, 8k

[2] Generate baseline images and select high-resolution front and three-quarter profile views, then obtain direct image URLs.

https://example.com/images/character-front.pnghttps://example.com/images/character-side.png

[3] In Midjourney, append the --cref parameter followed by the reference image URL to your scene prompt. The default --cw 100 setting copies the face, hair, and clothing directly from the reference.

/imagine prompt: A woman reading a book in a sunlit wooden library, wide shot --cref https://example.com/images/character-front.png

[4] Lower the character weight to --cw 0 when you want to change the character's clothing or hairstyle while keeping facial structure intact.

/imagine prompt: A woman wearing an astronaut space suit inside a spaceship cockpit --cref https://example.com/images/character-front.png --cw 0

[5] When prompt weighting and reference parameters fail to retain fine details, collect 15 to 20 reference images and fine-tune the base model using Hugging Face Diffusers LoRA training scripts.

$ accelerate launch train_text_to_image_lora.py \  --pretrained_model_name_or_path="stable-diffusion-v1-5/stable-diffusion-v1-5" \  --dataset_name="example-character-dataset" \  --dataloader_num_workers=8 \  --resolution=512 \  --center_crop \  --random_flip \  --train_batch_size=1 \  --gradient_accumulation_steps=4 \  --max_train_steps=1500 \  --learning_rate=1e-04 \  --output_dir="character-lora"

[6] Load the trained pytorch_lora_weights.safetensors adapter into a Diffusers pipeline and run inference.

from diffusers import AutoPipelineForText2Imageimport torch pipeline = AutoPipelineForText2Image.from_pretrained(    "stable-diffusion-v1-5/stable-diffusion-v1-5",    torch_dtype=torch.float16,).to("cuda") pipeline.load_lora_weights("character-lora", weight_name="pytorch_lora_weights.safetensors")image = pipeline("A character standing in a neon-lit cyberpunk alleyway").images[0]image.save("result.png")