ajaynegi0079/serene-sdxl-lora

Serene — SDXL LoRA

A custom SDXL LoRA that renders a consistent brand-ambassador likeness named Serene. Trained on a small curated dataset for use as a recurring face across social-media campaigns.

Trigger token

ohwx_serene

Include the trigger anywhere in your prompt. Best results when the trigger appears at the start of the prompt and the prompt clearly describes a single woman as the subject of the frame.

Quick example

ohwx_serene, a portrait photograph of a young woman, soft natural light,
neutral background, looking at the camera, magazine-cover quality,
professional studio lighting on her face.

Use with 🤗 Diffusers

from diffusers import StableDiffusionXLPipeline
import torch

pipe = StableDiffusionXLPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16,
).to("cuda")

pipe.load_lora_weights(
    "ajaynegi0079/serene-sdxl-lora",
    weight_name="pytorch_lora_weights.safetensors",
)

image = pipe(
    "ohwx_serene, portrait, soft studio light, magazine cover quality",
    num_inference_steps=30,
    guidance_scale=7.0,
).images[0]
image.save("serene.png")

Use on Replicate

The repo also ships a Replicate-compatible bundle (serene_lora_v2.tar) for use with cog-style SDXL+LoRA predictors such as fofr/sdxl-multi-controlnet-lora:

{
  "prompt": "ohwx_serene, ...",
  "lora_weights": "https://huggingface.co/ajaynegi0079/serene-sdxl-lora/resolve/main/serene_lora_v2.tar",
  "lora_scale": 0.95,
  "num_inference_steps": 30,
  "scheduler": "K_EULER"
}

Recommended settings

Setting	Value
lora_scale	0.85 – 0.95 (0.95 dominates product context, 0.85 lets the scene breathe)
Inference steps	30
Guidance scale	7.0
Negative prompt	blurry, deformed, extra fingers, bad anatomy, cartoon, plastic skin, watermark, low quality, distorted face

Training

Detail	Value
Base model	stabilityai/stable-diffusion-xl-base-1.0
Method	LoRA (rank 8) via 🤗 PEFT
Resolution	1024 × 1024
Steps	1000
Final loss	~0.017
Hardware	NVIDIA T4 (Google Colab)

Files in this repo

• pytorch_lora_weights.safetensors — raw diffusers/PEFT LoRA, for direct pipe.load_lora_weights() use
• serene_lora_v2.tar — Replicate cog-sdxl bundle (lora.safetensors with cog-style key names + empty embeddings.pti stub)
• preview/serene-1.png … preview/serene-6.png — sample images from the training dataset

License

CC BY-NC 4.0 — non-commercial use only. For commercial / brand-ambassador licensing, contact the repo owner.