LoRAs and Style Consistency (What Prompts Can't Do Alone)

TL;DR: A LoRA (Low-Rank Adaptation) is a small trained file that changes a diffusion model's weights, so its style or character applies automatically to every generation. A prompt, even a Midjourney style code or reference image, is a fresh instruction each time. LoRA prompting means writing prompts to activate a trained file; Midjourney, ChatGPT, and Gemini don't support installing one at all.

What Is a LoRA, Actually?

LoRA stands for Low-Rank Adaptation, and the term comes from a 2021 paper by Edward Hu and colleagues at Microsoft, written for large language models, not image generators. The paper states its method plainly: LoRA "freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture, greatly reducing the number of trainable parameters for downstream tasks." Applied to GPT-3 175B, the paper reports the technique "can reduce the number of trainable parameters by 10,000 times and the GPU memory requirement by 3 times" compared to full fine-tuning.

The image-generation community later adapted the same idea to diffusion models, and it stuck because the output is small and portable. Hugging Face's own diffusers documentation describes LoRA as working by "inserting a smaller number of new weights into the model and only these are trained," producing "smaller model weights (a few hundred MBs), which are easier to store and share" than a fully fine-tuned checkpoint. Automatic1111's WebUI, the most widely used local Stable Diffusion interface, documents the same technique in its own wiki as a way "to fine tune weights for CLIP and Unet, the language model and the actual image de-noiser used by Stable Diffusion."

Practically, a LoRA is a .safetensors file, usually somewhere between 10MB and 200MB depending on rank, trained on a specific style, character, or object using a small set of images. Load it alongside a base model and it changes what that base model tends to produce, on every generation, until you unload it.

Why Can't a Great Prompt Do What a LoRA Does?

Because a prompt and a LoRA operate on two different things. A prompt, no matter how carefully engineered, is an input you send to a model that already exists in a fixed state. A LoRA is a modification to that model's actual weights, layered on top of the frozen original. One is conditioning; the other is a small, persistent change to the machine doing the generating.

This is exactly why reference-image and style-code features, the tools people reach for first when they want consistency, top out where they do. Midjourney's Style Reference (--sref) can carry a look across a session, and its Edit Model now handles character-consistency tasks that used to sit under separate Character Reference and Omni Reference flags, but none of it is a file you train, export, or inspect. Our own deep dive on Midjourney's --sref system and on --cref and --oref character consistency covers the mechanics in full; the short version is that a style code is a pointer into Midjourney's own hosted style space, not a portable artifact you own.

Style LoRA, Character LoRA, or Concept LoRA: What's the Difference?

Black Forest Labs' training documentation for FLUX.2 groups LoRA use cases into four buckets: style transfer, character consistency, domain specialization, and concept learning. It describes the first as: "Create custom artistic styles that can be applied to any subject matter," and the second as: "Train models to generate specific characters or people with consistent features across different scenes and poses." In practice, almost every LoRA you'll encounter is one of these first two.

The distinction matters for how you train it, not just what you call it. fal.ai's training service for FLUX LoRAs exposes an explicit is_style boolean parameter. Its own documentation instructs: "Use the is_style: true parameter for style training to disable auto-captioning and segmentation masks that aren't needed for artistic style transfer." A style LoRA is trying to learn a look independent of subject matter, so captions that over-describe the subject work against it. A character LoRA needs the opposite: captions that describe everything except the identity you're trying to lock in, so the model doesn't accidentally bind the character to a specific pose or outfit. Black Forest Labs recommends lower learning rates (8e-5 to roughly 1e-4) for style and pushes higher within that range for characters, with character LoRAs typically needing more training steps, 1,500 to 3,000 against 1,500 to 2,500 for style.

Do Midjourney, ChatGPT, and Gemini Support Custom LoRAs?

No, and this is the gap that trips people up when they search for "lora prompting" and expect it to apply everywhere. None of the three major consumer image tools expose a way to upload or install a trained LoRA.

Midjourney has no customer-facing model training path; its style tools operate entirely inside its own hosted system. OpenAI's image-generation guide names character consistency as a real limitation, stating plainly that "the model may occasionally struggle to maintain visual consistency for recurring characters or brand elements across multiple generations." OpenAI's own cookbook, describing the same model family, claims the opposite: "Robust facial and identity preservation for edits, character consistency, and multi-step workflows." Both are OpenAI's own published pages; quote whichever you're citing, because they disagree with each other. Google publishes no equivalent admission for its Nano Banana models. What it does publish is per-model reference-image caps, and one specific line about Nano Banana 2 Lite: "Not optimized for multiple reference inputs or multi-turn sequential editing." None of that is a LoRA-upload path either.

Where custom LoRAs genuinely work is API-first and open pipelines: Stable Diffusion and FLUX.2 through ComfyUI or Automatic1111 locally, or hosted training and inference through Black Forest Labs' own dashboard, Replicate, or fal.ai. That's a real, load-bearing distinction, not a technicality.

How Do You Actually Write a Prompt for a LoRA?

Once a LoRA is trained and loaded, prompting for it is mostly the same discipline as any other prompt, plus one addition: the trigger word, and, in local tools, a weight tag.

In Automatic1111 and most ComfyUI setups, the syntax is documented in Automatic1111's own wiki as <lora:filename:multiplier>, where the multiplier is a weight defaulting to 1. A prompt using a character LoRA trained with the trigger word ohwx might look like this:

photo of ohwx man, standing in a quiet garden, overcast light, medium shot
<lora:my_character_v1:0.8>

The trigger word does the identity work; the rest of the prompt still does everything a prompt normally does, framing, lighting, pose, environment. Lowering the weight toward 0.6 tends to help when the LoRA is fighting the rest of your prompt for control of the composition; pushing it above 1 makes the trained style or likeness more dominant, at the cost of flexibility elsewhere in the scene.

Hosted inference looks similar in spirit. Black Forest Labs' own API takes a finetune_id and a finetune_strength parameter alongside the prompt:

response = requests.post(
    "https://api.bfl.ai/v1/flux-2-klein-9b-kv-finetuned",
    headers={"x-key": BFL_API_KEY, "Content-Type": "application/json"},
    json={
        "prompt": "a photo of ohwx in a garden on a sunny day",
        "finetune_id": "your-lora-id",
        "finetune_strength": 1.0,
    },
)

Same idea as the bracket tag, different transport: the trigger word activates the identity or style, a strength parameter dials how hard it's applied, and the rest of the prompt is still doing normal prompting work.

How Many Images Do You Need to Train One?

Fewer than most people assume, and the exact number matters less than consistency. Replicate's guide to fine-tuning FLUX.1 with your own images states directly: "Use 12-20 images for best results," alongside the usual advice to vary poses, settings, and lighting rather than reusing near-identical shots. fal.ai's training service accepts a steps parameter that defaults to 1,000 and recommends adjusting from there: "too few steps may not capture the style, while too many can lead to overfitting."

Black Forest Labs' FLUX.2 klein guide is more specific by use case: style LoRAs at 1,500-2,500 training steps, character LoRAs at 1,500-3,000, both trainable on a single consumer GPU with 12GB or more of VRAM in one to three hours. None of these figures are the whole picture on their own; a badly curated set of 20 images (inconsistent quality, repeated backgrounds, a wrong crop ratio) trains worse than a well-curated set of 12.

DreamBooth vs LoRA vs Textual Inversion: Which One Do You Actually Need?

These three techniques get conflated constantly, and they involve genuinely different amounts of fine-tuning. DreamBooth, from Google Research's original paper, works by taking "as input just a few images of a subject" and using them to "fine-tune a pretrained text-to-image model such that it learns to bind a unique identifier with that specific subject." That's a full-model fine-tune, historically the heaviest of the three in file size and training time.

Textual Inversion, published the same year, works completely differently: it doesn't touch the model's weights at all. Its paper describes the method as learning to represent a new concept through a placeholder embedding in a frozen text-to-image model's existing vocabulary, using "only 3-5 images" of that concept. It's the lightest of the three and the least expressive, because it can only steer a frozen model toward a concept it can already sort of express, rather than teaching the model something genuinely new.

LoRA sits in between by design: it freezes the base weights like Textual Inversion does, but injects new trainable matrices into the model's layers rather than only adjusting a text embedding, which is why it can capture more than Textual Inversion while staying far lighter than a full DreamBooth fine-tune. Hugging Face's diffusers documentation notes the two aren't mutually exclusive either: "LoRA can also be combined with other training techniques like DreamBooth to speedup training," which is how most character LoRAs you'll find on community sites are actually made.

When Do You Actually Need a LoRA Instead of a Better Prompt?

If you need one or a handful of images with a consistent look, a strong prompt plus a reference image (or, on Midjourney, a style code) usually gets you there without any training run. Our reasoning models vs chat models piece and why Claude does things ChatGPT won't cover a related honest-limits theme on the text side: a lot of what looks like a missing feature is actually a documented, structural boundary, not a prompting mistake.

The same honesty applies here. If you're producing a small ongoing set, a comic, a brand's product shots, a game's character portraits, and it genuinely needs the same face or house style across dozens or hundreds of generations without you re-supplying a reference each time, a LoRA is the tool built for that job, and no amount of prompt engineering closes the gap, because a prompt simply isn't the kind of object that persists between calls. For everything else, including the broader question of what AI images still can't reliably produce without a trained or structural aid, see our companion piece on diagrams and infographics.

Where Prompt Architects Fits (and Where It Doesn't)

Being direct about this: Prompt Architects doesn't generate images, train models, or host LoRA files, and nothing here changes that. If you need a LoRA trained, that work happens in ComfyUI, Automatic1111, or a hosted trainer like Black Forest Labs, Replicate, or fal.ai.

What it's genuinely useful for is the prompting half of a LoRA-based workflow, which is still real, recurring work. Once a LoRA is trained, you're writing the same shape of prompt over and over: trigger word, subject description, scene, weight, repeated across dozens of shots for a project. That's exactly the kind of reusable structure a prompt library and variables are built for, keeping your trigger word and caption phrasing consistent instead of retyping (or slightly misremembering) them every time.