Structure Control: ControlNet-Style Guidance in Prompts

TL;DR: "ControlNet prompting" is a category error for every hosted API checked here. ControlNet is a separate, open-source conditioning network built for self-hosted Stable Diffusion, not a phrase you type. Google, Ideogram, GPT Image 2, and FLUX each publish reference images or a mask instead, none of it pixel-aligned. That honest map, not a fake syntax, is what actually helps.

Type pose control, depth map, or canny edge into a hosted image generator's prompt box and nothing structural happens. The words get treated like any other prompt text, competing with your subject description for the model's attention, with no separate channel carrying a skeleton or an edge drawing into the network. That is not a bug in any one tool. It is what happens when a term coined for one specific open-source architecture gets carried into a market of hosted APIs that were never built to expose it.

What Is ControlNet, Actually?

A separate neural network, not a prompting trick. The paper that introduced it, "Adding Conditional Control to Text-to-Image Diffusion Models" by Lvmin Zhang, Anyi Rao, and Maneesh Agrawala, describes ControlNet as "a neural network architecture to add spatial conditioning controls to large, pretrained text-to-image diffusion models." It works by locking the weights of an existing Stable Diffusion model and attaching a second, trainable copy of its encoder, connected through zero-initialized convolutions so early training cannot damage the base model. The paper tested it with "edges, depth, segmentation, human pose, etc" as the conditioning input.

The part that matters for a prompt-only workflow: none of that conditioning arrives as text. It arrives as an image, an edge map traced from a photo, a depth map rendered by a separate estimator model, a pose skeleton extracted by a body-tracking model, aligned pixel-for-pixel with the output you want. ControlNet's job is to make the diffusion model respect that second image's geometry while your prompt still supplies the content. Take away that second image and the pixel-aligned mechanism, and what is left is just Stable Diffusion with an extra, unused input.

What Does Google's Gemini Image API Actually Give You?

Two documented fields, and neither is structural. The Gemini API reference publishes an ImageConfig object with exactly aspectRatio (a string enum covering ratios like 1:1, 16:9, and 21:9) and imageSize (512, 1K, 2K, or 4K). A seed field exists, but only on the separate, shared GenerationConfig object one level up, and Google's own image-generation guide never mentions the word seed at all. There is no published image-reproducibility contract here, let alone a structural one.

What Gemini does offer instead is reference images, and the caps differ sharply by model:

That is a count of how many pictures you can hand the model as inspiration, not a control over where a subject's arm ends up in the output. Google's own guide is explicit about its fastest tier: Nano Banana 2 Lite is "Not optimized for multiple reference inputs or multi-turn sequential editing." That rules it out for anything that depends on continuity across generations, structural or otherwise.

Does Ideogram Let You Lock Down Composition?

Not even your literal dimensions are guaranteed, which is worth knowing before you ask it for anything more precise than that. Ideogram's own aspect-ratio documentation states: "When custom dimensions are available, Ideogram may normalize your requested dimensions to a supported ratio, model size, or resolution tier." That is three separate silent adjustments in one sentence, to the ratio, the model's internal size bucket, and the resolution tier, on a vendor that actually documents doing it rather than leaving it as user folklore. If a generation keeps coming back at a size you did not ask for, our diagnostic on wrong-sized image output covers this same behavior in more depth, across more vendors than just Ideogram.

What Ideogram does offer is Character Reference, and its own docs frame the problem it solves in almost the same language this whole topic keeps circling back to: "Creating visually consistent characters across multiple images has long been a challenge in AI image generation". Character Reference lets you feed in a photo and reuse it across new prompts so facial features and traits carry over. That is continuity for one character across a series, not a spatial map that pins a pose or a camera angle in a single generation.

What About GPT Image 2's Masks and Reference Images?

Closer than most, but still not structural. OpenAI's own guide is direct: "Masking with GPT Image is entirely prompt-based. The model uses the mask as guidance, but may not follow its exact shape with complete precision." A mask marks the region you want changed and leaves the rest alone; it does not specify the geometry that should appear inside that region the way a pose skeleton or a depth map would.

The same guide names the adjacent gap under its own Composition Control heading: "Despite improved instruction following, the model may have difficulty placing elements precisely in structured or layout-sensitive compositions." Size, by contrast, is genuinely constraint-validated rather than a fixed enum. The size field on gpt-image-2 publishes four numeric rules, each stated in the API guide's own words:

• "Maximum edge length must be less than or equal to 3840px"
• "Both edges must be multiples of 16px"
• "Long edge to short edge ratio must not exceed 3:1"
• "Total pixels must be at least 655,360 and no more than 8,294,400"

That is real, checkable precision on the canvas shape. It is still not a way to specify where a hand ends up inside it.

Does FLUX Expose Anything Like ControlNet?

Its own OpenAPI spec settles this cleanly: grepping the full schema for controlnet, canny, depth, pose, openpose, scribble, and segmentation returns zero hits, on every FLUX.2 and FLUX 3 endpoint published. Flux2Inputs, the schema behind flux-2-pro and flux-2-max, does not even publish an aspect_ratio field, only width and height in pixels, plus up to eight numbered input_image slots for reference images. There is no separate structural channel hiding in there.

Version matters here in a way that trips people up. FLUX 1.1 ultra and Kontext do publish aspect_ratio, but it is typed as a plain string with a description reading "Aspect ratio of the image between 21:9 and 9:21", not a fixed enum of named ratios. That is a documented range, not a menu, and it still says nothing about pose or edges.

What Does Midjourney Offer Instead?

The same substitution, in its own words. Midjourney's own Image Prompts documentation describes the feature this way: "Image Prompts let you guide Midjourney by including an image along with your text prompt. Midjourney looks at your image's core elements, and uses it as a source of inspiration for your new and unique image." Inspiration, not conditioning. There is no separate edge, depth, or pose input anywhere in that flow.

The closest thing Midjourney has to a mask is Vary Region, which lets you select part of an already-generated image and regenerate just that area while the rest stays untouched. That is region-locked editing, the same category as GPT Image 2's masking, not a way to hand the model a skeleton or a depth map before the first generation exists. If you want the deeper mechanics of how Midjourney's style references or its seed parameter actually behave, both are covered in more depth elsewhere on this site, and neither one is a structural control either.

Where Does Real Structure Control Actually Live?

In self-hosted or open-source pipelines, not in a hosted API call. A ControlNet workflow through Automatic1111, ComfyUI, or the diffusers library loads the base Stable Diffusion checkpoint plus a separate ControlNet checkpoint trained for one conditioning type, then runs a preprocessor, a Canny edge detector, a depth estimator, an OpenPose skeleton extractor, against your reference image before generation starts. The prompt still supplies the content. The ControlNet checkpoint and the preprocessed map supply the geometry. That is two additional model loads and a preprocessing step no hosted text-to-image API call currently performs for you.

This is also why the strength dial on that structural input is the single most inconsistent parameter across every tool touched in this piece. A low value preserves the input on Stable Diffusion's own denoising strength, but a high value preserves it on Stability's control_strength, Midjourney's --iw, BFL's image_prompt_strength, and Ideogram's image_weight. Carry a number across tools without checking the direction and you can invert the result; our full per-tool denoising-strength reference lays out which direction each vendor actually uses.

The Practical Prompt-Only Stack

None of this is control in the ControlNet sense. It is mitigation, stacked:

1. Use a reference image on a model documented to accept one. Gemini, Ideogram's Character Reference, and FLUX's eight input_image slots all exist for continuity, a face, an outfit, a palette, carried across generations. None of them promises pixel-aligned geometry.
2. Use a mask or region tool where the API actually offers one. GPT Image 2's masking and Midjourney's Vary Region both lock down everything outside one area. Neither dictates what appears inside it beyond your prompt text.
3. Write a composition contract and repeat it verbatim. One block of wording naming camera angle, subject placement, and framing, pasted unchanged into every prompt in a sequence, removes the self-inflicted drift of describing the same shot slightly differently each time:

COMPOSITION CONTRACT (paste verbatim into every prompt in this set):
Camera at eye level, subject centered in the lower third, wide shot with
visible headroom, three-quarter angle facing camera-left, background out
of focus.

4. Budget a curation pass. Every vendor checked here documents a real limitation on consistency, layout precision, or both. A generation that misses the framing is the expected outcome on today's hosted models, not a sign the prompt was wrong.

What This Means for Planning a Shot

If the project is a sequence of shots rather than one image, the honest workaround stack above matters even more, because drift compounds across frames instead of costing you one regeneration. Our shot-by-shot storyboarding guide walks through exactly that case: a fixed subject description repeated verbatim, one style tag per project, and generating a full sequence in one sitting so a model has something to anchor to. The tools are the same reference-image and masking fields covered here; the difference is planning for a set of images that need to agree with each other, not a single frame that only needs to agree with your prompt.

Prompt Architects does not generate images and makes no promise about what any vendor's model will render structurally. What it does is keep a composition contract, a reference-image note, and the per-vendor field names above as a saved template in the Image Prompt Library, so the honest workaround gets reused instead of retyped. Image prompt generation starts on the Pro plan at $4.99 a month at the time of writing, and the Free plan runs 5 prompt enhancements a day, forever, per our FAQ.