Ask for the same thing three times
Eight architectural briefs, three models, three runs each. Not one of the 72 images is a repeat of another — floor plans vary a third as much as photorealistic scenes, and a tightly written brief steadies the framing where an open one does not.
| Model | Provider | Endpoint |
|---|---|---|
| Lightgoogle/gemini-3.1-flash-lite-image | Google (via Vercel AI Gateway) | ai-gateway.vercel.sh/v1/chat/completions |
| Nuit Ngoogle/gemini-3.1-flash-image | Google (via Vercel AI Gateway) | ai-gateway.vercel.sh/v1/chat/completions |
| Nuit Fflux-2-pro, image_size preset | Black Forest Labs (via fal) | fal.run/fal-ai/flux-2-pro |
We generated the same eight briefs three times each, through three models, and measured how much the repeats differ from one another. The question is narrow and practical: if you generate something and it is nearly right, can you ask again and expect a near-miss variation — or something else entirely?
The answer is the same for all three models, and it changes how you should work.
Nothing here is a repeat. It is a different picture every time.
Averaged over all 24 brief-and-model combinations, the composition overlap between two runs of the same brief is 0.072 out of 1.0. Not a variation on the first image — a different one, from the same words.
See it
The steadiest triple in the whole run and the wildest, both at three runs of one brief through one model. Even the steadiest is three different buildings.
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 What varies, and by how much
Computed by scripts/bench-repeatability.py. Variance is the mean pixel
difference between two runs — higher means more different. Composition is the
overlap of their edge maps: 1.0 would be the same framing, 0 a different picture.
By mode — the strongest signal in the run
Floor plans converge. Photorealistic scenes do not.
| Mode | Variance | Composition |
|---|---|---|
| floor-plan | 34.77 | 0.077 |
| exterior | 52.73 | 0.073 |
| masterplan | 52.84 | 0.061 |
| interiors | 55.62 | 0.075 |
By model
A narrower spread than the modes above — and the cross-model figure is indicative rather than exact, because the models delivered different pixel grids in this run.
| Model | Variance | Composition |
|---|---|---|
| Nuit F | 45.8 | 0.073 |
| Nuit N | 50.36 | 0.070 |
| Light | 50.8 | 0.072 |
Open brief against a brief with constraints
Constraints barely move the pixel spread but noticeably steady the framing.
| Brief | Variance | Composition |
|---|---|---|
| Open brief | 51.33 | 0.066 |
| Brief with constraints | 46.65 | 0.078 |
Every triple
All 72 images. Pick a model; each row is one brief run three times.
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 Run 1 Run 2 Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 Run 1 Run 2 Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 
Run 1 
Run 2 
Run 3 The eight briefs, verbatim
Published in full so the run can be repeated.
| Mode | Brief | Format |
|---|---|---|
| exterior | A two-storey family house on a gentle slope, timber and white render, large windows facing the valley, late afternoon light.Exterior, open brief | 16:9 |
| exterior | A single-storey courtyard house in a dry mediterranean landscape. Flat roof with a deep overhang on the south side. Walls in warm lime render, window frames in dark bronze. One olive tree in the courtyard. Gravel ground, no lawn. Morning light from the left. No cars, no people. Photographed at eye level from the entrance side.Exterior, eight constraints | 16:9 |
| interiors | A calm living room with a plaster fireplace, oak floor, linen sofa, and tall windows with sheer curtains.Interior, open brief | 3:2 |
| interiors | A kitchen in a converted stone barn. Exactly three materials: pale oak, honed black stone, and lime plaster. A single long island with no upper cabinets. One pendant light over the island. Original timber roof trusses visible. Overcast daylight from a window on the right. No decoration on the surfaces.Interior, material discipline | 3:2 |
| floor-plan | A clean architectural floor plan of a three-bedroom single-storey house, black lines on white, room labels, furniture indicated.Floor plan, open brief | 4:3 |
| floor-plan | An architectural floor plan, black line drawing on white. Programme: entrance hall, open kitchen and dining, separate living room, three bedrooms, one bathroom plus one WC, utility room, and a covered terrace on the south side. Single storey, roughly rectangular footprint. Label every room. Show wall thickness and door swings. No colour, no shading.Floor plan, programme given | 4:3 |
| masterplan | An aerial masterplan of a small residential cluster of eight houses around a shared green, with parking at the edge.Masterplan, open brief | 1:1 |
| masterplan | An aerial site plan of a hillside development. Twelve dwellings in four terraced rows following the contours. One access road entering from the north-west, no through traffic. Communal parking in two pockets at the road edge. A pedestrian path linking all rows to a viewpoint at the south end. Existing trees kept along the eastern boundary. Top-down orthographic view.Masterplan, site rules | 1:1 |
What we concluded
- No model repeats itself. Composition overlap between two runs of the same brief averages 0.07 out of 1.0 — you do not get the same picture back with small differences, you get a different picture. If a frame is worth keeping, keep it; you cannot re-roll your way back to it.
- The mode matters more than the model. Floor plans vary a third as much as photorealistic work (34.77 against 52.73 for exteriors, 55.62 for interiors and 52.84 for masterplans). Line drawings converge on a similar answer; a lit, furnished, weathered scene has far more room to differ.
- Writing a tighter brief is the one lever that works. Briefs carrying six to eight explicit requirements came back 46.65 against 51.33 for open one-liners, with composition agreement 0.078 against 0.066. More requirements do not just improve the answer — they narrow the range of answers.
- The gap between models is smaller than the gap between a floor plan and an interior (45.8 to 50.8 against 34.8 to 55.6). Pick the model for its character; none of them offers repeatability.
Reproducing this
Downloadable here, not a list of paths in a repository you cannot open. Bring your own gateway and fal keys; the harness reads them from the environment.
- run.json — every measurement on this page
- model-selection-test.mjs — the harness that produced the frames
- bench-repeatability.py — computes the variance and composition figures
- raw-run.json — the untouched run output, timings and costs included
The measurements, scripts and briefs are published under CC BY 4.0. Model versions change, so this page carries the date the run was executed.
Questions
Can I regenerate an AI image and get the same result again?
Not with these models on the same prompt. Across 24 brief-and-model combinations we measured composition overlap between repeat runs at 0.07 out of 1.0 — effectively a different photograph each time, not a variation of the first. The practical rule is that a generated image you like is a one-off: save it and branch from it, rather than trying to reproduce it from the prompt.
Do more detailed prompts make AI image output more consistent?
Yes, measurably — it is the one thing that reliably narrows the spread. Briefs carrying six to eight explicit requirements came back at 46.65 variance against 51.33 for open one-line briefs, with composition agreement 0.078 against 0.066. Extra requirements steady both the framing and the content — though they still do not make two runs the same picture.
Which AI images are most reproducible — plans or renderings?
Plans, by a wide margin. Floor-plan briefs varied at 34.77 against 52.73 for exteriors, 55.62 for interiors and 52.84 for masterplans. A black-line plan has a narrow space of plausible answers, so repeat runs converge; a photorealistic scene has to re-decide lighting, materials, vegetation and weather each time, and those choices compound.
How was repeatability measured?
Eight architectural briefs across four modes were each generated three times by each of three models — 72 images. For every brief-and-model triple we compared all three pairs two ways: the mean absolute pixel difference, and the intersection-over-union of their edge maps, which measures whether the composition is the same regardless of colour or lighting. Frames were resampled to a common long edge first so resolution is not a factor. The script and the raw run are published.
Related
- What eight sequential edits do to one image — the companion run: not repeating, but changing one image over and over
- Maintaining spatial consistency across generations