gpt-image-2 sits at the top of the public image-editing leaderboards. On our own five-step architectural edit chain it was the weakest model we have run this year. Both statements are true, and the gap between them is the whole point of this article: an arena scores one edit, and an architect edits the same frame ten times before lunch.
This is not a takedown. The model is very good at the thing it is being ranked on, and its ranking is deserved. What we are reporting is that the ranking did not transfer to our task, that we can say precisely where it stopped transferring, and that the reason is structural rather than accidental.
Where the model stands publicly
As of 25 August 2026, on the single-image editing board at arena.ai — 29.2 million votes across 53 models — gpt-image-2 (medium) is first, with an Elo of 1462. On the text-to-image board it is also first, at 1382. On Artificial Analysis’s editing board, gpt-image-2 (high) is fourth, at 1257.
That is an unusually consistent picture. Two independent boards, a very large vote count, and a model at or near the top of both.
What those boards measure is worth stating exactly, because it is the crux. A voter sees an input image, an instruction, and two candidate outputs. They pick the better one. One edit, judged once, by someone who did not have to live with the result afterwards.
Nobody works that way. Concept design is a chain: change the pool, then the roof material, then the posts, then add a pavilion, then remove a tree — each change applied to the output of the last one. By the fifth pass the model has seen its own output four times. Nothing in a single-edit vote observes what happens across that chain, so nothing in a single-edit Elo score predicts it. That is not a flaw in the arenas. It is a scope boundary that gets forgotten when a number gets quoted.
The method
Everything below comes from one run, on 1 September 2026, through our internal harness. It is described here in enough detail to be repeated against any editing model.
One anchor frame. A single image generated once from a long written brief: a contemporary villa in Bali, a modern architect’s reinterpretation of the Javanese joglo — two-tiered thatched pyramidal roof, four dark ironwood posts, two lower timber wings, a rectangular pool in the foreground, a frangipani tree at the left, flat overcast midday light, three-quarter view from the garden. Generated at 1536×1024, quality high. That single generation took 116 seconds.
The anchor is shared by every chain. That matters more than it sounds: if each chain started from its own generation, any later collapse could be blamed on a worse starting picture. With one anchor, the starting frame is a constant and everything measured afterwards is a property of the editing.
Five edits, always in this order. One of each kind — a form change, a material change over a large area, a material change over a small one, an addition, a removal:
- Replace the rectangular pool with an L-shaped pool wrapping the near corner.
- Replace the alang-alang thatch with dark grey clay roof tiles, same roof shape and pitch.
- Change the four ironwood posts and their beams from dark stained timber to pale honey teak.
- Add a small open bale bengong pavilion with its own thatched roof at the far end of the pool.
- Remove the frangipani tree at the left of the pool, leaving open lawn.
Every instruction carried the same preservation clause on every step: keep everything else identical, and — spelled out at length — do not change the camera in any way. No zoom, no re-framing, no change of focal length, viewpoint, height or angle. Keep the same flat overcast light, the same time of day, the same colour temperature. The clause was constant across all chains, so it can never be the variable.
Six arms. Two axes crossed, plus a control:
- quality
lowandmedium, - variant with a mask and without,
- giving four edit chains of five steps each:
nomask-low,mask-low,nomask-medium,mask-medium. - Plus two text-to-image controls — the anchor brief shot once at each quality level, no chain, no reference — so that if
lowalready loses detail in a single fresh frame, nothing later can be blamed on the editing.
Each step was applied to the previous output of its own chain, never to the anchor. That is the point: we wanted the cumulative effect, not five independent one-shot edits. Every intermediate frame was written to disk.
The masks are coarse rectangles, declared as fractions of the frame and written as RGBA PNGs the size of the source — opaque where the pixels must survive, transparent where the model may repaint. Rectangles rather than hand-cut silhouettes on purpose: a hand-drawn outline per step would make the run unreproducible, and it would quietly do part of the model’s job for it. Here is the mask for step 5, the frangipani removal — the bright box is the only region the model was permitted to touch:
How we measured mask compliance. For every step, we compared the output against that step’s own input — the previous frame in the same chain — pixel by pixel, and took the mean absolute RGB difference in two zones: inside the mask (the transparent region the model was allowed to repaint) and outside it (everything the mask declared protected). The zones come from the alpha channel of the exact PNG that was sent, not from a re-derived rectangle. If a mask is honoured, the outside number is zero. The same zones were then applied to the unmasked chains, where no mask was sent at all, to give a baseline for what “no protection” looks like.
That is the entire measurement. It needs no model-specific tooling, and it is the first thing we would run against any editing engine that claims to support masks.
Finding 1: the mask did not protect anything
Mean absolute pixel change against each step’s own input, on a 0–255 scale. The medium chains, which are directly comparable because the only difference between the two columns is whether a mask was sent:
| Step | Edit | Masked: inside | Masked: outside | No mask: inside | No mask: outside |
|---|---|---|---|---|---|
| 1 | L-shaped pool | 33.8 | 7.5 | 13.8 | 7.5 |
| 2 | Clay roof tiles | 48.5 | 6.1 | 16.8 | 8.1 |
| 3 | Teak posts | 35.8 | 7.4 | 23.1 | 6.6 |
| 4 | Add pavilion | 25.3 | 7.2 | 14.4 | 6.2 |
| 5 | Remove tree | 63.6 | 6.0 | 17.9 | 7.7 |
Read the two outside columns next to each other. 6.0 to 7.5 with a mask; 6.2 to 8.1 without one. The low chains behave the same way, in a slightly wider band: 7.0–11.6 outside the mask, against 5.7–11.6 with no mask sent.
The area the mask declared off-limits moved exactly as much as it moved when we declared nothing. The frame is being repainted in full on every call. There is no protected region.
The inside columns show the one thing the mask reliably did: the change inside the marked box is roughly twice as strong as when no mask is sent. That is a real effect, and it points at the mechanism. The mask is being used as a hint about where to concentrate — an attention pointer, functionally part of the prompt — rather than as a mechanical constraint on which pixels may be written.
One more detail that we think matters. The API contract we were working against says a warning is returned when the model cannot honour the mask. Across all ten masked calls, the warnings field came back empty. No refusal, no partial-compliance flag. From the client’s side, every masked call reported success.
The caveat, stated plainly: we called the model through an intermediary gateway rather than directly against OpenAI’s own endpoint. We therefore cannot say whether the mask was lost in transit at the gateway or ignored by the model. What we can say is that this is what a normal application integration produced, and that the failure was completely silent.
Here is step 5 of each medium chain — same anchor, same five instructions, same order, five passes deep. Left: the masked chain, where the only editable region on the final step was that narrow box at the left of the frame. Right: the chain that was given no mask at all.
These two frames are chain endpoints, so the difference between them is cumulative and not attributable to the mask alone — the per-step table above is where the mask claim actually lives. But they show what five passes cost. In both, the requested edits did land: the pool is L-shaped, the thatch is tiles, the posts are pale, a small pavilion sits at the end of the pool, the frangipani is gone. And in both, the building that arrived is not the building we started with. The two-tiered pyramidal joglo roof — the one architectural idea the whole brief was built around — is gone, replaced by a gable. We traced when: it happened at step 2, the roof-tile swap, whose instruction said in as many words “keeping the same roof shape and pitch”, and whose mask covered exactly that roof band. So this particular loss is not mask leakage — it is inside the marked region, and it is an instruction the model did not follow. It is worth separating from the mask finding rather than folding into it. What the mask evidence covers is everything else that moved on the same passes: the framing pushing in, the light shifting, the background reworking, all of it outside the box on every step, all of it banned in three separate sentences of the preservation clause.
That is the finding in one sentence: we asked for a fence and got a suggestion.
Finding 2: a different kind of degradation
Sequential editing degrades images. We have written about the mechanism before and measured it across three other engines. What is interesting here is that gpt-image-2 degrades in the opposite direction from the worst case we had previously recorded.
At fixed resolution — every frame in this run is 1536×1024, start to finish — file size fell monotonically along each chain:
| Chain | Step 1 | Step 5 | Change |
|---|---|---|---|
| No mask, low | 2545 KB | 2337 KB | −8% |
| No mask, medium | 2797 KB | 2670 KB | −5% |
The masked chains do the same thing at the same rate. Compare that with the weakest model in our earlier run, where the file grew by 28% over the chain: that is accumulation — noise, halos and edge artefacts piling up until the picture turns to mush.
This is the other failure mode. Nothing is accumulating. Detail is being removed. Each pass re-renders a slightly simpler picture than the one it was given: texture in the thatch, grain in the timber, foliage detail in the background ridge. Compression size at constant resolution is a crude proxy for detail, but a monotonic 8% slide over five steps is not noise, and you can see the corresponding smoothing in the frames.
This is the gentler of the two failures. A softened image is still a usable image, where a noisy one is not. But it is a real cost, it is visible by step five, and at low it runs at roughly twice the rate it does at medium.
Finding 3: what the quality parameter buys
The parameter takes low, medium, high or auto, and it applies to editing as well as generation. Functionally it is a budget of output tokens spent on the image. OpenAI publishes no per-level token table for this model, so the only honest way to describe it is to measure it.
Measured latency in this run:
| Setting | Operation | Latency |
|---|---|---|
high | one 1536×1024 generation | 116 s |
medium | one edit | ~43 s |
low | one edit | ~26 s |
For scale, in an earlier run of ours using the same anchor brief and the same five edits, Gemini 3.1 Flash Image turned an edit around in about 15 seconds and Gemini 3.1 Flash Lite in about 8.
A 116-second wait is defensible for a hero frame you will keep. It is not defensible as the response time of a canvas where the working method is to try a change, look at it, and try a different one. And low, the setting fast enough to iterate on, is the setting that washes out detail twice as fast.
That is the squeeze, and it is why we describe the result as a fit problem rather than a quality problem. There was no configuration of this model that was simultaneously fast enough to iterate with and stable enough to iterate on.
What this run does not show
Stating the limits is the difference between a benchmark and an opinion.
It is one anchor image, one typology, one lighting condition, one aspect ratio. A daylit tropical villa with a high-frequency jungle background is a specific test, not a general one.
It is one chain per arm, not repeated runs. These models are stochastic; a re-run would not reproduce the same frames. What we expect to hold is the shape of the behaviour — the outside-mask number tracking the no-mask baseline, the monotonic simplification — because both follow from the mechanism rather than from a lucky sample.
It went through a gateway. As above: the mask finding is a finding about the integration path we used. A direct integration might behave differently, and if it does we would want to know.
It says nothing about one-shot editing, which is what the arenas measure and what this model is excellent at. If your job is one good edit on one good image, the leaderboards are telling you the truth and this article is not about you.
It is dated. Run 1 September 2026, boards read 25 August 2026. Editing models change fast, and a benchmark without a date becomes a false claim within months. When we re-run this, the new run will be dated too, and this one will stay published.
Why we did not ship it
Nuit is built around a canvas where you branch from any image and keep editing. That means the model we ship is not chosen by how good its best single output is. It is chosen by how well it survives being asked the sixth question about the same frame.
Three things decided it. The masked path gave us no protected region, so the one mechanism we rely on for “change this and only this” was not available. The chain simplified detail monotonically, so late edits arrive on a softer picture than early ones. And the only latency that made iteration feel like iteration was the setting that simplified fastest.
None of that makes gpt-image-2 a weak model. It makes it a model tuned for a different unit of work than ours. Our editing today routes by how you asked: mark a region and the edit goes to a mask-based inpainting engine that genuinely honours the mask; type an instruction without marking anything and it stays on the full-frame path, because a change like “make it dusk” really does need the whole scene redrawn. That routing is only worth anything if the masked engine actually holds the unmasked pixels, which is exactly what this run was checking.
The broader lesson is not about one model. A leaderboard measures a task, not a job. Before adopting any editing model, run your own chain: fix one anchor, write an ordered list of edits, take each step from the previous output, save every frame, and measure what changed where. It takes an afternoon and it answers a question no public Elo score is designed to answer.
Our own measured runs, with the frames, the prompts verbatim, the measurement scripts and the raw numbers, live in the benchmarks section — including the results that do not flatter us.
Try Nuit free — 10 generations, no card required. Generate a direction, mark one region, and change one thing without losing the rest. Start your project →