Background, Clothing, and Multimodal Face Ratings

Experimental materials · 23 October 2026

Do a model’s face ratings change when the background or clothing changes but the face stays fixed? In an initial experiment with three photographs, face ratings stayed constant while overall-image ratings changed. In a follow-up using another photograph, one model’s face ratings changed, but differences also appeared after simple downsampling and restoration, and when the same file was evaluated again.

Explore four photographs of one person, 19 input files, and 116 model responses. The two experiments used different prompts and controls and are presented separately. A separate local detector was used for the facial landmark analysis.

Study design

Experiment flow comparing context edits, a processing control, and repeated evaluations while preserving face pixels

Photograph D illustrates the three follow-up comparisons: context edits, a processing control, and repeated responses to identical files. The initial experiment on A–C requested face and overall-image ratings in the same prompt.

Editable diagrams · five draw.io pages · Study overview SVG

Initial experiment · Photographs A–C

Photograph Model

Original photograph A, blue top — A · 1122 × 1402

Original photograph B, floral top — B · 1280 × 1075

Original photograph C, dark top — C · 969 × 701

Three photographs each had an original, a background edit, and a clothing edit. Two models evaluated each input twice, producing 36 responses. Face scores were unchanged in all 12 edited-versus-original comparisons; overall-image scores changed in eight. Each request asked for both face and overall-image ratings.

View all nine evaluated inputs

All nine original, background-edited, and clothing-edited inputs for photographs A, B, and C

Loading initial experiment records…

All repeated responses for the selected initial photograph and model
Condition	Face scores 1 / 2	Image scores 1 / 2	Recorded responses

Follow-up · Photograph D

Condition

Original portrait photographed in an outdoor woodland setting — Original

The same portrait with a café background — Café

This page displays reduced-size previews. Evaluation used the full-resolution PNGs in the downloads. The designated 220,716 face pixels and 407,183 head-and-neck pixels match the original in every input. Glasses and earrings were excluded because they could obscure the face area.

View compositing and pixel differences

Layered compositing of the original face and head with edited surroundings, and actual RGB differences for three conditions

The left panel separates the layers used to restore the protected original pixels. Difference maps on the right share a 0–255 scale; white indicates no difference. Protected face pixels have zero difference in every input. Compositing SVG

All 80 follow-up responses

Model Prompt

Loading records…

All repeated responses for the selected model and prompt
Condition	Scores 1 / 2	Mean change	Recorded responses

Scores are the models’ subjective ratings on a 1–7 scale. Mean changes are relative to the original image’s mean score under the same model and prompt. Expand a row to read both responses and their reasons in the original language.

Estimated landmarks and response consistency

Follow-up edit effects, processing-control effects, and disagreement on identical files by model and prompt

MediaPipe estimated facial landmarks in all 19 inputs. Even with identical face pixels, the locations estimated from the full photographs varied slightly. The sportswear edit of photograph D produced an RMS displacement of 2.30 pixels across 457 protected landmarks. Control inputs with the area outside the face filled with the same gray yielded identical estimates.

Observed landmark displacement, fixed-face input controls, and RMS displacement across nine conditions

Arrows are enlarged tenfold for visibility; values are reported in original pixels. Displacement measures variation in the detector’s coordinate estimates. This analysis does not establish GPT’s internal processing or the cause of rating changes. Full coordinates, settings, and hashes · Pixel measurements by region

View disagreements between repeated responses

Disagreement matrix for face scores, mouth, teeth, expression, and outline by model and prompt

Purple cells indicate disagreement between two responses; gray cells indicate agreement. Agreement does not establish whether an observation is correct.

Downloads

Extract the six ZIP files into one folder to assemble the full materials. These include input, prompt, and code hashes recorded before evaluation, generated candidates, and model responses. The original code retains the local paths used at the time. Reanalyzing saved results does not require model calls.