Checking synthetic labels against the reference photo: measure the floor first
How to check a synthetic defect image by comparing it with the reference photo
A reference comparison carries a floor: 3.83 grey levels at the median, everywhere outside the defect. That was on 70 synthetic images of one part. Any check that compares an image with the reference photograph is partly measuring the generator’s redraw of the whole frame. Comparing against a sibling image doesn’t remove it: in the worst areas, siblings differ from each other more.
What Visynex isVisynex makes synthetic defect images for training machine vision inspection models. Each image is made from one photograph of a good part, and every defect image ships only if a separate detection pass finds the defect ordered.
What was measured
- The images: 70 delivered synthetic defect images of a laser-cut stainless steel bracket (our own test part). All were made from one reference photograph, in the same 800 × 1280 frame.
- The method: mask each image’s defect box (padded 16 px), split the rest into 16 × 16 pixel blocks, and take each block’s mean grey-level difference against two things. One is the reference photo, aligned to each image on its own. The other is a sibling: the next image in the set, a different defect, with its box masked too.
- Controls first: an image against itself gave 0 everywhere. An image against itself brightened by 10 gave a median of 10.00.
| Outside every defect box | Median | 90th percentile |
|---|---|---|
| Synthetic image vs reference photo | 3.83 | 9.59 |
| Synthetic image vs another synthetic image | 3.25 | 14.62 |
On 54 of the 70 images, the reference comparison had the higher median. The rule, fixed before running: the claim holds only if the reference comparison is higher overall AND higher on at least 50 of 70. It holds.

What it means for a check
- A reference comparison carries a floor. Even where nothing was supposed to change, the synthetic image differs from the photograph by about 4 grey levels at the median. A check that calls any difference above a fixed number “changed” is partly measuring that floor.
- The floor is not uniform. At the 90th percentile, two synthetic images differ from each other more than either differs from the reference. Re-rendering varies from image to image, so “compare to a sibling instead” trades one artefact for another.
- What to do: measure the floor on your own images before you set a threshold, mask the defect, and report which comparison you used beside the result.
Limits
- One part, one reference photo, grey levels only, one generator.
- The bracket is a plain metal part with little texture to redraw. On a subject with printed detail, the redraw against the reference can be much larger.
Questions
Can I check a synthetic defect image by comparing it with the reference photo?
Yes, but part of the difference is the generator redrawing the whole frame, not the defect. Measure that floor on your own images, outside the defect, before you set a threshold.
Is comparing against another synthetic image better?
Not in the worst areas. At the 90th percentile, two synthetic images differed from each other by 14.62 grey levels, more than either differed from the reference (9.59).
Try it on your own part
Sign in with your work email, upload one photograph of a good part, and get up to 10 images of it back, labelled in YOLO and COCO. Each takes about a minute. Free, once per email address.
Try it freeTalk to an engineer
Thirty minutes with someone who built the pipeline. No upload, no account.
Book a call