VISYNEXTalk to an engineer
Measurement

Checking synthetic labels against the reference photo: measure the floor first

How to check a synthetic defect image by comparing it with the reference photo

A reference comparison carries a floor: 3.83 grey levels at the median, everywhere outside the defect. That was on 70 synthetic images of one part. Any check that compares an image with the reference photograph is partly measuring the generator’s redraw of the whole frame. Comparing against a sibling image doesn’t remove it: in the worst areas, siblings differ from each other more.

What Visynex isVisynex makes synthetic defect images for training machine vision inspection models. Each image is made from one photograph of a good part, and every defect image ships only if a separate detection pass finds the defect ordered.

What was measured

  • The images: 70 delivered synthetic defect images of a laser-cut stainless steel bracket (our own test part). All were made from one reference photograph, in the same 800 × 1280 frame.
  • The method: mask each image’s defect box (padded 16 px), split the rest into 16 × 16 pixel blocks, and take each block’s mean grey-level difference against two things. One is the reference photo, aligned to each image on its own. The other is a sibling: the next image in the set, a different defect, with its box masked too.
  • Controls first: an image against itself gave 0 everywhere. An image against itself brightened by 10 gave a median of 10.00.
Outside every defect boxMedian90th percentile
Synthetic image vs reference photo3.839.59
Synthetic image vs another synthetic image3.2514.62

On 54 of the 70 images, the reference comparison had the higher median. The rule, fixed before running: the claim holds only if the reference comparison is higher overall AND higher on at least 50 of 70. It holds.

Surface Scratch/Abrasion on a laser-cut steel bracket, synthetic defect image with its detection box
Surface Scratch/Abrasion, synthetic, from one photograph

What it means for a check

  • A reference comparison carries a floor. Even where nothing was supposed to change, the synthetic image differs from the photograph by about 4 grey levels at the median. A check that calls any difference above a fixed number “changed” is partly measuring that floor.
  • The floor is not uniform. At the 90th percentile, two synthetic images differ from each other more than either differs from the reference. Re-rendering varies from image to image, so “compare to a sibling instead” trades one artefact for another.
  • What to do: measure the floor on your own images before you set a threshold, mask the defect, and report which comparison you used beside the result.
difference from the referenceacross the image →the defecta fixed thresholdover it, where nothingwas meant to changethe floor: never zero
A schematic of the idea, not a result. There are no numbers on it: the floor depends on your images.

Limits

  • One part, one reference photo, grey levels only, one generator.
  • The bracket is a plain metal part with little texture to redraw. On a subject with printed detail, the redraw against the reference can be much larger.

Questions

Can I check a synthetic defect image by comparing it with the reference photo?

Yes, but part of the difference is the generator redrawing the whole frame, not the defect. Measure that floor on your own images, outside the defect, before you set a threshold.

Is comparing against another synthetic image better?

Not in the worst areas. At the 90th percentile, two synthetic images differed from each other by 14.62 grey levels, more than either differed from the reference (9.59).

For the engineer

Try it on your own part

Sign in with your work email, upload one photograph of a good part, and get up to 10 images of it back, labelled in YOLO and COCO. Each takes about a minute. Free, once per email address.

Try it free
For whoever signs off

Talk to an engineer

Thirty minutes with someone who built the pipeline. No upload, no account.

Book a call