Why good laboratory results do not prove production readiness

Development conditions are controllable: illumination, part position, camera position and data quality are largely known. Production introduces many small deviations at the same time. That is where a system proves whether it is merely demonstrable or genuinely production-ready.

1. The dataset does not fully represent the production line

A good test dataset is necessary, but rarely complete. New batches, surfaces, contamination, reflections or previously unseen defect types can change the data distribution. The central question is therefore not only how well the model performs on available data, but which deviations were deliberately tested.

2. Averages hide critical cases

High accuracy can coexist with an unacceptable false-accept risk. Industrial decisions require metrics to be evaluated by class, scenario and failure consequence. A missed defect has a different economic impact from a good part being rejected.

3. Disturbance factors are not assessed in isolation

When detection performance drops, the search for a cause begins: illumination, optics, position, noise or the pipeline? Targeted sweeps show which parameters are especially sensitive and at which point performance begins to fail.

Production readiness does not mean eliminating every deviation. It means knowing, measuring and controlling the relevant deviations.

4. Decisions are not linked to technical evidence

Requirements, test cases, measurements and engineering decisions often live in different files. Every later change then becomes expensive: after a product change, it is unclear which tests must be repeated and whether the new version is still comparable with the released baseline.

5. Changes do not trigger a structured reassessment

A new camera, a different lens, modified illumination or a pipeline update can affect system performance. A defensible engineering process therefore needs versioning, baseline comparison and defined decision points.

A pragmatic starting point

1. Define the economically critical wrong decisions.
2. Collect real disturbance factors and their expected value ranges.
3. Structure measurement data by scenario and product variant.
4. Set KPI limits before testing begins.
5. Document results, worst cases and actions together.