Part I asked whether the model understands physics. Part II asks which data you trust about what it learned. “Trained on synthetic data” says how a model learned, not whether it may run, and a training set and an acceptance set do two different jobs. Blur them and you send a model to the line that passed every test except the ones the line would have given it.
Synthetic data is for coverage: rare faults, class balance, development before production data exists, edge cases you won’t stage live. Real plant data is for truth: evaluation, acceptance, drift detection. The evidence supports synthetic as a tool, not a verdict, ~24% recall uplift from added synthetic clips, ~0.95 mAP at Corning from eight real images plus synthetic, but an independent spill study shows the catch: models scored 0.81–0.84 on a public set and 0.64–0.71 on 100 of a plant’s own proprietary images. Accuracy often peaks near a 25% synthetic share and falls below real-only once synthetic dominates.
The vendor owns the generator; the plant owns the acceptance set, labeled real data, held out, refreshed when lighting, suppliers or fixturing change. Three records make it auditable: what was real, what share was synthetic, which real evaluation authorized release. Your action this week: ask who owns your real hold-out set and when it was last refreshed. Full framework at renegrywnow.com.
Reflection questions
Who owns your real plant hold-out set, and when was it last refreshed?
Could your last go-live reproduce its numbers on your own proprietary images, not just a public or synthetic set?
Is the synthetic share of your final training mix capped and documented?
Keywords: Synthetic Data, Real Plant Data, Hold-Out Set, Acceptance Testing, Data Provenance, Sim-to-Real, Defect Detection, Evaluation Protocol, Go-Live Gate, Industrial AI
Series: Energy Dominance · Week 44 · Part II
Previous: Part I — World Models in Manufacturing: Physics, Not Language.
