Computer vision for manufacturing defect detection

Computer vision can detect a wide range of manufacturing defects including surface defects (scratches, dents, cracks, discolouration, pitting), dimensional defects (wrong size, mis-alignment, missing features), assembly defects (missing or wrong components, incorrect orientation), contamination (foreign particles, inclusions), label and printing defects (blurred print, wrong label, barcode errors), and process-specific defects like solder bridging in PCB manufacturing or weld spatter and porosity in metal joining. The appropriate model type varies by defect category.

Anomaly detection is a machine learning approach that trains exclusively on defect-free (normal) samples and flags anything that deviates from the learned normal distribution as a potential defect. It is popular in manufacturing because it solves the fundamental data challenge: defects are rare, so collecting enough labelled defect samples for supervised learning is difficult and time-consuming. Anomaly detection models like PatchCore and EfficientAD can achieve high detection accuracy with only good-part training images, making them ideal for new product launches and low-volume production where defect data is scarce.

The most common edge inference hardware for manufacturing inspection is NVIDIA Jetson AGX Orin or Jetson Orin NX for AI-accelerated inference at the edge, paired with industrial-grade cameras (Basler, FLIR, Cognex). For higher throughput, industrial PCs with dedicated NVIDIA RTX or A-series GPUs are used. Intel's OpenVINO toolkit with Intel Iris Xe or Arc GPUs is an alternative for lower power consumption. Purpose-built industrial vision controllers from Cognex and Keyence integrate hardware and software in a single unit. The choice depends on required inference throughput, environmental conditions, and integration requirements.

Class imbalance (far fewer defect images than good images) is addressed through several strategies: using anomaly detection models that train only on good parts, eliminating the imbalance problem; synthetic defect augmentation using cut-and-paste, GANs, or diffusion-based inpainting to generate additional defect images; weighted loss functions during training that penalise misclassification of rare defect classes more heavily; and oversampling defect images during training. Active learning pipelines that prioritise uncertain or novel images for labelling also help build up rare defect categories more efficiently.

The leading models in 2026 are: PatchCore and EfficientAD for unsupervised anomaly detection on surfaces; YOLOv8 and RT-DETR for object detection tasks like assembly verification and defect localisation; U-Net and SegFormer for semantic segmentation tasks requiring pixel-precise defect mapping; and EfficientNet and Vision Transformers (ViT) for high-speed classification tasks. Most production deployments use a combination — anomaly detection for unknown defects, object detection for assembly checks, and classification for high-throughput pass/fail decisions.

Integration requires connections at multiple levels: physical (camera positioning, lighting, part triggering via encoder or photoelectric sensor); edge compute (inference hardware mounted near the line, connected to cameras via GigE Vision or USB3 Vision); PLC integration (pass/fail signals via digital I/O or OPC-UA/Modbus for reject actuator control and process data exchange); MES integration (inspection results with lot and serial number for traceability, defect records with images); and quality management system integration for formal defect recording and CAPA tracking. OPC-UA is the preferred protocol for PLC/SCADA integration in modern deployments.

Modern deep learning-based inspection systems routinely achieve 90–99%+ detection accuracy on well-defined defect types with sufficient training data. The achievable accuracy depends heavily on defect definition consistency, image quality, lighting design, and training data quantity. Surface anomaly detection on textured surfaces typically achieves 95%+ accuracy with PatchCore-based models. Assembly verification (presence/absence detection) can achieve 99.9%+ with YOLOv8 on high-contrast, well-lit setups. Achieving high accuracy on subtle defects in complex textures remains challenging and often requires multi-camera setups, 3D imaging, or multi-spectral imaging.

2D vision systems use standard area scan or line scan cameras to capture flat images, making them suitable for surface defects, colour variation, label inspection, and 2D dimensional measurements. They are lower cost, simpler to set up, and faster than 3D systems. 3D vision systems (structured light, laser triangulation, time-of-flight, stereo vision) capture depth information in addition to colour, enabling height measurement, volume calculation, and detection of surface deformations that are invisible to 2D cameras (e.g., subtle dents, raised solder joints, warped parts). 3D systems are used where dimensional tolerances are critical or where defects have a height component not visible in 2D.