Steel Defect Monitoring Through Convolutional Neural Network Image Segmentation and Neural Network Classification Integrated with U-Control Chart
Abstract
Automated visual inspection can detect steel surface defects, but image-level detection alone does not indicate whether the production process is statistically stable. This study proposes an image-based monitoring procedure that integrates deep-learning-based defect detection with a U-control chart. Using the Severstal steel defect dataset, run-length-encoded (RLE) annotations were reconstructed into binary masks, and the steel surface images were resized and normalized. U-Net segmentation was used to localize defective regions, while a MobileNetV2-based model supported multi-label defect classification. The predicted masks were thresholded, and external contour detection was applied to count distinct defect regions in each image. The image-level counts were aggregated into subgroups of 50 images and monitored using initial Phase I, revised Phase I, and Phase II U-chart analyses. The initial Phase I chart signaled one out-of-control subgroup; after this subgroup was screened, the revised Phase I subgroups fell within the control limits, and the Phase II subgroups generally remained within the revised limits. The results show that the proposed workflow can transform image segmentation outputs into defect-per-unit statistics and interpretable monitoring signals, linking automated visual inspection with statistical process control.
Downloads
References
[2] A. Sioma, “Vision System in Product Quality Control Systems,” Applied Sciences, vol. 13, no. 2, p. 751, Jan. 2023, doi: 10.3390/app13020751.
[3] M. Babic, M. A. Farahani, and T. Wuest, “Image Based Quality Inspection in Smart Manufacturing Systems: A Literature Review,” Procedia CIRP, vol. 103, pp. 262–267, 2021, doi: 10.1016/j.procir.2021.10.042.
[4] A. A. P. Chazhoor, E. S. L. Ho, B. Gao, and W. L. Woo, “A Review and Benchmark on State-of-the-Art Steel Defects Detection,” SN Comput. Sci., vol. 5, no. 1, p. 114, Dec. 2023, doi: 10.1007/s42979-023-02436-2.
[5] T. S. Adeyemi, “Defect Detection in Manufacturing: An Integrated Deep Learning Approach,” Journal of Computer and Communications, vol. 12, no. 10, pp. 153–176, 2024, doi: 10.4236/jcc.2024.1210011.
[6] A. Khan, A. Sohail, U. Zahoora, and A. S. Qureshi, “A survey of the recent architectures of deep convolutional neural networks,” Artif. Intell. Rev., vol. 53, no. 8, pp. 5455–5516, Dec. 2020, doi: 10.1007/s10462-020-09825-6.
[7] J. Wang, T. Zhang, Y. Cheng, and N. Al-Nabhan, “Deep Learning for Object Detection: A Survey,” Computer Systems Science and Engineering, vol. 38, no. 2, pp. 165–182, 2021, doi: 10.32604/csse.2021.017016.
[8] W. Zhao, F. Chen, H. Huang, D. Li, and W. Cheng, “A New Steel Defect Detection Algorithm Based on Deep Learning,” Comput. Intell. Neurosci., vol. 2021, no. 1, Jan. 2021, doi: 10.1155/2021/5592878.
[9] F. Chen, L. Fu, Y. Zhang, J. Li, Q. Zhang, and S. Bi, “A Review of Deep Learning-Based Steel Surface Defect Detection,” Academic Journal of Science and Technology, vol. 15, no. 1, pp. 198–202, Apr. 2025, doi: 10.54097/g36nm962.
[10] N. Siddique, S. Paheding, C. P. Elkin, and V. Devabhaktuni, “U-Net and Its Variants for Medical Image Segmentation: A Review of Theory and Applications,” IEEE Access, vol. 9, pp. 82031–82057, 2021, doi: 10.1109/ACCESS.2021.3086020.
[11] C. Williams, F. Falck, G. Deligiannidis, C. C. Holmes, A. Doucet, and S. Syed, “A Unified Framework for U-Net Design and Analysis,” in Advances in Neural Information Processing Systems 36, San Diego, California, USA: Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2023, pp. 27745–27782. doi: 10.52202/075280-1206.
[12] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” 2015, pp. 234–241. doi: 10.1007/978-3-319-24574-4_28.
[13] S. Albawi, T. A. Mohammed, and S. Al-Zawi, “Understanding of a convolutional neural network,” in 2017 International Conference on Engineering and Technology (ICET), IEEE, Aug. 2017, pp. 1–6. doi: 10.1109/ICEngTechnol.2017.8308186.
[14] S. Chen and L. Wells, “A multi‐image monitoring framework for statistical process control to improve manufacturing systems,” Qual. Reliab. Eng. Int., vol. 38, no. 2, pp. 679–702, Mar. 2022, doi: 10.1002/qre.3008.
[15] F. M. Megahed, W. H. Woodall, and J. A. Camelio, “A Review and Perspective on Control Charting with Image Data,” Journal of Quality Technology, vol. 43, no. 2, pp. 83–98, Apr. 2011, doi: 10.1080/00224065.2011.11917848.
[16] J. M. Prats-Montalbán and A. Ferrer, “Statistical process control based on Multivariate Image Analysis: A new proposal for monitoring and defect detection,” Comput. Chem. Eng., vol. 71, pp. 501–511, Dec. 2014, doi: 10.1016/j.compchemeng.2014.09.014.
[17] Y. Okhrin, W. Schmid, and I. Semeniuk, “A control chart for monitoring image processes based on convolutional neural networks,” Stat. Neerl., vol. 79, no. 1, Feb. 2025, doi: 10.1111/stan.12366.
[18] M. Ahsan, M. Mashuri, H. Kuswanto, and D. D. Prastyo, “Intrusion Detection System Using Multivariate Control Chart Hotelling’s T2 Based on PCA,” Int. J. Adv. Sci. Eng. Inf. Technol., vol. 8, no. 5, pp. 1905–1911, Oct. 2018, doi: 10.18517/ijaseit.8.5.3421.
[19] S. Bersimis, A. Sachlas, and P. Economou, “Monitoring defects on products’ surface by incorporating scan statistics into process monitoring procedures,” Qual. Reliab. Eng. Int., vol. 41, no. 1, pp. 293–317, Feb. 2025, doi: 10.1002/qre.3652.
[20] S. Dahari, M. A. Talib, and A. A. Ghapor, “Robust Control Chart Application in Semiconductor Manufacturing Process,” Journal of Advanced Research in Applied Sciences and Engineering Technology, vol. 43, no. 2, pp. 203–219, 2024, doi: 10.37934/araset.43.2.203219.
[21] Amitava. Mitra, Fundamentals of quality control and improvement. John Wiley & Sons, Inc., 2016.
[22] A. Pipatprapa, “Intelligent Attribute Control Chart System Establishing Fuzzy Image Processing Technique,” Kasem Bundit Engineering Journal, vol. 8, no. Special Issue, pp. 1221–132, May 2018, Accessed: Jun. 18, 2026. [Online]. Available: https://ph01.tci-thaijo.org/index.php/kbej/article/view/120382/102523
[23] D. C. Montgomery, Introduction to Statistical Quality Control, 8th ed. John Wiley & Sons, 2020.
[24] A. Grishin, BorisV, iBardintsev, inversion, and Oleg, “Severstal: Steel Defect Detection,” Kaggle. Accessed: Jul. 02, 2026. [Online]. Available: https://kaggle.com/competitions/severstal-steel-defect-detection
[25] A. Bano and M. Salamat, “Image Processing Methods: A Systematic Literature Review of Classical and Modern Approaches,” The Asian Bulletin of Big Data Management, vol. 5, no. 4, pp. 91–123, Nov. 2025, doi: 10.62019/s6z7we84.
[26] R. C. . Gonzalez and R. E. . Woods, Digital image processing. Pearson, 2018.
[27] S. Minaee, Y. Y. Boykov, F. Porikli, A. J. Plaza, N. Kehtarnavaz, and D. Terzopoulos, “Image Segmentation Using Deep Learning: A Survey,” IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–1, 2021, doi: 10.1109/TPAMI.2021.3059968.
[28] R. Yulianto et al., “Innovative UNET-Based Steel Defect Detection Using 5 Pretrained Models,” Evergreen, vol. 10, no. 4, pp. 2365–2378, Dec. 2023, doi: 10.5109/7160923.
[29] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Jun. 2018, pp. 4510–4520. doi: 10.1109/CVPR.2018.00474.
[30] V. Nair and G. E. Hinton, “Rectified Linear Units Improve Restricted Boltzmann Machines,” in Proceedings of the 27th International Conference on Machine Learning, Haifa, 2010, pp. 807–804.
[31] D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” Jan. 2017, [Online]. Available: http://arxiv.org/abs/1412.6980
[32] C. Shorten and T. M. Khoshgoftaar, “A survey on Image Data Augmentation for Deep Learning,” J. Big Data, vol. 6, no. 1, p. 60, Dec. 2019, doi: 10.1186/s40537-019-0197-0.
[33] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” J. Mach. Learn. Res., vol. 15, no. 1, pp. 1929–1958, Jan. 2014.
[34] T. Fawcett, “An introduction to ROC analysis,” Pattern Recognit. Lett., vol. 27, no. 8, pp. 861–874, Jun. 2006, doi: 10.1016/j.patrec.2005.10.010.
Copyright (c) 2026 Inas Wulan Ramadhani, Wibawati Wibawati, Muhammad Ahsan

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.














.jpg)





