Abstract
In binder jetting followed by sintering, the porosity characterization is critical to understand how the process affects the structure of the printed parts. Image-based porosity detection methods are widely used but the current solutions are limited to specific materials and conditions and require manual tuning that precludes real-time porosity detection. The application of machine learning for automating porosity detection has been also limited to specific materials and conditions and requires a large training dataset for successful implementation. However, large datasets are difficult to acquire experimentally in binder jetting due to prohibited material costs and experiment time. To bridge the knowledge gap, this paper investigates the application of machine learning on automated porosity detection using a small dataset consisting of highly varied cross-section images of metal parts produced by binder jetting followed by sintering. Stylegan3, a type of generative adversarial network, is used to increase the number of training images by image augmentation, and YOLOv5, a convolutional neural network specialized for object detection, is used to detect porosities. The resulting model achieves an F1 score of 88% and detection time of 3–15 ms per image. Generalized porosity detection is also assessed on a set of images containing highly varied materials, resolutions, magnifications, and pore densities. Furthermore, morphological information of the classified porosities such as the distribution of their orientations are automatically extracted using image processing algorithms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The data that support the findings of this study are openly available in MendeleyData at https://data.mendeley.com/datasets/vhxs23krz4/draft?a=37f6a434-bd7e-4895-8995-e32547f42421 [https://doi.org/10.17632/vhxs23krz4.1].
References
Abdelrahman, M., Reutzel, E. W., Nassar, A. R., & Starr, T. L. (2017). Flaw detection in powder bed fusion using optical imaging. Additive Manufacturing, 15, 1–11. https://doi.org/10.1016/j.addma.2017.02.001
Aminzadeh, M., & Kurfess, T. R. (2019). Online quality inspection using Bayesian classification in powder-bed additive manufacturing from high-resolution visual camera images. Journal of Intelligent Manufacturing, 30(6), 2505–2523. https://doi.org/10.1007/s10845-018-1412-0
An, C., Wen, X., Wang, J., & Wu, B. (2016). GAP/DNTF based PBX explosives: A novel formula used in small sized explosive circuits. Central European Journal of Energetic Materials. https://doi.org/10.22211/cejem/64992
Andreeva, R., Stoyanova, E., Tsanev, A., & Stoychev, D. (2016). Corrosion behavior of anodically formed oxide films on aluminum, sealed in cerium-ions containing solutions. Bulgarian Chemical Communications, 48, 96–102.
Antunes, R. A., Ichikawa, R. U., Martinez, L. G., & Costa, I. (2014). Characterization of corrosion products on carbon steel exposed to natural weathering and to accelerated corrosion tests. International Journal of Corrosion. https://doi.org/10.1155/2014/419570
ASTM. (2012). F2792-12a, Standard Terminology for Additive Manufacturing Technologies, (Withdrawn 2015). ASTM International.
Baek, J. Y., Xing, Z. C., Kwak, G., Yoon, K. B., Park, S. Y., Park, L. S., & Kang, I. K. (2012). Fabrication and characterization of collagen-immobilized porous PHBV/HA nanocomposite scaffolds for bone tissue engineering. Journal of Nanomaterials. https://doi.org/10.1155/2012/171804
Bai, Y., & Williams, C. B. (2015). An exploration of binder jetting of copper. Rapid Prototyping Journal. https://doi.org/10.1108/RPJ-12-2014-0180
Bergmann, C., Lindner, M., Zhang, W., Koczur, K., Kirsten, A., Telle, R., & Fischer, H. (2010). 3D printing of bone substitute implants using calcium phosphate and bioactive glasses. Journal of the European Ceramic Society. https://doi.org/10.1016/j.jeurceramsoc.2010.04.037
Bochkovskiy, A., Wang, C.-Y., & Liao, H.-Y. M. (2020). YOLOv4: optimal speed and accuracy of object detection. arXiv Preprint. https://doi.org/10.48550/ARXIV.2004.10934
Chen, F., Qiu, B., Ye, Y., Lv, Y., Shangguan, Y., Song, Y., & Zheng, Q. (2015). Reconstruction of core-shell dispersed particles in impact polypropylene copolymer during extrusion. Chinese Journal of Polymer Science (English Edition). https://doi.org/10.1007/s10118-015-1616-0
Chen, H., & Zhao, Y. F. (2016). Process parameters optimization for improving surface quality and manufacturing accuracy of binder jetting additive manufacturing process. Rapid Prototyping Journal, 22(3), 527–538. https://doi.org/10.1108/RPJ-11-2014-0149
Cheng, J. C. P., & Wang, M. (2018). Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques. Automation in Construction. https://doi.org/10.1016/j.autcon.2018.08.006
Chia, H. (2012). Southern California Society for microscopy and microanalysis. Retrieved June 16, 2022, from http://www.scsmm.org/gallery.html
Community, O. (2010). The OpenCV reference manual. October.
Cui, X., Shi, J., Ye, Z., Zhang, Z., Xu, B., & Chen, S. (2014). Layer-by-layer assembly and photocatalytic activity of titania nanosheets on coal fly ash microspheres. International Journal of Photoenergy. https://doi.org/10.1155/2014/823078
Da’as, E. H., Irvine, J. T. S., Traversa, E., & Boulfrad, S. (2013). Controllable impregnation via Inkjet printing for the fabrication of solid oxide cell air electrodes. ECS Transactions. https://doi.org/10.1149/05701.1851ecst
Do, T., Kwon, P., & Shin, C. S. (2017). Process development toward full-density stainless steel parts with binder jetting printing. International Journal of Machine Tools and Manufacture, 121, 50–60. https://doi.org/10.1016/j.ijmachtools.2017.04.006
Do, T., Shin, C. S., Stetsko, D., Vanconant, G., Vartanian, A., Pei, S., & Kwon, P. (2015). Improving structural integrity with boron-based additives for 3D printed 420 stainless steel. Procedia Manufacturing. https://doi.org/10.1016/j.promfg.2015.09.019
Dojcinovic, M., Eric, O., Rajnovic, D., Sidanin, L., & Balos, S. (2012). The morphology of ductile cast iron surface damaged by cavitation. Metallurgical & Materials Engineering, 18(3), 165–176.
Dollár, P., & Zitnick, C. L. (2015). Fast edge detection using structured forests. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2014.2377715
Doyle, M., Agarwal, K., Sealy, W., & Schull, K. (2015). Effect of layer thickness and orientation on mechanical behavior of binder jet stainless steel 420 + bronze parts. Procedia Manufacturing. https://doi.org/10.1016/j.promfg.2015.09.016
Enneti, R. K., Prough, K. C., Wolfe, T. A., Klein, A., Studley, N., & Trasorras, J. L. (2018). Sintering of WC-12%Co processed by binder jet 3D printing (BJ3DP) technology. International Journal of Refractory Metals and Hard Materials. https://doi.org/10.1016/j.ijrmhm.2017.10.023
Fisher Scientific. (2022). MilliporeSigmaTM ExpressTM PLUS membrane filters. Fisher Scientific Product Catalog. Retrieved November 20, 2022, from https://www.fishersci.com/shop/products/emd-millipore-express-plus-membrane-filters-10/p-3303398
García-Moreno, A. I., Alvarado-Orozco, J. M., Ibarra-Medina, J., & Martínez-Franco, E. (2020). Image-based porosity classification in Al-alloys by laser metal deposition using random forests. International Journal of Advanced Manufacturing Technology. https://doi.org/10.1007/s00170-020-05887-6
Gaytan, S. M., Cadena, M. A., Karim, H., Delfin, D., Lin, Y., Espalin, D., MacDonald, E., & Wicker, R. B. (2015). Fabrication of barium titanate by binder jetting additive manufacturing technology. Ceramics International. https://doi.org/10.1016/j.ceramint.2015.01.108
Gibson, I., Rosen, D., & Stucker, B. (2015). Additive manufacturing technologies: 3D printing and direct digital manufacturing. Springer.
Gobert, C., Reutzel, E. W., Petrich, J., Nassar, A. R., & Phoha, S. (2018). Application of supervised machine learning for defect detection during metallic powder bed fusion additive manufacturing using high resolution imaging. Additive Manufacturing, 21, 517–528. https://doi.org/10.1016/j.addma.2018.04.005
Gonzalez, J. A., Mireles, J., Lin, Y., & Wicker, R. B. (2016). Characterization of ceramic components fabricated using binder jetting additive manufacturing technology. Ceramics International. https://doi.org/10.1016/j.ceramint.2016.03.079
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE computer society conference on computer vision and pattern recognition (Vol. 2016-December). https://doi.org/10.1109/CVPR.2016.90
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017). GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in neural information processing systems (Vol. 2017-December).
Hong, D., Chou, D. T., Velikokhatnyi, O. I., Roy, A., Lee, B., Swink, I., Issaev, I., Kuhn, H., & Kumta, P. N. (2016). Binder-jetting 3D printing and alloy development of new biodegradable Fe–Mn–Ca/Mg alloys. Acta Biomaterialia. https://doi.org/10.1016/j.actbio.2016.08.032
Huang, H., Zhou, H., Yang, X., Zhang, L., Qi, L., & Zang, A. Y. (2019). Faster R-CNN for marine organisms detection and recognition using data augmentation. Neurocomputing. https://doi.org/10.1016/j.neucom.2019.01.084
Hussain, M., Bird, J. J., & Faria, D. R. (2019). A study on CNN transfer learning for image classification. Advances in Intelligent Systems and Computing. https://doi.org/10.1007/978-3-319-97982-3_16
Imani, F., Gaikwad, A., Montazeri, M., Rao, P., Yang, H., & Reutzel, E. (2018). Process mapping and in-process monitoring of porosity in laser powder bed fusion using layerwise optical imaging. Journal of Manufacturing Science and Engineering, Transactions of the ASME. https://doi.org/10.1115/1.4040615
Islam, M. N., & Sacks, S. (2016). An experimental investigation into the dimensional error of powder-binder three-dimensional printing. International Journal of Advanced Manufacturing Technology. https://doi.org/10.1007/s00170-015-7482-7
Jamion, N. A. B., Hafiff, N. H. B. A., Halim, N. H. A., Izzaddin, S. A., Ghazali, S. M., & Sapari, J. M. (2017). Preparation of date seed activation for surfactant recovery. Malaysian Journal of Analytical Science. https://doi.org/10.17576/mjas-2017-2105-06
Jocher, G., Stoken, A., Borovec, J., Christopher, S., Changyu, L., Hogan, A., Laughing, A., Diaconu, L., Poznanski, J., Yu, L., Rai, R., Ferriday, R., Sullivan, T., Claramunt, E. R. (2020). ultralytics/yolov5: v2.0. GitHub.
Jocher, G., Stoken, A., Borovec, J., Chaurasia, A., Changyu, L., Laughing, A., Hogan, A., Hajek, J., Diaconu, L., Marc, Y. (2021). ultralytics/yolov5: v5.0—YOLOv5-P6 1280 models, AWS, Supervise.ly and YouTube integrations. Zenodo.
Karras, T., Aittala, M., Laine, S., Härkönen, E., Hellsten, J., Lehtinen, J., & Aila, T. (2021). Alias-free generative adversarial networks. arXiv Preprint. https://doi.org/10.48550/ARXIV.2106.12423
Lanzetta, M., & Sachs, E. M. (2001). The line formation with alumina powders in drop on demand three dimensional printing. In Proceedings of the 1st international seminar on: Progress in innovative manufacturing engineering (PRIME 2001).
Lee, G., Carrillo, M., McKittrick, J., Martin, D. G., & Olevsky, E. A. (2020). Fabrication of ceramic bone scaffolds by solvent jetting 3D printing and sintering: Towards load-bearing applications. Additive Manufacturing. https://doi.org/10.1016/j.addma.2020.101107
Lei, H. W., Wang, B., Wu, H. H., & Wang, A. H. (2018). Defect detection for polymeric polarizer based on faster R-CNN. Journal of Information Hiding and Multimedia Signal Processing, 9(6), 1414–1420.
Li, S., Gu, X., Xu, X., Xu, D., Zhang, T., Liu, Z., & Dong, Q. (2021). Detection of concealed cracks from ground penetrating radar images based on deep learning algorithm. Construction and Building Materials. https://doi.org/10.1016/j.conbuildmat.2020.121949
Li, Y., Yang, C., Zhao, H., Qu, S., Li, X., & Li, Y. (2014). New developments of ti-based alloys for biomedical applications. Materials. https://doi.org/10.3390/ma7031709
Liu, Y., Cen, C., Che, Y., Ke, R., Ma, Y., & Ma, Y. (2020). Detection of maize tassels from UAV RGB imagery with faster R-CNN. Remote Sensing. https://doi.org/10.3390/rs12020338
Manière, C., & Olevsky, E. A. (2017). Porosity dependence of powder compaction constitutive parameters: Determination based on spark plasma sintering tests. Scripta Materialia. https://doi.org/10.1016/j.scriptamat.2017.07.026
Milewski, J. O. (2017). Additive manufacturing metal, the art of the possible. In Springer series in materials science. Springer. https://doi.org/10.1007/978-3-319-58205-4_2
Miyanaji, H., Zhang, S., Lassell, A., Zandinejad, A., & Yang, L. (2016). Process development of porcelain ceramic material with binder jetting process for dental applications. JOM Journal of the Minerals Metals and Materials Society. https://doi.org/10.1007/s11837-015-1771-3
Moon, J., Caballero, A. C., Hozer, L., Chiang, Y. M., & Cima, M. J. (2001). Fabrication of functionally graded reaction infiltrated SiC-Si composite by three-dimensional printing (3DP™) process. Materials Science and Engineering A. https://doi.org/10.1016/s0921-5093(00)01282-x
Mostafaei, A., Stevens, E. L., Hughes, E. T., Biery, S. D., Hilla, C., & Chmielus, M. (2016). Powder bed binder jet printed alloy 625: Densification, microstructure and mechanical properties. Materials and Design. https://doi.org/10.1016/j.matdes.2016.06.067
Olevsky, E. A. (1998). Theory of sintering: From discrete to continuum. Materials Science and Engineering R: Reports, 23(2), 41–100. https://doi.org/10.1016/S0927-796X(98)00009-6
Ozols, A., Thern, G., Rozenberg, S., Barreiro, M., & Marajofsky, A. (2004). Moldeo de Acero AISI 316 por Colada Gel. CONGRESO CONAMET/SAM.
Pinkney, J. (2022, June 12). Awesome pretrained StyleGAN3. GitHub. Retrieved June 14, 2022, from https://github.com/justinpinkney/awesome-pretrained-stylegan3
Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2016.2577031
Sachs, E., Cima, M., & Cornie, J. (1990). Three-dimensional printing: Rapid tooling and prototypes directly from a CAD model. CIRP Annals—Manufacturing Technology. https://doi.org/10.1016/S0007-8506(07)61035-X
Sachs, E., Cima, M., Williams, P., Brancazio, D., & Cornie, J. (1992). Three dimensional printing: Rapid tooling and prototypes directly from a CAD model. Journal of Manufacturing Science and Engineering, Transactions of the ASME. https://doi.org/10.1115/1.2900701
Sasaki, Y. (2007). The truth of the F-measure. Teach Tutor mater.
Satterlee, N., Torresani, E., Olevsky, E., & Kang, J. S. (2022). Comparison of machine learning methods for automatic classification of porosities in powder-based additive manufactured metal parts. The International Journal of Advanced Manufacturing Technology, 120(9), 6761–6776. https://doi.org/10.1007/s00170-022-09141-z
Scheiba, F. (2021). Carbon based materials for electrochemical energy storage. Institute for Applied Materials Energy Storage Systems. Retrieved June 16, 2022, from https://www.iam.kit.edu/ess/english/667.php
Scime, L., & Beuth, J. (2018). A multi-scale convolutional neural network for autonomous anomaly detection and classification in a laser powder bed fusion additive manufacturing process. Additive Manufacturing, 24, 273–286. https://doi.org/10.1016/j.addma.2018.09.034
Shrestha, S., & Manogharan, G. (2017). Optimization of binder jetting using Taguchi method. JOM Journal of the Minerals Metals and Materials Society, 69(3), 491–497. https://doi.org/10.1007/s11837-016-2231-4
Snelling, D., Li, Q., Meisel, N., Williams, C. B., Batra, R. C., & Druschitz, A. P. (2015). Lightweight metal cellular structures fabricated via 3D printing of sand cast molds. Advanced Engineering Materials. https://doi.org/10.1002/adem.201400524
Srinivas, M., & Babu, B. S. (2017). A critical review on recent research methodologies in additive manufacturing. Materials Today: Proceedings. https://doi.org/10.1016/j.matpr.2017.07.258
Sun, X., Gu, J., Huang, R., Zou, R., & Palomares, B. G. (2019). Surface defects recognition of wheel hub based on improved faster R-CNN. Electronics (Switzerland). https://doi.org/10.3390/electronics8050481
Tang, Y., Zhou, Y., Hoff, T., Garon, M., & Zhao, Y. F. (2016). Elastic modulus of 316 stainless steel lattice structure fabricated via binder jetting process. Materials Science and Technology (United Kingdom). https://doi.org/10.1179/1743284715Y.0000000084
Wang, Y., & Zhao, Y. F. (2017). Investigation of sintering shrinkage in binder jetting additive manufacturing process. Procedia Manufacturing. https://doi.org/10.1016/j.promfg.2017.07.077
Winkel, A., Meszaros, R., Reinsch, S., Müller, R., Travitzky, N., Fey, T., Greil, P., & Wondraczek, L. (2012). Sintering of 3D-printed glass/HAp composites. Journal of the American Ceramic Society. https://doi.org/10.1111/j.1551-2916.2012.05368.x
Wu, W., Yin, Y., Wang, X., & Xu, D. (2019). Face detection with different scales based on faster R-CNN. IEEE Transactions on Cybernetics. https://doi.org/10.1109/TCYB.2018.2859482
Wusatowska-Sarnek, A. M., Ghosh, G., Olson, G. B., Blackburn, M. J., & Aindow, M. (2003). Characterization of the microstructure and phase equilibria calculations for the powder metallurgy superalloy IN100. Journal of Materials Research. https://doi.org/10.1557/JMR.2003.0371
Xia, W., Zhang, Y., Yang, Y., Xue, J.-H., Zhou, B., & Yang, M.-H. (2021). GAN inversion: A survey. arXiv Preprint. https://doi.org/10.48550/ARXIV.2101.05278
Yao, J., Qi, J., Zhang, J., Shao, H., Yang, J., & Li, X. (2021). A real-time detection algorithm for kiwifruit defects based on yolov5. Electronics (Switzerland). https://doi.org/10.3390/electronics10141711
Yao-jian, R., Zhi, S., Jie, T., & Wei, L. (2009). Study on electrolytic aluminium carbon anode preparation with calcined anthracite. In Procedia Earth and planetary science (Vol. 1). https://doi.org/10.1016/j.proeps.2009.09.109
Yoo, J., Cima, M., Sachs, E., & Suresh, S. (2008). Fabrication and microstructural control of advanced ceramic components by three dimensional printing. In Proceedings of the 19th annual conference on composites, advanced ceramics, materials, and structures—B: Ceramic engineering and science proceedings. https://doi.org/10.1002/9780470314784.ch17
Zhang, B., Liu, S., & Shin, Y. C. (2019). In-Process monitoring of porosity during laser additive manufacturing process. Additive Manufacturing. https://doi.org/10.1016/j.addma.2019.05.030
Zhang, S., Miyanaji, H., Yang, L., Zandinejad, A. A., Dilip, J. J. S., & Stucker, B. (2014). An experimental study of ceramic dental porcelain materials using a 3D print (3DP) process. In 25th Annual international solid freeform fabrication symposium; An additive manufacturing conference, SFF 2014.
Zhang, X., Hao, Y., Shangguan, H., Zhang, P., & Wang, A. (2020). Detection of surface defects on solar cells by fusing multi-channel convolution neural networks. Infrared Physics and Technology. https://doi.org/10.1016/j.infrared.2020.103334
Zhao, J., Zhang, X., Yan, J., Qiu, X., Yao, X., Tian, Y., Zhu, Y., & Cao, W. (2021). A wheat spike detection method in UAV images based on improved YOLOv5. Remote Sensing, 13(16), 3095. https://doi.org/10.3390/rs13163095
Zhou, Y., Tang, Y., Hoff, T., Garon, M., & Zhao, F. Y. (2015). The verification of the mechanical properties of binder jetting manufactured parts by instrumented indentation testing. Procedia Manufacturing. https://doi.org/10.1016/j.promfg.2015.09.038
Zhu, X., Lyu, S., Wang, X., & Zhao, Q. (2021). TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-captured Scenarios. In Proceedings of the IEEE international conference on computer vision (Vol. 2021-October). https://doi.org/10.1109/ICCVW54120.2021.00312
Acknowledgements
The support of the National Science Foundation, under Award DMR-2119832 is gratefully acknowledged.
Funding
Funding was provided by National Science Foundation (Grant Number 2119832).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Satterlee, N., Torresani, E., Olevsky, E. et al. Automatic detection and characterization of porosities in cross-section images of metal parts produced by binder jetting using machine learning and image augmentation. J Intell Manuf 35, 1281–1303 (2024). https://doi.org/10.1007/s10845-023-02100-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-023-02100-9