Conference ICSIF23
Conference ICSIF23
Conference ICSIF23
1 Introduction
Over last two centuries, every industrial revolution has been propelled by a change in production methods
through an emerging technology of that era. In this context, the steam engine was the flame that started
the first revolution in industry, the second started by the application of electricity, and the third by the
widespread use of information technology and electronics. In recent times, the popularity of the Internet of
Things (IoT) [1] and Cyber- Physical Systems (CPS) has captured the attention of entreprise and academia.
The use of these technologies promises to enable better connectivity between information, products, and peo-
ple, thereby improving current production models. This phenomenon is known as Industry 4.0 [2]. It leads to
several innovations, from digital technologies, advanced materials, innovative products to new manufacturing
processes. Industry 4.0 is a revolutionary concept that is having a major impact on several areas of industrial
manufacturing . It helps to enhance and improve development and growth through the use and adaptation
of new technology and engineering tools, for creating intelligent products through intelligent processes. This
evolution in industry is realised including various technologies such as : ML, Internet of Things, Big Data, and
Cloud Computing (CC) which are driving together a revolutionary transformation in the fields of industry .
Smart manufacturing, a key aspect of Industry 4.0, epitomize the forthcoming paradigm of manufacturing,
wherein the uninterrupted transmission and meticulous analysis of shop floor data play a pivotal role in gen-
erating profound manufacturing insights. In this context of industry of the future a wide range of applications
is envisioned, providing a solution that can help manufacturing companies to further optimise their processes
and significantly increase their internal efficiency, thanks to the system’s automated procedures, easy setup
including simple, demand-driven installation and, finally, its high degree of scalability. Depending on theses
application, a various of ML algorithms, have already been explored to achieve characteristics such as fault
detection, prediction and autonomy.
AI and ML, gave a tremendous boost to Industry 4.0. Thanks to intelligent software, the large volumes of
information and data being collected from the industrial environment through IoT devices. AI solutions can
learn to read ”between the lines”, enabling the discovery of many complex interrelationships that human
beings would not or no longer be able to grasp, this enhanced capability is made possible through the use of
advanced ML analysis techniques. The increasing amount of real-time data generated in industrial environ-
ment has led factories to the adopt ML’s techniques, algorithmes to respond specific industrial needs.
According to Arthur L. Samuel [35], ML refers to the ability of a system to behave in a smart way like
human, in terms of performance and solving complex and hard problems. There are different types of ML
models, which are categorised[36] as follows: supervised learning (include algorithms of classification and
regression such as Linear Discriminant Analysis (LDA) and Linear Regression(LR)), unsupervised learn-
ing (include algorithms of clustering such as K-Means), reinforcement learning (RL) (is based on agent,
include algorithm such as Deep Q-learning, AlphaGo [37] is an example).
Recently ML applications have known a large expansion Not surprisingly, many works point to ML being the
key to the development of the present industry towards industry 4.0 evolution .[38][39] The rest of this paper
focus in section 2 on the methodology used to identify pertinent research and mention some related work to
the use of machine learning in industry 4.0 . The section 3 contains a definition of Reinforcement learning
approach and their application in industry 4.0 including some related work.
2 RELATED WORK
In this part, we discuss our used methodology and then we introduce some state of art in machine learning
applications for industry 4.0 between 2018 and 2023 according to several criteria. First, we start with ML
algorithms used in manufac- turing, then we mention their application in industry.
(a) ML publications in industry 4.0 in the past five years (b) Machine Learning and industry 4.0 score Interest
3 Reinforcement learning
Reinforcement Learning (RL) algorithms concerned two essential elements, an agent operating in a space. The
agent model interacts directly with the environment and manipulates it to perform a desired learning process.
According to [5], the , the agent start to receives the information about state called S of the environment.
Based on this state and its probabilistic decision process, the agent selects an action A. Subsequent to
executing the action in the previous state, the environment move to a novel state, and the agent is rewarded
R. This cycle of reinforcement learning persists, with the ultimate objective of maximizing, aiming to maximise
the rewards by improving the choice of actions towards an optimal control policy. In essence, the agent strives
to acquire the most effective control strategy for the environment.
RL holds great promise for achieving autonomous control in various industrial applications, such as
processes, machines and systems. In addition, multi-agent RL is gaining attention in the research community
and is showing potential for product control. Despite its potential, RL remains under-utilised in industry
compared to other machine learning techniques.
This part provide a review of related work in the application of reinforcement learning in the areas of industry
4.0. RL has been applied effeciently in different areas in industry
It is widely acknowledged that AI and ML are assuming increasingly pivotal roles in the digital transforma-
tion of manufacturing towards Industry 4.0, contributing to enhanced quality, productivity, and operational
efficiency. In contemporary times, ML algorithms find extensive utility across diverse manufacturing domains,
including optimization, control and security. The heightened transparency achieved within the manufacturing
ecosystem through these technologies proves advantageous for cost reduction without compromising produc-
tion quality. RL methods also hold substantial promise in Industry 4.0 applications. One key rationale is that
new tools necessitate a level of optimization that surpasses what can be achieved through human interven-
tion. The present paper has furnished a comprehensive survey of the state-of-the-art ML and RL methods
developed since 2018. Recent trends reveal the ascendancy of RL-based approaches in the research landscape,
largely propelled by advancements in smart manufacturing. Various machine learning paradigms, including
supervised , unsupervised, deep learning, and reinforcement learning, are briefly introduced alongside their
applications within the manufacturing sector. To address these explorations, we undertook a literature re-
view, appraising the most recent scientific publications. As we delve into the potential of AI/ML and RL,
we remain optimistic about their capacity to drive sustainable growth, operational efficiency, and innovation
within the industries of the future.
References
1. M. McKnight, ”IoT, Industry 4.0, Industrial IoT: Why Connected Devices are the Future of Design,” 2017.
2. F. A. Germán, D. L. Santos, and A. N. Fabián, ”Industry 4.0 technologies: implementation patterns in manufac-
turing companies,” International Journal of Production Economics,Vol.210, pp 15-26, 2019.
3. Hernavs, J., Ficko, M., Berus, L., Rudolf, R., Klančnik, S., ”DEEP LEARNING IN INDUSTRY 4.0-Bref
overview”, vol.21, 2018.
4. J.Dalzochioa, R.Kunsta, E. Pignatonb, A.Binottoc, S.Sanyalc, J.Favillad, J.Barbosaa, ”Machine learning and
reasoning for predictive maintenance in Industry 4.0: Current status and challenges”, Vol.123, 2020
5. ChenT., Sampath V., May M.C., Shan S., Jorg O.J., AguilarMartı́n J.J., Stamer F., Fantoni G.,Tosello G., Calaon
M. ”Machine Learning in Manufacturing towards Industry 4.0: From ‘For Now’ to‘Four-Know’”. , vol 13, 2023
6. B. Bajic, I. Cosic, M. Lazarevic, N. Sremcev, A. Rikalovic, ”Machine Learning Techniques for Smart Manufac-
turing: Applications and Challenges in Industry 4.0”, pp 1-31, 2018
7. Iqbal H. Sarker, ”Machine Learning: Algorithms, Real World Applications and Research Directions”, vol.2, 2021
8. Jinjiang Wanga, Yulin Maa, Laibin Zhanga, Robert X. Gao b, Dazhong Wuc, ”Deep learning for smart manufac-
turing: Methods and applications”, Vol.48, pp 144-156, 2018
9. Francesco Zangaro, Stefan Minner, Daria Battini, ”A supervised machine learning approach for the optimisation
of the assembly line feeding mode selection”, International Journal of Production Research, vol.59,2020
10. Yanzhou Fu, Austin R.J. Downey, Lang Yuan, Tianyu Zhang, Avery Pratt, Yunusa Balogun, ”Machine learning
algorithms for defect detection in metal laser-based additive manufacturing”, Vol.75, pp 693-710, 2022
11. Huimin Zhao, Xiaoxu Yang, Baojie Chen, Huayue Chen, Wu Deng, ”Bearing fault diagnosis using transfer learning
and optimized deep belief network”,Measurement Science and Technology, vol.33, 2022
12. Abdul Wahid, John G. Breslin, Muhammad Ali Intizar, ”Prediction of Machine Failure in Industry 4.0: A Hybrid
CNN-LSTM Framework”, Appl. Sci., vol.12, 2022
13. A.Rivas, JM. Fraile, Pablo Chamoso, A.González-Briones, I.Sittón, JM. Corchado, ”A Predictive Maintenance
Model Using Recurrent Neural Networks”,advances in Intelligent Systems and Computing, vol.950, 2019
14. Dorina Weichert, Patrick Link, Anke Stoll, Stefan Ruping, Steffen Ihlenfeldt, Stefan Wrobel, ”A review of machine
learning for the optimization of production processes”, vol.104, pp 1889–1902, 2019
15. J.Schmitt, J.Bönig, T.Borggräfe, G.Beitinger, J.Deuse, ”Predictive model-based quality inspection using Machine
Learning and Edge Cloud Computing”, Advanced Engineering Informatics, vol.45,2020
16. Daniel Ramotsoela, Adnan Abu-Mahfouz, Gerhard Hancke, ”A Survey of Anomaly Detection in Industrial Wire-
less Sensor Networks with Critical Water System Infrastructure as a Case Study”, Sensors, vol.18, 2018
17. Dazhong Wu, Connor Jennings, Janis Terpenny, Robert X. Gao, Soundar Kumara, ”A Comparative Study on
Machine Learning Algorithms for Smart Manufacturing: Tool Wear Prediction Using Random Forests”, Vol.139
18. Durga Prasad Penumuru, Sreekumar Muthuswamy, Premkumar Karumbu, ”Identification and classification of
materials using machine vision and machine learning in the context of industry 4.0”, vol.31, pp 1229–1241, 2020
19. Hongyi Liu, Tongtong Fang, Tianyu Zhou, Yuquan Wang, Lihui Wang, ”Deep Learning-based Multimodal Control
Interface for Human-Robot Collaboration”, Vol.72, pp 3-8, 2018
20. Benjamin Rolf, Ilya Jackson, Marcel Müller, Sebastian Lang, Tobias Reggelin, Dmitry Ivanov, ”A review on
reinforcement learning algorithms and applications in supply chain management”, 2022
21. Yudha P. Pane, Subramanya P. Nageshrao, Jens Kober, Robert Babuška, ”Reinforcement learning based com-
pensation methods for robot manipulators”,Vol.78, pp 236-247, 2019
22. Ali Fırat ˙Inal, Ça ˘grı Sel, Adnan Aktepe, Ahmet Kür¸sad Türker, Süleyman Ersöz, ”A Multi-Agent Reinforce-
ment Learning Approach to the Dynamic Job Shop Scheduling Problem”, Sustainability, vol.15, 2023
23. Shiva Raj Pokhrel and Sahil Garg, ”Multipath Communication With Deep Q-Network For Industry 4.0 Automa-
tion and Orchestration”, Vol.17, pp 2852 - 2859, 2021
24. Bernd Waschneck, Andre Reichstaller, Lenz Belzner, Thomas Altenmuller, Thomas Bauernhansl, Alexander
Knapp, Andreas Kyek, ”Optimization of global production scheduling with deep reinforcement learning”, 51st
CIRP Conference on Manufacturing Systems, vol.72, pp 1264-1269, 2018
25. Ling Wang, Zixiao Pan, Jingjing Wang, ”A Review of Reinforcement Learning Based Intelligent Optimization for
Manufacturing Scheduling”, vol.1, pp 257-270, 2021
26. Rasmus E. Andersen, Steffen Madsen, Alexander B. K. Barloa, Sebastian B. Johansena, Morten Nøra, Simon
Bøgha, ”Self-learning Processes in Smart Factories: Deep Reinforcement Learning for Process Control of Robot
Brine Injection”,Vol.38, pp 171-177, 2019
27. F. Li, Y. Chen, J. Wang, X. Zhou, B. Tang, ”A reinforcement learning unit matching recurrent neural network
for the state trend prediction of rolling bearings”, Vol.145, pp 191-203, 2019
28. Md. Shirajum Munir, GSarder Fakhrul Abedin, Nguyen H. Tran,Eui-Nam Huh, and Choong Seon Hong, ”Risk-
Aware Energy Scheduling for Edge Computing With Microgrid: A Multi-Agent Deep Reinforcement Learning
Approach”, vol.18, pp 3476-3497, 2021
29. Xiangyu Ma, Wei Shi, ”AESMOTE: Adversarial Reinforcement Learning With SMOTE for Anomaly Detection”
,Vol.8, pp 943-956, 2021
30. Tamás Kegyes,Zoltán Süle,János Abonyi, ”The Applicability of Reinforcement Learning Methods in the Devel-
opment of Industry 4.0 Applications,” pp 1-31, 2021
31. Sabri Abderrazzak, Allaoui Hamid and Souissi Omar , ”Stochastic Dynamic Programming for Earliness-Tardiness
Single Machine Scheduling with Maintenance Considerations”, 2021
32. S.Abderrazzak, A.Hamid and S.Omar , ”Reinforcement learning and stochastic dynamic programming for jointly
scheduling jobs and preventive maintenance on a single machine to minimise earliness-tardiness”, 2023
33. Sabri Abderrazzak, Allaoui Hamid and Souissi Omar , ”Adaptive Large Neighborhood Search for the Just-In-Time
Job-shop Scheduling Problem”, 2022
34. Yuanyuan Li, Stefano Carabelli, Edoardo Fadda, Daniele Manerba, Roberto Tadei, Olivier Terzo1 ”Machine
learning and optimization for production rescheduling in Industry 4.0”, 2020
35. A. L. Samuel ”Some Studies in Machine Learning Using the Game of Checkers”, page 601-614, 1967
36. Mohammed M, Khan MB, Bashier Mohammed BE. ”Machine learning: algorithms and applications.” , 2016
37. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneer-
shelvam V ”Mastering the game of go with deep neural networks and tree search. nature.”, 2016
38. Gao et al. ”Industry 4.0: A Survey on Technologies, Applications and Open Research Issues” , 2019
39. Massimo Bertolini a, Davide Mezzogori b, Mattia Neroni b, Francesco Zammori b ”Machine Learning for industrial
applications: A comprehensive literature review”, 2021