Deep Reinforcement Learning

This paper concentrates on the design, analysis, and development of fixed-wing hand launch unmanned aerial vehicle (UAV). This flight can able to carry the payloads of 0.8. The design process involves the conceptual, preliminary, and detailed design. This paper involves the investigation of the aerodynamic characteristics over the wing to enhance the aerodynamic design of the UAV. This analysis includes estimating the best gliding ratio to increase the flight mission and attain the maximum altitude. This simulation will be performed for subsonic flow with Mach number 0.04202(14.3m/s). The manufacturing of the UAV is done using composite materials like glass fiber of both (1mm and 2 mm) thickness, carbon fiber of 2mm, and carbon rod is used for connecting the empennage to the fuselage. The detailed design has been done in CATIA V5 and the analysis of the wing has been done using XFLR, ANSYS (fluent).

In recent years some researchers have explored the use of reinforcement learning (RL) algorithms as key components in the solution of various natural language processing tasks. For instance, some of these algorithms leveraging deep neural learning have found their way into conversational systems. This paper reviews the state of the art of RL methods for their possible use for different problems of natural language processing, focusing primarily on conversational systems, mainly due to their growing relevance. We provide detailed descriptions of the problems as well as discussions of why RL is well-suited to solve them. Also, we analyze the advantages and limitations of these methods. Finally, we elaborate on promising research directions in natural language processing that might benefit from reinforcement learning.

This paper concentrates on the design, analysis, and development of fixed-wing hand launch unmanned aerial vehicles (UAV). This flight can able to carry payloads of 0.8. The design process involves the conceptual, preliminary, and detailed design. This paper involves the investigation of the aerodynamic characteristics over the wing to enhance the aerodynamic design of the UAV. This analysis includes estimating the best gliding ratio to increase the flight mission and attain the maximum altitude. This simulation will be performed for subsonic flow with Mach number 0.04202(14.3m/s). The manufacturing of the UAV is done using composite materials like glass fiber of both (1mm and 2 mm) thickness, carbon fiber of 2mm, and carbon rod is used for connecting the empennage to the fuselage. The detailed design has been done in CATIA V5 and the analysis of the wing has been done using XFLR, ANSYS (fluent).

Forged documents specifically passport, driving licence and VISA stickers are used for fraud purposes including robbery, theft and many more. So detecting forged characters from documents is a significantly important and challenging task in digital forensic imaging. Forged characters detection has two big challenges. First challenge is, data for forged characters detection is extremely difficult to get due to several reasons including limited access of data, unlabeled data or work is done on private data. Second challenge is, deep learning (DL) algorithms require labeled data, which poses a further challenge as getting labeled is tedious, time-consuming, expensive and requires domain expertise. To end these issues, in this paper we propose a novel algorithm, which generates the three datasets namely forged characters detection for passport (FCD-P), forged characters detection for driving licence (FCD-D) and forged characters detection for VISA stickers (FCD-V). To the best of our kn...

In this way, during a pilot project to evaluate the potential and return of the application of Artificial Intelligence in a water concession company in the Latin American region executed in early 2019, we face this very pressing problem and, taking care of that Digital DNA that makes us different, the answer to the problem of NRW had to be equally disruptive. So it was. Very quickly, in the initial phases of our “GTD10X” methodology especially aimed at digital adoption by large corporations, we understood the intrinsic difficulties arising from the great heterogeneity in the planning, management and maintenance of urban systems.

The emerging cryptocurrency market has lately received great attention for asset allocation due to its decentralization uniqueness. However, its volatility and brand new trading mode has made it challenging to devising an acceptable automatically-generating strategy. This study proposes a framework for automatic high-frequency bitcoin transactions based on a deep reinforcement learning algorithm-proximal policy optimization (PPO). The framework creatively regards the transaction process as actions, returns as awards and prices as states to align with the idea of reinforcement learning. It compares advanced machine learning-based models for static price predictions including support vector machine (SVM), multi-layer perceptron (MLP), long short-term memory (LSTM), temporal convolutional network (TCN), and Transformer by applying them to the real-time bitcoin price and the experimental results demonstrate that LSTM outperforms. Then an automatically-generating transaction strategy is constructed building on PPO with LSTM as the basis to construct the policy. Extensive empirical studies validate that the proposed method perform superiorly to various common trading strategy benchmarks for a single financial product. The approach is able to trade bitcoins in a simulated environment with synchronous data and obtains a 31.67% more return than that of the best benchmark, improving the benchmark by 12.75%. The proposed framework can earn excess returns through both the period of volatility and surge, which opens the door to research on building a single cryptocurrency trading strategy based on deep learning. Visualizations of trading the process show how the model handles high-frequency transactions to provide inspiration and demonstrate that it can be expanded to other financial products.

Nowadays, distance learning becomes more diverse and popular. Increasingly universities are currently working to offer their online courses (MOOC, SPOC, SMOC, SSOC, etc.) in the form of courses providing learners with a wide variety of choices. However, this multi-criteria choice is complex. In this paper, we propose a personalized recommendation system based on Deep Reinforcement Learning that suggests for learners a most appropriate course according to specificities of each one such as their profile, needs and competences. To validate our system, the later has been tested over a set of real students. The obtained results of our study are in favor of the robustness of our system.

In this paper, we propose a set of algorithms to design signal
timing plans via deep reinforcement learning.  The core idea of
this approach is to set up a deep neural network (DNN) to learn
the Q-function of reinforcement learning from the sampled traffic
state/control inputs and the corresponding traffic system
performance output. Based on the obtained DNN, we can find the
appropriate signal timing policies by implicitly modeling the
control actions and the change of system states. We explain the
possible benefits and implementation tricks of this new approach.
The relationships between this new approach and some existing
approaches are also carefully discussed.

Citation: Li Li, Yisheng Lv, Fei-Yue Wang. Traffic signal timing via deep reinforcement learning. IEEE/CAA Journal of Automatica Sinica, 2016, 3(3): 247-254

Video abstract: YouTube: https://www.youtube.com/watch?v=0tlF35qJ0sE&feature=youtu.be;
Youku: http://v.youku.com/v_show/id_XMTcxMDcwMzU2NA==.html?beta&

DEEP REINFORCEMENT LEARNING - 2 ECTS | ADVANCED BACHELOR'S, 375-650 € ON CAMPUS. SESSION 2: 18-22 JULY 2022

Reinforcement Learning (RL) is a set of techniques that can be used to solve sequential decision-making tasks. When combined with deep learning as a function approximator, these algorithms are referred to as deep RL and they have been able to solve a variety of tasks that were previously out of reach for a machine (e.g. the game of Go). Thus, deep RL opens up many new applications in domains such as healthcare, robotics, smart grids, finance, and many more.

This course will provide an introduction to deep reinforcement learning techniques and algorithms. Particular focus will be given to aspects related to generalization and how deep RL can be used for practical applications. The course will need of some basic knowledge in RL but will provide the theoretical foundations on the topics covered as well as an introduction to the practical skills necessary to apply (deep) RL techniques. Students will also improve their coding skills by developing (deep) RL agents.

The advanced topics covered during the course will cover algorithms with rich structures that allow for the combination of reasoning and learning as well as how one agent can learn useful abstract representations. Concepts such as temporal abstraction, meta-learning, the reality gap and lifelong learning will also be discussed so that the students get a good understanding of the current challenges in research. Examples of applications such as video games, robotics, self-driving cars, etc. will also be covered.

APPLICATION DEADLINES & DISCOUNTS Apply before the 1st May 2022. To get our Early Bird Discount (-50€) apply before 15 March 2022. Visit our website for more information.

The Vrije Universiteit Amsterdam (VU) is an internationally renowned research university. Since its founding in 1880 interdisciplinary research has been leading in its education. The university consists of nine faculties with teaching facilities for 24.000 students. With over 140 English-taught Bachelor's and Master's programmes, the university offers a wide range of study options. The campus is a centrally located hub where people easily feel at home. https://vu.nl/en/education/summerschool | AMSTERDAMSUMMERSCHOOL@VU.NL | +31 20 598 4717

Data augmentation has been broadly applied in training deep-learning models to increase the diversity of data. This study ingestigates the effectiveness of different data augmentation methods for deep-learning based human intention prediction when only limited training data is available. A human participant pitches a ball to nine potential targets in our experiment. We expect to predict which target the participant pitches the ball to. Firstly, the effectiveness of 10 data augmentation groups is evaluated on a single-participant data set using RGB images. Secondly, the best data augmentation method (i.e., random cropping) on the single-participant data set is further evaluated on a multi-participant data set to assess its generalization ability. Finally, the effectiveness of random cropping on fusion data of RGB images and optical flow is evaluated on both single- and multi-participant data sets. Experiment results show that: 1) Data augmentation methods that crop or deform images can improve the prediction performance; 2) Random cropping can be generalized to the multi-participant data set (prediction accuracy is improved from 50% to 57.4%); and 3) Random cropping with fusion data of RGB images and optical flow can further improve the prediction accuracy from 57.4% to 63.9% on the multi-participant data set.

Developing efficient embedded vision applications requires exploring various algorithmic optimization trade-offs and a broad spectrum of hardware architecture choices. This makes navigating the solution space and finding the design points with optimal performance trade-offs a challenge for developers. To help provide a fair baseline comparison, we conducted comprehensive benchmarks of accuracy, run-time, and energy efficiency of a wide range of vision kernels and neural networks on multiple embedded platforms: ARM57 CPU, Nvidia Jetson TX2 GPU and Xilinx ZCU102 FPGA. Each platform utilizes their optimized libraries for vision kernels (OpenCV, VisionWorks and xfOpenCV) and neural networks (OpenCV DNN, TensorRT and Xilinx DPU). For vision kernels, our results show that the GPU achieves an energy/frame reduction ratio of 1.1-3.2× compared to the others for simple kernels. However, for more complicated kernels and complete vision pipelines, the FPGA outperforms the others with energy/frame reduction ratios of 1.2-22.3×. For neural networks [Inception-v2 and ResNet-50, ResNet-18, Mobilenet-v2 and SqueezeNet], it shows that the FPGA achieves a speed up of [2.5, 2.1, 2.6, 2.9 and 2.5]× and an EDP reduction ratio of [1.5, 1.1, 1.4, 2.2 and 1.5]× compared to the GPU FP16 implementations, respectively.

Deep reinforcement learning enables algorithms to learn complex behavior, deal with continuous action spaces and find good strategies in environments with high dimensional state spaces. With deep reinforcement learning being an active area of research and many concurrent inventions, we decided to focus on a simple robotic task to evaluate a set of ideas that might help to solve recent reinforcement learning problems. The focus is on enabling distributed set up to execute and run experiments with the least amount of time and benefit from the available computational power. Another focus is on the transferability between different physics engines, where we experiment on how to use a trained agent from one environment into another different environment with a different physics engine.
The purpose of this thesis is to unify the differences between reinforcement learning environments by implementing simple abstract classes between the selected environments which can be extended to support more environment. With this, the focus was on setting and enabling distribution for training to reduce the time of the experiment. We select two of the state of the art reinforcement learning methods to train, evaluate and test the distributed and transferability. The goal of this strategy is to reduce training time and eventually help the algorithms to scale, collect experiences, and train the agents effectively. The concluding evaluation and results prove the general applicability of the described concepts by testing them using selected environments. In our experiments, the effect of distribution was shown in the training time between the experiments. Furthermore, the last performed experiment we demonstrate how to use transfer learning and trained agents in a new learning environment. These concepts might be reused for future experiments.

The paper attempts to analyze if the sentiment stability of financial 10-K reports over time can determine the company’s future mean returns. A diverse portfolio of stocks was selected to test this hypothesis. The proposed framework downloads 10-K reports of the companies from SEC’s EDGAR database. It passes them through the preprocessing pipeline to extract critical sections of the filings to perform NLP analysis. Using Loughran and McDonald sentiment word list, the framework generates sentiment TF-IDF from the 10-K documents to calculate the cosine similarity between two consecutive 10-K reports and proposes to leverage this cosine similarity as the alpha factor. For analyzing the effectiveness of our alpha factor at predicting future returns, the framework uses the alphalens library to perform factor return analysis, turnover analysis, and for comparing the Sharpe ratio of potential alpha factors. The results show that there exists a strong correlation between the sentiment stability of our portfolio’s 10-K statements and its future mean returns. For the benefit of the research community, the code and Jupyter notebooks related to this paper have been open-sourced on Github1.

By applying RapidMiner workflows has been processed a dataset originated from different data files, and containing information about the sales over three years of a large chain of retail stores. Subsequently, has
been constructed a Deep Learning model performing a predictive algorithm suitable for sales forecasting. This model is based on artificial neural network –ANN- algorithm able to learn the model starting from
sales historical data and by pre-processing the data. The best built model uses a multilayer neural network together with an “optimized operator” able to find automatically the best parameter setting of the
implemented algorithm.

In many different fields, there is a high demand for storing information to a computer storage disk from the data available in printed or handwritten documents or images so that it can be reutilized later. One simple way is to scan the documents and store them into images but to re-utilize this data for further processing is very difficult e.g. calculating the net amount from scanned bills is a laborious task. It uses two main approaches to accomplish this task: classifying words directly and character segmentation. For the former, it uses Machine Learning technique i.e. Convolutional Neural Network (CNN) with various architectures to train a model that can accurately classify words. For the latter, it uses Long Short Term Memory networks (LSTM) with convolution to construct bounding boxes for each character. It then pass the segmented characters to a CNN for classification, and then reconstruct each word according to the results of classification and segmentation. It uses real time scanning of the evaluation sheet is done and the data is stored in the excel sheet using the proposed feature representation.

Accurate Object Detection was always a big deal and an important part of the Information Technology era. After the arrival of Machine Learning and Deep Learning technologies, the efficiency and accuracy for Object Detection increased significantly. These technologies greatly assisted in the evolution of the computer vision systems. This project focuses to integrate state-of-the-art technique for object detection with the aim of the achieving high accuracy. In this project we are using a deep learning part which is Tensorflow and OpenCV, which is a library of programming functions mainly aimed at real-time computer vision. There are few other libraries used which helped in object detection to make the system more accurate and reliable in the long run. We trained the network on various objects which are ordinary and easily available in the market. This projects aim to reduce the billing time in super markets with fast and accurate detection.

Many construction buildings and Structural barrier members that was built before 90s in USA, they weren’t designed for Seismic resistance until Federal Emergency Management Agency FEMA started to do seismic evaluation and rehabilitation and suggested reinforcement methods. In this article we evaluate FRP sheet reinforcement.

Reinforcement learning (RL) algorithms have been around for decades and employed to solve various sequential decision-making problems. These algorithms however have faced great challenges when dealing with high-dimensional environments. The recent development of deep learning has enabled RL methods to drive optimal policies for sophisticated and capable agents, which can perform efficiently in these challenging environments. This paper addresses an important aspect of deep RL related to situations that demand multiple agents to communicate and cooperate to solve complex tasks. A survey of different approaches to problems related to multi-agent deep RL (MADRL) is presented, including non-stationarity, partial observability, continuous state and action spaces, multi-agent training schemes, multi-agent transfer learning. The merits and demerits of the reviewed methods will be analyzed and discussed, with their corresponding applications explored. It is envisaged that this review provides insights about various MADRL methods and can lead to future development of more robust and highly useful multi-agent learning methods for solving real-world problems.

Automation is a powerful word that lies everywhere. It shows that without automation, application will not get developed. In a semiconductor industry, artificial intelligence played a vital role for implementing the chip based design through automation .The main advantage of applying the machine learning & deep learning technique is to improve the implementation rate based upon the capability of the society. The main objective of the proposed system is to apply the deep learning using data driven approach for controlling the system. Thus leads to a improvement in design, delay ,speed of operation & costs. Through this system, huge volume of data’s that are generated by the system will also get control.

3rd International Conference on Advances in Artificial Intelligence Techniques (ArIT 2022) will provide an excellent international forum for sharing knowledge and results in theory, methodology and applications of Artificial Intelligence and its advances. The Conference looks for significant contributions to all major fields of the Artificial Intelligence in theoretical and practical aspects. The aim of the Conference is to provide a platform to the researchers and practitioners from both academia as well as industry to meet and share cutting-edge development in the field.

Deep learning (DL) algorithms have recently emerged from machine learning and soft computing techniques. Since then, several deep learning (DL) algorithms have been recently introduced to scientific communities and are applied in various application domains. Today the usage of DL has become essential due to their intelligence, efficient learning, accuracy and robustness in model building. However, in the scientific literature, a comprehensive list of DL algorithms has not been introduced yet. This paper provides a list of the most popular DL algorithms, along with their applications domains.

The scale of Internet-connected systems has increased considerably, and these systems are being exposed to cyber attacks more than ever. The complexity and dynamics of cyber attacks require protecting mechanisms to be responsive, adaptive, and large-scale. Machine learning, or more specifically deep reinforcement learning (DRL), methods have been proposed widely to address these issues. By incorporating deep learning into traditional RL, DRL is highly capable of solving complex, dynamic, and especially high-dimensional cyber defense problems. This paper presents a survey of DRL approaches developed for cyber security. We touch on different vital aspects , including DRL-based security methods for cyber-physical systems, autonomous intrusion detection techniques, and multi-agent DRL-based game theory simulations for defense strategies against cyber attacks. Extensive discussions and future research directions on DRL-based cyber security are also given. We expect that this comprehensive review provides the foundations for and facilitates future studies on exploring the potential of emerging DRL to cope with increasingly complex cyber security problems.

Deep learning has taken over - both in problems beyond the realm of traditional, hand-crafted machine learning paradigms as well as in capturing the imagination of the practitioner sitting on top of petabytes of data. While the public perception about the efficacy of deep neural architectures in complex pattern recognition tasks grows, sequentially up-to-date primers on the current state of affairs must follow. In this review, we seek to present a refresher of the many different stacked, connectionist networks that make up the deep learning architectures followed by automatic architecture optimization protocols using multiagent approaches. Further, since guaranteeing system uptime is
fast becoming an indispensable asset across multiple industrial modalities, we include an investigative section on testing neural networks for fault detection and subsequent mitigation. This is followed by an exploratory survey of several application areas where deep learning has emerged as a game-changing technology - be it anomalous behavior detection in financial applications or financial time-series forecasting, predictive and prescriptive analytics, medical imaging, natural language processing or power systems research. The thrust of this review is on outlining emerging areas of application-oriented research within the deep learning community as well as to provide a handy reference to researchers seeking to embrace deep learning in their work for what it is: statistical pattern recognizers with unparalleled hierarchical structure learning capacity with the ability to scale
with information.

Deep learning applications, especially multilayer neural network models, result in network intrusion detection with high accuracy. This study proposes a model that combines a multilayer neural network with Dense Sparse Dense (DSD) multi-stage training to simultaneously improve the criteria related to the performance of intrusion detection systems on a comprehensive dataset UNSW-NB15. We conduct experiments on many neural network models such as Recurrent Neural Network (RNN), Long-Short Term Memory (LSTM), Gated Recurrent Unit (GRU), etc. to evaluate the combined efficiency with each model through many criteria such as accuracy, detection rate, false alarm rate, precision, and F1-Score.

Today’s cyber defense capabilities in many organizations consist of a diversity of tools, products, and solutions, which are very challenging for Security Operations Centre (SOC) teams to manage in current advanced and dynamic cyber threat environments. Security researchers and industry practitioners have proposed security orchestration, automation, and response (SOAR) solutions designed to integrate and automate the disparate security tasks, processes, and applications in response to security incidents to empower SOC teams. The next big step for cyber threat detection, mitigation, and prevention efforts is to leverage AI/ML in SOAR solutions. AI/ML will act as a force multiplier empowering SOC analysts even further. We conducted a detailed survey by studying work by both security researchers and industry practitioners on SOAR, including its interpretations, from an AI/ML perspective by reviewing works published in academic journals, conferences, websites, blogs, white papers, etc. (a multi-vocal view). We report on our findings and future research directions in this area.

Iris is a powerful tool for reliable human identification. It has the potential to identify individuals with a high degree of assurance. Extracting good features is the most significant step in the iris recognition system. In the past, different features have been used to implement iris recognition system. Most of them are depend on hand-crafted features designed by biometrics specialists. Due to the success of deep learning in computer vision problems, the features learned by the Convolutional Neural Network (CNN) have gained much attention to be applied for iris recognition system. In this paper, we evaluate the extracted learned features from a pre-trained Convolutional Neural Network (Alex-Net Model) followed by a multi-class Support Vector Machine (SVM) algorithm to perform classification. The performance of the proposed system is investigated when extracting features from the segmented iris image and from the normalized iris image. The proposed iris recognition system is tested on four public datasets IITD, iris databases CASIA-Iris-V1, CASIA-Iris-thousand and, CASIA-Iris-V3 Interval. The system achieved excellent results with the very high accuracy rate.

Speech impediment affecting children with hearing difficulties and speech disorders requires speech therapy and much practice to overcome. To motivate the children to practice more, serious games can be used because children are more inclined to play games. In this paper, we have designed and implemented a serious game in which children can learn to speak specific words that they are expected to know before the age of 7. The game consists of an avatar controlled by the child through speech, with the objective of moving the avatar around the environment to earn coins. The avatar is controlled by voice commands such as Jump, Ahead, Back, Left, Right. Children will be guided by an arrow during the game instead of a getting help from a therapist or a teacher to guide the child to the next coin. This allows the child to practice longer hours, compared to clinical approaches under the supervision of a therapist, which are time-limited.

Reinforcement learning (RL) has distinguished itself as a prominent learning method to augment the efficacy of autonomous systems. Recent advances in deep learning studies have complemented existing RL methods and led to a crucial breakthrough in the effort of applying RL to automation and robotics. Artificial agents based on deep RL can take selective and intelligent actions comparable with those of a human to maximize the feedback reward from the interactive environment. In this paper, we survey recent developments in the literature regarding deep RL methods for building human-level agents. As a result, prominent studies that involve modeling every aspect of a human-level agent will be examined. We also provide an overview of constructing a framework for prospective autonomous systems. Moreover, various toolkits and frameworks are suggested to facilitate the development of deep RL methods. Finally, we open a discussion that potentially raises a range of future research directions in deep RL.

In this paper, a streamlined working pipeline for an end-to-end deep reinforcement learning framework for autonomous driving was introduced. It integrates the usage of a choice combination of Algorithm-Policy for training the simulator by streamlining the integration of Microsoft AirSim, OpenGym, and Stable-Baselines. The pipeline is tested for goal reaching (represented by waypoints) with penalized collision. Several algorithm-policy combinations were tested and performance was measured using success rate (reaching the goal), number of collisions, and goal distance. Several of the models reached 30% to 100% success rates over 100 test episodes demonstrating the learning of autonomous goal reaching and collision avoidance.

The paper introduces a novel approach to the classical adaptive traffic signal control (TSC) problem. Instead of the traditional optimization or simple rule-based approach, Artificial Intelligence is applied. Reinforcement Learning is a spectacularly evolving realm of Machine Learning which owns the key features such as generalization, scalability, real-time applicability for solving the traffic signal control problem. Nevertheless, the researchers' responsibilities become more serious regarding the formulation of state representation and the rewarding system. These Reinforcement Learning features are also the most fascinating and controversial virtues since the utilized abstractions decide whether the algorithm solves the problem or not. This paper proposes a new interpretation for the feature-based state representation and rewarding concept that makes the TSC problem highly generalizable and has scaling potential. The proposed method's feasibility is demonstrated via a simulation study using a high-fidelity microscopic traffic simulator. The results justify that the Deep Reinforcement Learning based approach is a real candidate for real-time traffic light control.

The LoRa wireless connectivity has become a de facto technology for intelligent critical infrastructures such as transport systems. Achieving high Quality of Service (QoS) in cooperative systems remains a challenging task in LoRa. However, high QoS can be achieved via optimizing the transmission policy parameters such as spreading factor, bandwidth, code rate and carrier frequency. Yet existing approaches have not optimized the complete LoRa parameters. Furthermore, the star of stars topology used by LoRa causes more energy consumption and a low packet reception ratio. Motivated by this, this paper presents transmission policy enforcement and multi-hop routing for QoS-aware LoRa networks (MQ-LoRa). A hybrid cluster root rotated tree topology is constructed in which gateways follow a tree topology and Internet of Things (IoT) nodes follow a cluster topology. A 'membrane' inspired form the cell tissues which form clusters to sharing the correct information. The membrane inspired clustering algorithm is developed to form clusters and an optimal header node is selected using the influence score. Data QoS ranking is implemented for IoT nodes where priority and non-priority information is identified by the new field of LoRa frame structure (QRank). The optimal transmission policy enforcement uses fast deep reinforcement learning called Soft Actor Critic (SAC) that utilizes the environmental parameters including QRank, signal quality and signal-to-interference-plus-noise-ratio. The transmission policy is optimized with respect to the spreading factor, code rate, bandwidth and carrier frequency. Then, a concurrent optimization multi-hop routing algorithm that uses mayfly and shuffled shepherd optimization to rank routes based on the fitness criteria. Finally, a weighted duty cycle is implemented using a multi-weighted sum model to reduce resource wastage and information loss in LoRa IoT networks. Performance evaluation is implemented using a NS3.26 LoRaWAN module. The performance is examined for various metrics such as packet reception ratio, packet rejection ratio, energy consumption, delay and throughput. Experimental results prove that the proposed MQ-LoRa outperforms the well-known LoRa methods.

The popularity of deep reinforcement learning (DRL) applications in economics has increased exponentially. DRL, through a wide range of capabilities from reinforcement learning (RL) to deep learning (DL), offers vast opportunities for handling sophisticated dynamic economics systems. DRL is characterized by scalability with the potential to be applied to high-dimensional problems in conjunction with noisy and nonlinear patterns of economic data. In this paper, we initially consider a brief review of DL, RL, and deep RL methods in diverse applications in economics, providing an in-depth insight into the state-of-the-art. Furthermore, the architecture of DRL applied to economic applications is investigated in order to highlight the complexity, robustness, accuracy, performance, computational tasks, risk constraints, and profitability. The survey results indicate that DRL can provide better performance and higher efficiency as compared to the traditional algorithms while facing real economic problems in the presence of risk parameters and the ever-increasing uncertainties.

This paper concentrates on the design, analysis, and development of fixed-wing hand launch unmanned aerial vehicle (UAV). This flight can able to carry the payloads of 0.8. The design process involves the conceptual, preliminary, and detailed design. This paper involves the investigation of the aerodynamic characteristics over the wing to enhance the aerodynamic design of the UAV. This analysis includes estimating the best gliding ratio to increase the flight mission and attain the maximum altitude. This simulation will be performed for subsonic flow with Mach number 0.04202(14.3m/s). The manufacturing of the UAV is done using composite materials like glass fiber of both (1mm and 2 mm) thickness, carbon fiber of 2mm, and carbon rod is used for connecting the empennage to the fuselage. The detailed design has been done in CATIA V5 and the analysis of the wing has been done using XFLR, ANSYS (fluent).