Reinforcement Learning (RL) as a paradigm aims to develop algorithms that allow to train an agent... more Reinforcement Learning (RL) as a paradigm aims to develop algorithms that allow to train an agent to optimally achieve a goal with minimal feedback information about the desired behavior, which is not precisely specified. Scalar rewards are returned to the agent as response to its actions endorsing or opposing them. RL algorithms have been successfully applied to robot control design. The extension of the RL paradigm to cope with the design of control systems for Multi-Component Robotic Systems (MCRS) poses new challenges, mainly related to coping with scaling up of complexity due to the exponential state space growth, coordination issues, and the propagation of rewards among agents. In this paper, we identify the main issues which offer opportunities to develop innovative solutions towards fully-scalable cooperative multi-agent systems.
International Conference on Advanced Computing, Feb 8, 2014
Mining the Data is also known as Discovery of Knowledge in Databases, is to get correlations, tre... more Mining the Data is also known as Discovery of Knowledge in Databases, is to get correlations, trends, patterns, anomalies from the databases which can help out to make exact future decisions. However data mining is not the natural. No one can assure that the decision will lead to good quality results. It only helps experts to understand the data and lead to good decisions. Association Mining is the discovery of relations or correlations among an item set. An objective is to make rules from given multiple sources of customer database transaction. It needs increasingly deepening the knowledge mining process for finding refined knowledge from data. Earlier work is on mining association rules at one level. Though mining association rules at various levels is necessary. Finding of interesting association relationship among large amount of data will helpful to decision building, marketing, & business managing. For generating frequent item set we are using Apriori Algorithm in multiple levels so called Multilevel Relationship algorithm (MRA). MRA works in first two stages. In third stage of MRA uses Bayesian probability to find out the dependency & relationship among different shops, pattern of sales & generates the rule for learning. This paper gives detail idea about concepts of association mining, mathematical model development for Multilevel Relationship algorithm, Implementation & Result Analysis of MRA and performance comparison of MRA and Apriori algorithm.
In many applications, use of all relevant data to extract more information from multiple sources ... more In many applications, use of all relevant data to extract more information from multiple sources of information and achieve higher accuracy in prediction is desirable. Cooperative learning is observed in human and some animal societies. Sound knowledge and information acquisition, cooperation in learning amongst multi-agent systems may result in a higher effectiveness compared to individual learning. Cooperative learning in multi agent systems is generally expected to improve both quality & speed of learning. According to survey maximum research papers focus on coordinated approach of agents. Multiple sources of data can be viewed as different, related views of the same learning problem, where dependencies between the views could potentially take on complex structures. This gives rise to interesting and challenging machine learning problems where data sources are combined for learning. This framework encompasses several data fusion tasks and related topics, such as transfer learning, multitask learning, multiview learning, and learning under covariate shift. The advantages of the multiple source learning paradigm is seen in situations where individual data sources are noisy, incomplete, and learning from more than one source can filter out problem-independent noise. Cooperative learning is an approach where one or more team of learners work together towards reaching a better understanding of a specified task. The purpose of this paper is to use this approach to describe a proposal for designing and building a cooperative machine learning system (Multi-Learning system) that contains two or more machine learners that cooperate together.
Advances in intelligent systems and computing, 2018
Traffic crisis frequently happens because of traffic demands by the large number vehicles on the ... more Traffic crisis frequently happens because of traffic demands by the large number vehicles on the path. Increasing transportation move and decreasing the average waiting time of each vehicle are the objectives of cooperative intelligent traffic control system. Each signal wishes to catch better travel move. During the course, signals form a strategy of cooperation in addition to restriction for neighboring signals to exploit their individual benefit. A superior traffic signal scheduling strategy is useful to resolve the difficulty. The several parameters may influence the traffic control model. So it is hard to learn the best possible result. The lack of expertise of traffic light controllers to study from previous practice results makes them to be incapable of incorporating uncertain modifications of traffic flow. Defining instantaneous features of the real traffic scenario, reinforcement learning algorithm based traffic control model can be used to obtain fine timing rules. The projected real-time traffic control optimization model is able to continue with the traffic signal scheduling rules successfully. The model expands traffic value of the vehicle, which consists of delay time, the number of vehicles stopped at the signal, and the newly arriving vehicles to learn and establish the optimal actions. The experimentation outcome illustrates a major enhancement in traffic control, demonstrating the projected model is competent of making possible real-time dynamic traffic control.
Cooperation in learning (CL) can be understood in a multiagent system. In this the agents are cap... more Cooperation in learning (CL) can be understood in a multiagent system. In this the agents are capable of learning from both their own experiments and other agents' knowledge and expertise. Implementation of CL is a complicated task in the real world. In distributed systems several agents cooperate to achieve a common goal or accomplish a shared task. In particular, if there are different people or organizations with different goals and information, then a multiagent system (MAS) is needed to handle their interactions. In this paper, various issues related with cooperative machine learning are studied and implemented. A new set of improved cooperative learning algorithms is proposed in the paper. Expertness measuring criteria which were used in earlier work is further enhanced in proposed method. Six methods for measuring the agents' expertness are used i.e. Normal (Nrm), Absolute (Abs), Positive (P), Negative (N), Certainty (Cer) and Entropy (Ent). The novelty of this approach lies in the implementation of Weighted Strategy Sharing with expertness measuring criteria by means of Q-learning, Sarsa learning, Q(λ) and Sarsa(λ) learning algorithms. The paper shows implementation results and performance comparison of all these algorithms.
Communications in computer and information science, 2010
Abstract. Routing protocols for mobile ad hoc networks (MANETs) have been explored extensively in... more Abstract. Routing protocols for mobile ad hoc networks (MANETs) have been explored extensively in recent years. Much of this work is targeted at finding a feasible route from a source to a destination without considering current net-work traffic or application requirements. Therefore, the ...
Communications in computer and information science, 2016
We propose an innovative approach towards Cooperation Models for Multi-agent Reinforcement Learni... more We propose an innovative approach towards Cooperation Models for Multi-agent Reinforcement Learning (CMMARL) using reinforcement learning methods. Communication methods for reinforcement learning depend on multiagent scheme is proposed & implemented. Different cooperation methods for cooperative reinforcement learning based on expertness measure of each agent proposed here i.e. group method, dynamic method, goal-oriented method and expert agent method. Implementation results have demonstrated that the suggested communication and cooperation methods are able to accelerate the aggregation of the agents that accomplish best action strategies. This approach is developed for dynamic products availability in a three retailer shops in the market. Retailers can cooperate with each other and can get benefit from cooperative information by their own policies that accurately represent their goals and interests. The retailers are the learning agents in the problem and apply reinforcement learning to learn cooperatively from the situation. By making considerable theory on the dealer’s inventory strategy, refill period, and entry procedure of the customers, the problem turn out to be Markov decision process model thus facilitating to apply learning algorithms.
International Journal of Computer Trends and Technology, May 25, 2016
The output of the system is a sequence of actions in some applications. There is no such measure ... more The output of the system is a sequence of actions in some applications. There is no such measure as the best action in any in-between state; an action is excellent if it is part of a good policy. A single action is not important; the policy is important that is the sequence of correct actions to reach the goal. In such a case, machine learning program should be able to assess the goodness of policies and learn from past good action sequences to be able to generate a policy. A multi-agent environment is one in which there is more than one agent, where they interact with one another, and further, where there are restrictions on that environment such that agents may not at any given time know everything about the world that other agents know. Two features of multi-agent learning which establish its study as a separate field from ordinary machine learning. Parallelism, scalability, simpler construction and cost effectiveness are main characteristics of multi-agent systems. Multiagent learning model is given in this paper. Two multiagent learning algorithms i. e. Strategy Sharing & Joint Rewards algorithm are implemented. In Strategy Sharing algorithm simple averaging of Q tables is taken. Each Q-learning agent learns from all of its teammates by taking the average of Qtables. Joint reward learning algorithm combines the Q learning with the idea of joint rewards. Paper shows result and performance comparison of the two multiagent learning algorithms.
Cooperation in agent learning (CL) is understood in a multiagent environment. The agents are comp... more Cooperation in agent learning (CL) is understood in a multiagent environment. The agents are competent for learning from both other agents' knowledge and expertise and their own experience. This paper proposes a new move toward Enhanced Cooperative Multi-agent Learning Algorithms (ECMLA) using reinforcement learning methods. The paper shows the performance comparison between multi-agent learning algorithms and enhanced cooperative multi-agent learning algorithms using reinforcement learning methods. We explore a new approach for dynamic products availability in a three retailer shops in the market. Retailers can cooperate with each other and can get benefit from cooperative information by their own policies that accurately represent their goals and interests. The retailers are the learning agents in the problem and use reinforcement learning to learn cooperatively from the environment. By making considerable theory on the seller's inventory policy, refill period, and the arrival procedure of the customers, the problem turn out to be Markov decision process model thus facilitating to apply learning algorithms. The novelty of this approach lies in the enhanced implementation of the reinforcement learning by means of Sarsa learning and Sarsa(λ) learning algorithms. The paper shows implementation results and performance comparison between multi-agent learning algorithms i.e. Strategy Sharing and Joint Reward algorithm and proposed cooperative multi-agent learning algorithms using reinforcement learning methods.
We explore a new approach for dynamic products availability in a three retailer shops in the mark... more We explore a new approach for dynamic products availability in a three retailer shops in the market. Retailers can cooperate with each other and can get benefit from cooperative information by their own policies that accurately represent their goals and interests. The retailers are the learning agents in the system and use RL to learn cooperatively from the environment. The system becomes Markov decision process model on the basis of logical theory on the seller's inventory policy, the arrival process of the customers and refill times. Cooperation in learning (CL) can be understood in a multiagent system. The agents are capable of learning from both their own trials and other agents' knowledge. In this paper, we proposed a new approach for Advanced Cooperative Learning Algorithms using RL methods (ACLA). We have shown the performance comparison between cooperative learning algorithms and advanced cooperative learning algorithms using RL method with expertness measure. Expertness measuring criteria which were used in earlier work is further enhanced & improved in proposed method. Four methods for measuring the agents' expertness are used i.e. Normal (Nrm), Absolute (Abs), Positive (P), Negative (N). The novelty of this approach lies in the implementation of the RL algorithms with expertness measuring criteria by means of Sarsa learning and Sarsa(λ) learning algorithms. The paper shows implementation results and performance comparison of all these algorithms.
Reinforcement Learning (RL) as a paradigm aims to develop algorithms that allow to train an agent... more Reinforcement Learning (RL) as a paradigm aims to develop algorithms that allow to train an agent to optimally achieve a goal with minimal feedback information about the desired behavior, which is not precisely specified. Scalar rewards are returned to the agent as response to its actions endorsing or opposing them. RL algorithms have been successfully applied to robot control design. The extension of the RL paradigm to cope with the design of control systems for Multi-Component Robotic Systems (MCRS) poses new challenges, mainly related to coping with scaling up of complexity due to the exponential state space growth, coordination issues, and the propagation of rewards among agents. In this paper, we identify the main issues which offer opportunities to develop innovative solutions towards fully-scalable cooperative multi-agent systems.
International Conference on Advanced Computing, Feb 8, 2014
Mining the Data is also known as Discovery of Knowledge in Databases, is to get correlations, tre... more Mining the Data is also known as Discovery of Knowledge in Databases, is to get correlations, trends, patterns, anomalies from the databases which can help out to make exact future decisions. However data mining is not the natural. No one can assure that the decision will lead to good quality results. It only helps experts to understand the data and lead to good decisions. Association Mining is the discovery of relations or correlations among an item set. An objective is to make rules from given multiple sources of customer database transaction. It needs increasingly deepening the knowledge mining process for finding refined knowledge from data. Earlier work is on mining association rules at one level. Though mining association rules at various levels is necessary. Finding of interesting association relationship among large amount of data will helpful to decision building, marketing, & business managing. For generating frequent item set we are using Apriori Algorithm in multiple levels so called Multilevel Relationship algorithm (MRA). MRA works in first two stages. In third stage of MRA uses Bayesian probability to find out the dependency & relationship among different shops, pattern of sales & generates the rule for learning. This paper gives detail idea about concepts of association mining, mathematical model development for Multilevel Relationship algorithm, Implementation & Result Analysis of MRA and performance comparison of MRA and Apriori algorithm.
In many applications, use of all relevant data to extract more information from multiple sources ... more In many applications, use of all relevant data to extract more information from multiple sources of information and achieve higher accuracy in prediction is desirable. Cooperative learning is observed in human and some animal societies. Sound knowledge and information acquisition, cooperation in learning amongst multi-agent systems may result in a higher effectiveness compared to individual learning. Cooperative learning in multi agent systems is generally expected to improve both quality & speed of learning. According to survey maximum research papers focus on coordinated approach of agents. Multiple sources of data can be viewed as different, related views of the same learning problem, where dependencies between the views could potentially take on complex structures. This gives rise to interesting and challenging machine learning problems where data sources are combined for learning. This framework encompasses several data fusion tasks and related topics, such as transfer learning, multitask learning, multiview learning, and learning under covariate shift. The advantages of the multiple source learning paradigm is seen in situations where individual data sources are noisy, incomplete, and learning from more than one source can filter out problem-independent noise. Cooperative learning is an approach where one or more team of learners work together towards reaching a better understanding of a specified task. The purpose of this paper is to use this approach to describe a proposal for designing and building a cooperative machine learning system (Multi-Learning system) that contains two or more machine learners that cooperate together.
Advances in intelligent systems and computing, 2018
Traffic crisis frequently happens because of traffic demands by the large number vehicles on the ... more Traffic crisis frequently happens because of traffic demands by the large number vehicles on the path. Increasing transportation move and decreasing the average waiting time of each vehicle are the objectives of cooperative intelligent traffic control system. Each signal wishes to catch better travel move. During the course, signals form a strategy of cooperation in addition to restriction for neighboring signals to exploit their individual benefit. A superior traffic signal scheduling strategy is useful to resolve the difficulty. The several parameters may influence the traffic control model. So it is hard to learn the best possible result. The lack of expertise of traffic light controllers to study from previous practice results makes them to be incapable of incorporating uncertain modifications of traffic flow. Defining instantaneous features of the real traffic scenario, reinforcement learning algorithm based traffic control model can be used to obtain fine timing rules. The projected real-time traffic control optimization model is able to continue with the traffic signal scheduling rules successfully. The model expands traffic value of the vehicle, which consists of delay time, the number of vehicles stopped at the signal, and the newly arriving vehicles to learn and establish the optimal actions. The experimentation outcome illustrates a major enhancement in traffic control, demonstrating the projected model is competent of making possible real-time dynamic traffic control.
Cooperation in learning (CL) can be understood in a multiagent system. In this the agents are cap... more Cooperation in learning (CL) can be understood in a multiagent system. In this the agents are capable of learning from both their own experiments and other agents' knowledge and expertise. Implementation of CL is a complicated task in the real world. In distributed systems several agents cooperate to achieve a common goal or accomplish a shared task. In particular, if there are different people or organizations with different goals and information, then a multiagent system (MAS) is needed to handle their interactions. In this paper, various issues related with cooperative machine learning are studied and implemented. A new set of improved cooperative learning algorithms is proposed in the paper. Expertness measuring criteria which were used in earlier work is further enhanced in proposed method. Six methods for measuring the agents' expertness are used i.e. Normal (Nrm), Absolute (Abs), Positive (P), Negative (N), Certainty (Cer) and Entropy (Ent). The novelty of this approach lies in the implementation of Weighted Strategy Sharing with expertness measuring criteria by means of Q-learning, Sarsa learning, Q(λ) and Sarsa(λ) learning algorithms. The paper shows implementation results and performance comparison of all these algorithms.
Communications in computer and information science, 2010
Abstract. Routing protocols for mobile ad hoc networks (MANETs) have been explored extensively in... more Abstract. Routing protocols for mobile ad hoc networks (MANETs) have been explored extensively in recent years. Much of this work is targeted at finding a feasible route from a source to a destination without considering current net-work traffic or application requirements. Therefore, the ...
Communications in computer and information science, 2016
We propose an innovative approach towards Cooperation Models for Multi-agent Reinforcement Learni... more We propose an innovative approach towards Cooperation Models for Multi-agent Reinforcement Learning (CMMARL) using reinforcement learning methods. Communication methods for reinforcement learning depend on multiagent scheme is proposed & implemented. Different cooperation methods for cooperative reinforcement learning based on expertness measure of each agent proposed here i.e. group method, dynamic method, goal-oriented method and expert agent method. Implementation results have demonstrated that the suggested communication and cooperation methods are able to accelerate the aggregation of the agents that accomplish best action strategies. This approach is developed for dynamic products availability in a three retailer shops in the market. Retailers can cooperate with each other and can get benefit from cooperative information by their own policies that accurately represent their goals and interests. The retailers are the learning agents in the problem and apply reinforcement learning to learn cooperatively from the situation. By making considerable theory on the dealer’s inventory strategy, refill period, and entry procedure of the customers, the problem turn out to be Markov decision process model thus facilitating to apply learning algorithms.
International Journal of Computer Trends and Technology, May 25, 2016
The output of the system is a sequence of actions in some applications. There is no such measure ... more The output of the system is a sequence of actions in some applications. There is no such measure as the best action in any in-between state; an action is excellent if it is part of a good policy. A single action is not important; the policy is important that is the sequence of correct actions to reach the goal. In such a case, machine learning program should be able to assess the goodness of policies and learn from past good action sequences to be able to generate a policy. A multi-agent environment is one in which there is more than one agent, where they interact with one another, and further, where there are restrictions on that environment such that agents may not at any given time know everything about the world that other agents know. Two features of multi-agent learning which establish its study as a separate field from ordinary machine learning. Parallelism, scalability, simpler construction and cost effectiveness are main characteristics of multi-agent systems. Multiagent learning model is given in this paper. Two multiagent learning algorithms i. e. Strategy Sharing & Joint Rewards algorithm are implemented. In Strategy Sharing algorithm simple averaging of Q tables is taken. Each Q-learning agent learns from all of its teammates by taking the average of Qtables. Joint reward learning algorithm combines the Q learning with the idea of joint rewards. Paper shows result and performance comparison of the two multiagent learning algorithms.
Cooperation in agent learning (CL) is understood in a multiagent environment. The agents are comp... more Cooperation in agent learning (CL) is understood in a multiagent environment. The agents are competent for learning from both other agents' knowledge and expertise and their own experience. This paper proposes a new move toward Enhanced Cooperative Multi-agent Learning Algorithms (ECMLA) using reinforcement learning methods. The paper shows the performance comparison between multi-agent learning algorithms and enhanced cooperative multi-agent learning algorithms using reinforcement learning methods. We explore a new approach for dynamic products availability in a three retailer shops in the market. Retailers can cooperate with each other and can get benefit from cooperative information by their own policies that accurately represent their goals and interests. The retailers are the learning agents in the problem and use reinforcement learning to learn cooperatively from the environment. By making considerable theory on the seller's inventory policy, refill period, and the arrival procedure of the customers, the problem turn out to be Markov decision process model thus facilitating to apply learning algorithms. The novelty of this approach lies in the enhanced implementation of the reinforcement learning by means of Sarsa learning and Sarsa(λ) learning algorithms. The paper shows implementation results and performance comparison between multi-agent learning algorithms i.e. Strategy Sharing and Joint Reward algorithm and proposed cooperative multi-agent learning algorithms using reinforcement learning methods.
We explore a new approach for dynamic products availability in a three retailer shops in the mark... more We explore a new approach for dynamic products availability in a three retailer shops in the market. Retailers can cooperate with each other and can get benefit from cooperative information by their own policies that accurately represent their goals and interests. The retailers are the learning agents in the system and use RL to learn cooperatively from the environment. The system becomes Markov decision process model on the basis of logical theory on the seller's inventory policy, the arrival process of the customers and refill times. Cooperation in learning (CL) can be understood in a multiagent system. The agents are capable of learning from both their own trials and other agents' knowledge. In this paper, we proposed a new approach for Advanced Cooperative Learning Algorithms using RL methods (ACLA). We have shown the performance comparison between cooperative learning algorithms and advanced cooperative learning algorithms using RL method with expertness measure. Expertness measuring criteria which were used in earlier work is further enhanced & improved in proposed method. Four methods for measuring the agents' expertness are used i.e. Normal (Nrm), Absolute (Abs), Positive (P), Negative (N). The novelty of this approach lies in the implementation of the RL algorithms with expertness measuring criteria by means of Sarsa learning and Sarsa(λ) learning algorithms. The paper shows implementation results and performance comparison of all these algorithms.
Uploads
Papers by Dr. Deepak Vidhate