Learning From Sleeping Experts: Rewarding Informative, Available, and Accurate Experts
Article No.: 77, Pages 1 - 18
Abstract
We consider a generalized model of learning from expert advice in which experts could abstain from participating at some rounds. Our proposed online algorithm falls into the class of weighted average predictors and uses a time-varying multiplicative weight update rule. This update rule changes the weight of an expert based on his or her relative performance compared to the average performance of available experts at the current round. This makes the algorithm suitable for recommendation systems in the presence of an adversary with many potential applications in the new emerging area of the Internet of Things. We prove the convergence of our algorithm to the best expert, defined in terms of both availability and accuracy, in the stochastic setting. In particular, we show the applicability of our definition of best expert through convergence analysis of another well-known algorithm in this setting. Finally, through simulation results on synthetic and real datasets, we justify the out-performance of our proposed algorithms compared to the existing ones in the literature.
References
[1]
K. Agrawal, A. Vempaty, H. Chen, and P. K. Varshney. 2011. Target localization in wireless sensor networks with quantized data in the presence of byzantine attacks. In Proceedings of the 45th Asilomar Conference on Signals, Systems and Computers (ASILOMAR’11). 1669--1673.
[2]
R. Arnott, A. de Palma, and R. Lindsey. 1991. Does providing information to drivers reduce traffic congestion? Transport. Res. A 25, 5 (1991), 309--318.
[3]
S. Asur and B. Huberman. 2010. Predicting the future with social media. In Proceedings of 2010 IEEE/WIC/ACM International Conference on Web Intelligence. 492--499.
[4]
B. Awerbuch and R. Kleinberg. 2004. Adaptive routing with end-to-end feedback: Distributed learning and geometric approaches. In Proceedings of the 36th Annual ACM Symposium on Theory of Computing. ACM, New York, NY, 45--53.
[5]
N. C. Bianchi and G. Lugosi. 2006. Prediction, Learning, and Games. Cambridge University Press, Cambridge.
[6]
D. Blackwell. 1956. An analog of the minimax theorem for vector payoffs. Pacific J. Math. 6, 1 (1956), 1--8.
[7]
A. Blum and C. Burch. 1997. On-line learning and the metrical task system problem. In Proceedings of the 10th Annual Conference on Computational Learning Theory. ACM, New York, NY, 45--53.
[8]
A. Blum, C. Burch, and A. Kalai. 1999. Finely-competitive paging. In Proceedings of the 40th Annual Symposium on Foundations of Computer Science. 450--457.
[9]
A. Blum and Y. Mansour. 2007. From external to internal regret. J. Mach. Learn. Res. 8 (Dec. 2007), 1307--1324.
[10]
V. Borkar. 2008. Stochastic Approximation: A Dynamical Systems Viewpoint. Cambridge University Press, Cambridge.
[11]
Y. Freund, R. E. Schapire, Y. Singer, and M. K. Warmuth. 1997. Using and combining predictors that specialize. In Proceedings of the 29th Annual ACM Symposium on the Theory of Computing. ACM, New York, NY, 334--343.
[12]
András György and György Ottucsák. 2006. Adaptive routing using expert advice. Comput. J. 49, 2 (Mar. 2006), 180--189.
[13]
J. Hannan. 1957. Approximation to bayes risk in repeated play. Contributions to the Theory of Games 3 (1957), 97--139.
[14]
M. Joshi, D. Das, K. Gimpel, and N. Smith. 2010. Movie reviews and revenues: An experiment in text regression. In Proceedings of the North American Chapter of the Association for Computational Linguistics Human Language Technologies Conference.
[15]
A. Kalai and S. Vempala. 2005. Efficient algorithms for online decision problems. J. Comput. System Sci. 71, 3 (Oct. 2005), 291--307.
[16]
V. Kanade, B. McMahan, and B. Bryan. 2009. Sleeping experts and bandits with stochastic action availability and adversarial rewards. In Proceedings of the 12th International Conference on Artificial Intelligence and Statistics. Florida, USA, 272--279.
[17]
R. D. Kleinberg, A. Niculescu-Mizil, and Y. Sharma. 2008. Regret bounds for sleeping experts and bandits. In Proceedings of the 21st Annual Conference on Learning Theory (COLT’08). 425--436.
[18]
N. Littlestone and M. K. Warmuth. 1989. The weighted majority algorithm. In Proceedings of the 30th Annual Symposium on Foundations of Computer Science. NC, 212--261.
[19]
L. Liu, J. Xu, S. Liao, and H. Chen. 2014. A real-time personalized route recommendation system for self-drive tourists based on vehicle to vehicle communication. Expert Syst. Appl. 41, 7 (2014), 3409--3417.
[20]
A. Truong, S. R. Etesami, J. Etesami, and N. Kiyavash. 2017. Optimal attack strategies against predictors - learning from expert advice. IEEE Trans. Inf. Forens. Secur. 13, 1 (2017), 6--19.
[21]
A. Truong and N. Kiyavash. 2013. Optimal adversarial strategies in learning with expert advice. In Proceedings of the 52th IEEE Conference on Decision and Control. Florence, Italia, 7315--7320.
[22]
A. Truong, N. Kiyavash, and V. Borkar. 2011. Convergence analysis for an online recommendation system. In Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference. 3889--3894.
[23]
A. Vempaty, O. Ozdemir, K. Agrawal, H. Chen, and P. K. Varshney. 2013. Localization in wireless sensor networks: Byzantines and mitigation techniques. IEEE Trans. Sign. Process. 61, 6 (2013), 1495--1508.
[24]
A. Vempaty, O. Ozdemir, and P. K. Varshney. 2012. Mitigation of byzantine attacks for target location estimation in wireless sensor networks. In Proceedings of 46th Annual Conference on Information Sciences and Systems (CISS’12). 1--6.
[25]
V. G. Vovk. 1990. Aggregating strategies. In Proceedings of the 3rd Annual Workshop on Computational Learning Theory (COLT’90). San Francisco, CA, USA, 371--386.
[26]
H. Yu, C. Shi, M. Kaminsky, P. B. Gibbons, and F. Xiao. 2009. DSybil: Optimal sybil-resistance for recommendation systems. In Proceedings of the 2009 30th IEEE Symposium on Security and Privacy. 283--298.
[27]
W. Zhang and S. Skiena. 2009. Improving movie gross prediction through news analysis. In Proceedings of 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology. 301--304.
Index Terms
- Learning From Sleeping Experts: Rewarding Informative, Available, and Accurate Experts
Recommendations
Integrating machine learning with knowledge acquisition through direct interaction with domain experts
Knowledge elicitation from experts and empirical machine learning are two distinct approaches to knowledge acquisition with differing and mutually complementary capabilities. Learning apprentices have provided environments in which a knowledge engineer ...
Task-Structures, Knowledge Acquisition and Learning
One of the old saws about learning in AI is that an agent can only learn what it can be told, i.e., the agent has to have a vocabulary for the target structure which is to be acquired by learning. What this vocabulary is, for various tasks, is an issue ...
Knowledge acquisition from multiple experts
Special issue on knowledge acquisitionThe traditional method of knowledge acquisition when constructing a knowledge-based (expert) system is to depend primarily upon a single source, usually an expert in the domain under consideration. This approach has several advantages: no need to ...
Comments
Information & Contributors
Information
Published In
November 2018
288 pages
Copyright © 2018 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Journal Family
Publication History
Published: 06 November 2018
Accepted: 01 June 2018
Revised: 01 May 2018
Received: 01 October 2017
Published in TODAES Volume 23, Issue 6
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 175Total Downloads
- Downloads (Last 12 months)17
- Downloads (Last 6 weeks)1
Reflects downloads up to 25 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in