research-article

Social Skill Validation at LinkedIn

Authors:

Mikhail Obukhov,

Qi HeAuthors Info & Claims

KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Pages 2943 - 2951

https://doi.org/10.1145/3292500.3330752

Published: 25 July 2019 Publication History

Abstract

The main mission of LinkedIn is to connect 610M+ members to the right opportunities. To find the right opportunities, LinkedIn needs to understand each member's skill set and their expertise levels accurately. However, estimating members' skill expertise is challenging due to lack of ground-truth. So far, the industry relied on either hand-created small scale data, or large scale social gestures containing a lot of social bias (e.g., endorsements).

In this paper, we develop the Social Skill Validation, a novel framework of collecting validations for members' skill expertise at the scale of billions of member-skill pairs. Unlike social gestures, we collect signals in an anonymous way to ensure objectiveness. We also develop a machine learning model to make smart suggestions to collect validations more efficiently.

With the social skill validation data, we discover the insights on how people evaluate other people in professional social networks. For example, we find that the members with higher seniority do not necessarily get positive evaluations compared to more junior members. We evaluate the value of social skill validation data on predicting who is hired for a job requiring a certain skill, and model using social skill validation outperforms the state-of-the art methods on skill expertise estimation by 10%. Our experiments show that the Social Skill Validation we built provides a novel way to estimate the members' skill expertise accurately at large scale and offers a benchmark to validate social theories on peer evaluation.

References

[1]

{n. d.}. LinkedIn Official Website. https://news.stg.linkedin.com/about-us#statistics.

[2]

Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2012. Discovering Value from Community Activity on Focused Question Answering Sites: A Case Study of Stack Overflow. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '12). ACM, New York, NY, USA, 850--858.

Digital Library

[3]

Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2012. Effects of User Similarity in Social Media. In Proceedings of the Fifth ACM International Conference on Web Search and Data Mining (WSDM '12). ACM, New York, NY, USA, 703--712.

Digital Library

[4]

Krisztian Balog and Maarten de Rijke. 2007. Determining Expert Profiles (With an Application to Expert Finding). In IJCAI 2007, Proceedings of the 20th International Joint Conference on Artificial Intelligence, Hyderabad, India, January 6--12, 2007. 2657--2662. http://ijcai.org/Proceedings/07/Papers/427.pdf

Digital Library

[5]

Moira Burke and Robert Kraut. 2008. Mopping Up: Modeling Wikipedia Promotion Decisions. In Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work (CSCW '08). ACM, New York, NY, USA, 27--36.

Digital Library

[6]

Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007. Learning to Rank: From Pairwise Approach to Listwise Approach. In Proceedings of the 24th International Conference on Machine Learning (ICML '07). ACM, New York, NY, USA, 129--136.

Digital Library

[7]

Olivier Chapelle and Ya Zhang. 2009. A Dynamic Bayesian Network Click Model for Web Search Ranking. In Proceedings of the 18th International Conference on World Wide Web (WWW '09). ACM, New York, NY, USA, 1--10.

Digital Library

[8]

Albert C. Chen and Xin Fu. 2017. Data + Intuition: A Hybrid Approach to Developing Product North Star Metrics. In Proceedings of the 26th International Conference on World Wide Web Companion (WWW '17 Companion). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 617--625.

Digital Library

[9]

Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '16). ACM, New York, NY, USA, 785--794.

Digital Library

[10]

Cristian Danescu-Niculescu-Mizil, Gueorgi Kossinets, Jon Kleinberg, and Lillian Lee. 2009. How Opinions Are Received by Online Communities: A Case Study on Amazon.Com Helpfulness Votes. In Proceedings of the 18th International Conference on World Wide Web (WWW '09). ACM, New York, NY, USA, 141--150.

Digital Library

[11]

Georges E. Dupret and Benjamin Piwowarski. 2008. A User Browsing Model to Predict Search Engine Click Data from Past Observations. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '08). ACM, New York, NY, USA, 331--338.

Digital Library

[12]

Fredric C. Gey. 1994. Inferring Probability of Relevance Using the Method of Logistic Regression. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '94). Springer-Verlag New York, Inc., New York, NY, USA, 222--231. http://dl.acm.org/citation.cfm?id=188490.188560

Digital Library

[13]

Carlos A. Gomez-Uribe and Neil Hunt. 2015. The Netflix Recommender System: Algorithms, Business Value, and Innovation. ACM Trans. Manage. Inf. Syst. 6, 4, Article 13 (Dec. 2015), 19 pages.

Digital Library

[14]

Laura A. Granka, Thorsten Joachims, and Geri Gay. 2004. Eye-tracking Analysis of User Behavior inWWWSearch. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '04). ACM, New York, NY, USA, 478--479.

Digital Library

[15]

V. Ha-Thuc, G. Venkataraman, M. Rodriguez, S. Sinha, S. Sundaram, and L. Guo. 2015. Personalized expertise search at LinkedIn. In 2015 IEEE International Conference on Big Data (Big Data). 1238--1247.

Digital Library

[16]

Will Hill, Larry Stead, Mark Rosenstein, and George Furnas. 1995. Recommending and Evaluating Choices in a Virtual Community of Use. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 194--201.

Digital Library

[17]

Cho-Jui Hsieh, Mitul Tiwari, Deepak Agarwal, Xinyi (Lisa) Huang, and Sam Shah. 2013. Organizational Overlap on Social Networks and Its Applications. In Proceedings of the 22Nd International Conference on World Wide Web (WWW '13). ACM, New York, NY, USA, 571--582.

Digital Library

[18]

Pei Lee, Laks V.S. Lakshmanan, Mitul Tiwari, and Sam Shah. 2014. Modeling Impression Discounting in Large-scale Recommender Systems. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '14). ACM, New York, NY, USA, 1837--1846.

Digital Library

[19]

Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon.Com Recommendations: Item-to-Item Collaborative Filtering. IEEE Internet Computing 7, 1 (Jan. 2003), 76--80.

Digital Library

[20]

Bo Pang and Lillian Lee. 2008. Opinion Mining and Sentiment Analysis. Found. Trends Inf. Retr. 2, 1--2 (Jan. 2008), 1--135.

Digital Library

[21]

Tao Qin, Xu-Dong Zhang, De-Sheng Wang, Tie-Yan Liu, Wei Lai, and Hang Li. 2007. Ranking with Multiple Hyperplanes. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '07). ACM, New York, NY, USA, 279--286.

Digital Library

[22]

Alexander Ratner, Stephen H. Bach, Henry Ehrenberg, Jason Fries, Sen Wu, and Christopher Ré. 2017. Snorkel: Rapid Training Data Creation with Weak Supervision. Proc. VLDB Endow. 11, 3 (Nov. 2017), 269--282.

Digital Library

[23]

Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2000. Analysis of Recommendation Algorithms for e-Commerce. In Proceedings of the 2Nd ACM Conference on Electronic Commerce (EC '00). ACM, New York, NY, USA, 158--167.

Digital Library

[24]

Ming-Feng Tsai, Tie-Yan Liu, Tao Qin, Hsin-Hsi Chen, and Wei-Ying Ma. 2007. FRank: A Ranking Method with Fidelity Loss. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '07). ACM, New York, NY, USA, 383--390.

Digital Library

[25]

Yisong Yue, Thomas Finley, Filip Radlinski, and Thorsten Joachims. 2007. A Support Vector Method for Optimizing Average Precision. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '07). ACM, New York, NY, USA, 271--278.

Digital Library

[26]

Yuchen Zhang, Weizhu Chen, Dong Wang, and Qiang Yang. 2011. User-click Modeling for Understanding and Predicting Search-behavior. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '11). ACM, New York, NY, USA, 1388--1396.

Digital Library

[27]

Guangyou Zhou, Siwei Lai, Kang Liu, and Jun Zhao. 2012. Topic-sensitive Probabilistic Model for Expert Finding in Question Answer Communities. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management (CIKM '12). ACM, New York, NY, USA, 1662--1666.

Digital Library

Cited By

Welsh ARuda A(2024)Exploration of Skillification and Its Use in Awarding Credit for Prior LearningThe Journal of Continuing Higher Education10.1080/07377363.2023.2279808(1-17)Online publication date: 16-Jan-2024
https://doi.org/10.1080/07377363.2023.2279808
Qin CZhu HShen DSun YYao KWang PXiong H(2023)Automatic Skill-Oriented Question Generation and Recommendation for Intelligent Job InterviewsACM Transactions on Information Systems10.1145/360455242:1(1-32)Online publication date: 13-Jun-2023
https://dl.acm.org/doi/10.1145/3604552
Strukova SRuipérez-Valiente JGómez Mármol F(2023)Computational approaches to detect experts in distributed online communities: a case study on RedditCluster Computing10.1007/s10586-023-04076-w27:2(2181-2201)Online publication date: 23-Jun-2023
https://dl.acm.org/doi/10.1007/s10586-023-04076-w
Show More Cited By

Index Terms

Social Skill Validation at LinkedIn
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Classification and regression trees

Recommendations

Cognitive modelling of human social signals
SSPW '10: Proceedings of the 2nd international workshop on Social signal processing

The paper defines as "social signal" a communicative or informative signal that, either directly or indirectly, conveys information about social actions, social interactions, social emotions, social attitudes and social relationships. It proposes a ...
Coporate IT skill needs: a case study of BigCo.

This paper reports on a case study of the IT skills and needs at one site. The data is organized by assessing the members of BigCo's Corporate IT group regarding their present IT skill levels and the perception of skill needs both now and three years in ...
LinkedIn and Facebook in Belgium: The Influences and Biases of Social Network Sites in Recruitment and Selection Procedures

This study investigates whether Belgian recruitment and selection (R&S) professionals use LinkedIn and Facebook during their R&S procedures and to what extent. A total of 398 and 353 respondents, respectively, from various sectors and from organizations ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

July 2019

3305 pages

ISBN:9781450362016

DOI:10.1145/3292500

General Chairs:
Ankur Teredesai
KenSci
,
Vipin Kumar
University of Minnesota
,
Program Chairs:
Ying Li
EV Analysis Corporation
,
Rómer Rosales
LinkedIn
,
Evimaria Terzi
Boston University
,
George Karypis
University of Minnesota

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '19

Sponsor:

KDD '19: The 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 4 - 8, 2019

AK, Anchorage, USA

Acceptance Rates

KDD '19 Paper Acceptance Rate 110 of 1,200 submissions, 9%;

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '24

Sponsor:
sigkdd
sigkdd

The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 25 - 29, 2024

Barcelona , Spain

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
986
Total Downloads

Downloads (Last 12 months)74
Downloads (Last 6 weeks)6

Reflects downloads up to 09 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Welsh ARuda A(2024)Exploration of Skillification and Its Use in Awarding Credit for Prior LearningThe Journal of Continuing Higher Education10.1080/07377363.2023.2279808(1-17)Online publication date: 16-Jan-2024
https://doi.org/10.1080/07377363.2023.2279808
Qin CZhu HShen DSun YYao KWang PXiong H(2023)Automatic Skill-Oriented Question Generation and Recommendation for Intelligent Job InterviewsACM Transactions on Information Systems10.1145/360455242:1(1-32)Online publication date: 13-Jun-2023
https://dl.acm.org/doi/10.1145/3604552
Strukova SRuipérez-Valiente JGómez Mármol F(2023)Computational approaches to detect experts in distributed online communities: a case study on RedditCluster Computing10.1007/s10586-023-04076-w27:2(2181-2201)Online publication date: 23-Jun-2023
https://dl.acm.org/doi/10.1007/s10586-023-04076-w
Strukova SRuipérez-Valiente J(2022)Using Online Digital Data to Infer Valuable Skills for the Modern WorkforceHandbook of Research on New Media, Training, and Skill Development for the Modern Workforce10.4018/978-1-6684-3996-8.ch005(89-109)Online publication date: 13-May-2022
https://doi.org/10.4018/978-1-6684-3996-8.ch005
Urdaneta-Ponte MOleagordia-Ruíz IMéndez-Zorrilla A(2022)Using LinkedIn Endorsements to Reinforce an Ontology and Machine Learning-Based Recommender System to Improve Professional SkillsElectronics10.3390/electronics1108119011:8(1190)Online publication date: 8-Apr-2022
https://doi.org/10.3390/electronics11081190
Guo ZLiu HZhang LZhang QZhu HXiong HZhang ARangwala H(2022)Talent Demand-Supply Joint Prediction with Dynamic Heterogeneous Graph Enhanced Meta-LearningProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3534678.3539139(2957-2967)Online publication date: 14-Aug-2022
https://dl.acm.org/doi/10.1145/3534678.3539139
Di Lauro STursunbayeva AAntonelli GMoschera L(2022)Alignment of employees’ competencies with espoused organizational valuesInternational Studies of Management & Organization10.1080/00208825.2022.214838853:1(1-18)Online publication date: 26-Nov-2022
https://doi.org/10.1080/00208825.2022.2148388
Pena LCurado COliveira M(2022)The contribution of LinkedIn use to career outcome expectationsJournal of Business Research10.1016/j.jbusres.2021.09.047144(788-796)Online publication date: May-2022
https://doi.org/10.1016/j.jbusres.2021.09.047
Yan XMa AYang JZhu LJing HBollinger JHe QDemartini GZuccon GCulpepper JHuang ZTong H(2021)Contextual Skill Proficiency via Multi-task Learning at LinkedInProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3481904(4273-4282)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3481904
Song CLin QLing GZhang ZChen HLiao JChen C(2020)LoCEC: Local Community-based Edge Classification in Large Online Social Networks2020 IEEE 36th International Conference on Data Engineering (ICDE)10.1109/ICDE48307.2020.00150(1689-1700)Online publication date: Apr-2020
https://doi.org/10.1109/ICDE48307.2020.00150
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents