short-paper

Continuation Methods and Curriculum Learning for Learning to Rank

Authors:

Nicola Ferro,

Claudio Lucchese,

Maria Maistro,

Raffaele PeregoAuthors Info & Claims

CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge Management

Pages 1523 - 1526

https://doi.org/10.1145/3269206.3269239

Published: 17 October 2018 Publication History

Get Access

Abstract

In this paper we explore the use of Continuation Methods and Curriculum Learning techniques in the area of Learning to Rank. The basic idea is to design the training process as a learning path across increasingly complex training instances and objective functions. We propose to instantiate continuation methods in Learning to Rank by changing the IR measure to optimize during training, and we present two different curriculum learning strategies to identify easy training examples. Experimental results show that simple continuation methods are more promising than curriculum learning ones since they allow for slightly improving the performance of state-of-the-art λ-MART models and provide a faster convergence speed.

References

[1]

E. L. Allgower, and K. Georg. Numerical Continuation Methods. An Introduction. Springer-Verlag, 1980.

Digital Library

Google Scholar

[2]

Y. Bengio, J. Louradour, R. Collobert, and J. Weston, Curriculum Learning. In ICML, pages 41--48, ACM, 2009.

Digital Library

Google Scholar

[3]

C. J. C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender. Learning to Rank Using Gradient Descent. In ICML, pages 89--96, ACM, 2005.

Digital Library

Google Scholar

[4]

O. Chapelle, M. Chi, and A. Zien. A Continuation Method for Semi-supervised SVMs. In ICML, pages 185--192, ACM, 2006.

Digital Library

Google Scholar

[5]

B. Chen, and N. Xiu. A Global Linear and Local Quadratic Noninterior Continuation Method for Nonlinear Complementarity Problems Based on Chen-Mangasarian Smoothing Functions. SIAM Journal on Optimization, 9, 3, pages 605--623, SIAM, 1999.

Digital Library

Google Scholar

[6]

X. Chen, and A. Gupta. Webly Supervised Learning of Convolutional Networks. In ICCV, pages 1431--1439, IEEE, 2015.

Digital Library

Google Scholar

[7]

T. F. Coleman, and Z. Wu. Parallel Continuation-based Global Optimization for Molecular Conformation and Protein Folding. Journal of Global Optimization, 8, 1, pages 49--65, Springer, 1996.

Crossref

Google Scholar

[8]

T. Liu. Learning to Rank for Information Retrieval. Foundations and Trends in Information Retrieval, 3, pages 225--331, 2009.

Digital Library

Google Scholar

[9]

J. J. Moré, and Z. Wu. Global Continuation for Distance Geometry Problems. SIAM Journal on Optimization, 7, 3, pages 814--836, SIAM, 1997.

Digital Library

Google Scholar

[10]

R. Nagamune. A Robust Solver Using a Continuation Method for Nevanlinna-Pick Interpolation with Degree Constraint. IEEE Transactions on Automatic Control, 48, 1, pages 8113--117, IEEE, 2003.

Google Scholar

[11]

T. Qin, and T. Liu. Introducing LETOR 4.0 Datasets. In CoRR, 2013.

Google Scholar

[12]

M. Qu, J. Tang, and J. Han. Curriculum Learning for Heterogeneous Star Network Embedding via Deep Reinforcement Learning. In WSDM, pages 468--476, ACM, 2018.

Digital Library

Google Scholar

[13]

E. Rabani, D. R. Reichman, G. Krilov, and B. J. Berne. The Calculation of Transport Properties in Quantum Liquids Using the Maximum Entropy Numerical Analytic Continuation Method: Application to Liquid Para-hydrogen. Proceedings of the National Academy of Sciences, 99, 3, pages 1129--1133, National Academy of Sciences, 2002.

Crossref

Google Scholar

[14]

M. Taylor, J. Guiver, S. Robertson, and T. Minka Softrank: optimizing non-smooth rank metrics. In WSDM, pages 77--86, ACM, 2008.

Digital Library

Google Scholar

Cited By

View all

Zhu YNie JSu YChen HZhang XDou ZAl Hasan MXiong L(2022)From Easy to HardProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557328(2784-2794)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3511808.3557328
Zhang LLiu J(2022)Curriculum Contrastive Learning for COVID-19 FAQ Retrieval2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM55620.2022.9995534(3228-3234)Online publication date: 6-Dec-2022
https://doi.org/10.1109/BIBM55620.2022.9995534
Gong YLiu CYuan JYang FCai XWan GChen JNiu RWang HDemartini GZuccon GCulpepper JHuang ZTong H(2021)Density-Based Dynamic Curriculum Learning for Intent DetectionProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482082(3034-3037)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482082
Show More Cited By

Index Terms

Continuation Methods and Curriculum Learning for Learning to Rank
1. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results
      1. Retrieval effectiveness
    2. Retrieval models and ranking
      1. Learning to rank

Recommendations

Curriculum Meta Learning: Learning to Learn from Easy to Hard
EITCE '21: Proceedings of the 2021 5th International Conference on Electronic Information Technology and Computer Engineering

Meta-learning is a machine learning paradigm that extracts crosstask knowledge by learning a large number of subtasks, to fast adapt to new tasks. Many meta-learning methods are widely applied in few-shot classification. These methods adopt an episodic ...
Learning to rank with groups
CIKM '10: Proceedings of the 19th ACM international conference on Information and knowledge management

An essential issue in document retrieval is ranking, and the documents are ranked by their expected relevance to a given query. Multiple labels are used to represent different level of relevance for documents to a given query, and the corresponding ...
Compression-Based Selective Sampling for Learning to Rank
CIKM '16: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

Learning to rank (L2R) algorithms use a labeled training set to generate a ranking model that can be later used to rank new query results. These training sets are very costly and laborious to produce, requiring human annotators to assess the relevance or ...

Comments

Information & Contributors

Information

Published In

CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge Management

October 2018

2362 pages

ISBN:9781450360142

DOI:10.1145/3269206

General Chair:
Alfredo Cuzzocrea
University of Trieste, Italy
,
Program Chairs:
James Allan
University of Massachusetts, USA
,
Norman Paton
University of Manchester, United Kingdom
,
Divesh Srivastava
AT&T Labs Research, USA
,
Rakesh Agrawal
Data Insights Lab, USA
,
Andrei Broder
Google Research, USA
,
Mohammed Zaki
Rensselaer Polytechnic Institute, USA
,
Selcuk Candan
Arizona State University, USA
,
Alexandros Labrinidis
University of Pittsburgh, USA
,
Assaf Schuster
Technion, Israel
,
Haixun Wang
Google Research, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

CIKM '18

Sponsor:

CIKM '18: The 27th ACM International Conference on Information and Knowledge Management

October 22 - 26, 2018

Torino, Italy

Acceptance Rates

CIKM '18 Paper Acceptance Rate 147 of 826 submissions, 18%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
270
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zhu YNie JSu YChen HZhang XDou ZAl Hasan MXiong L(2022)From Easy to HardProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557328(2784-2794)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3511808.3557328
Zhang LLiu J(2022)Curriculum Contrastive Learning for COVID-19 FAQ Retrieval2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM55620.2022.9995534(3228-3234)Online publication date: 6-Dec-2022
https://doi.org/10.1109/BIBM55620.2022.9995534
Gong YLiu CYuan JYang FCai XWan GChen JNiu RWang HDemartini GZuccon GCulpepper JHuang ZTong H(2021)Density-Based Dynamic Curriculum Learning for Intent DetectionProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482082(3034-3037)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482082
Wang XChen YZhu W(2021)A Survey on Curriculum LearningIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2021.3069908(1-1)Online publication date: 2021
https://doi.org/10.1109/TPAMI.2021.3069908
Gupta KMukherjee DNajjaran H(2021)Extending the Capabilities of Reinforcement Learning Through Curriculum: A Review of Methods and ApplicationsSN Computer Science10.1007/s42979-021-00934-93:1Online publication date: 29-Oct-2021
https://doi.org/10.1007/s42979-021-00934-9
Penha GHauff C(2020)Curriculum Learning Strategies for IRAdvances in Information Retrieval10.1007/978-3-030-45439-5_46(699-713)Online publication date: 8-Apr-2020
https://doi.org/10.1007/978-3-030-45439-5_46
Ferro NLucchese CMaistro MPerego R(2019)Boosting learning to rank with user dynamics and continuation methodsInformation Retrieval Journal10.1007/s10791-019-09366-9Online publication date: 5-Nov-2019
https://doi.org/10.1007/s10791-019-09366-9

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Curriculum Meta Learning: Learning to Learn from Easy to Hard

Learning to rank with groups

Compression-Based Selective Sampling for Learning to Rank