Article

A genetic algorithm for dynamic modelling and prediction of activity in document streams

Authors:

Lourdes Araujo,

Juan Julian MereloAuthors Info & Claims

GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation

Pages 1896 - 1903

https://doi.org/10.1145/1276958.1277340

Published: 07 July 2007 Publication History

Get Access

Abstract

This paper presents an evolutionary algorithm for modeling the arrival dates of document streams, which is any time-stamped collection of documents, such as newscasts, e-mails, scientific journals archives and weblog postings. The goal is to find a frequency curve that fits the data circumventing the unavoidable noise. Classical dynamic programming algorithms are limited by memory and efficiency requirements, which can be a problem when dealing with long streams. This suggests to explore alternative search methods which although do not guarantee optimality, are far more efficient. Experiments have shown that the designed evolutionary algorithm is able to reach high quality solutions in a short time. We have also explored different approaches to infer whether new arrivals increase or decrease interest in the topic the document stream is about. In particular, we present a variant of the evolutionary algorithm, which is able to very quickly fit a stream extended with new data, by taking advantage of the fit obtained for the original substream. These mechanisms can be used for real time detection of changes in the trend of interest in a topic, an important application of this kind of models.

References

[1]

J. Allan. Topic Detection and Tracking: Event-Based Information Organization. Kluwer Academic Publishers, 2002.

Digital Library

Google Scholar

[2]

J. Allan, J. Carbonell, G. Doddington, J. Yamron, and Y. Yang. Topic detection and tracking pilot study, 1998.

Google Scholar

[3]

E. Bingham, A. Kabán, and M. Girolami. Topic identification in dynamical text by complexity pursuit. Neural Process. Lett., 17(1):69--83, 2003.

Digital Library

Google Scholar

[4]

M. Charikar, K. Chen, and M. Farach-Colton. Finding frequent items in data streams. In M. Charikar, K. Chen, and M. Farach-Colton. Finding frequent items in data streams. In Proceedings of the 29th International Colloquium on Automata, Languages, and Programming, 2002., 2002.

Digital Library

Google Scholar

[5]

A. I. Elwalid and D. Mitra. Effective bandwidth of general markovian traffic sources and admission control of high speed networks. IEEE/ACM Trans. Netw., 1(3):329--343, 1993.

Digital Library

Google Scholar

[6]

G. D. Forney. The Viterbi algorithm. Proceedings of The IEEE, 61(3):268--278, 1973.

Crossref

Google Scholar

[7]

M. Girolami and A. Kaban. Simplicial mixtures of Markov chains: Distributed modelling of dynamic user profiles. In S. Thrun, L. Saul, and B. Schölkopf, editors, Advances in Neural Information Processing Systems 16. MIT Press, Cambridge, MA, 2004.

Google Scholar

[8]

J. Kleinberg. Bursty and hierarchical structure in streams. In Proc. 8th ACM SIGKDD Intl. Conf. on Knowledge Discovery and Data Mining, pages 91--101. ACM, 2002.

Digital Library

Google Scholar

[9]

J. Kleinberg. Temporal dynamics of on-line information streams. In M. Garofalakis, J. Gehrke, and R. Rastogi, editors, Data Stream Management: Processing High-Speed Data Streams (to appear). Springer, 2005.

Google Scholar

[10]

R. Papka. On-line New Event Detection, Clustering and Tracking. PhD thesis, Department of Computer Science, University of Massachusetts, 1999.

Digital Library

Google Scholar

[11]

L. R. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. In Readings in speech recognition, pages 267--296. Morgan Kaufmann Publishers Inc., 1990.

Crossref

Google Scholar

Cited By

View all

Pal KSaha CDas SCoello Coello C(2013)Dynamic Constrained Optimization with offspring repair based Gravitational Search Algorithm2013 IEEE Congress on Evolutionary Computation10.1109/CEC.2013.6557858(2414-2421)Online publication date: Jun-2013
https://doi.org/10.1109/CEC.2013.6557858
Nguyen TYao X(2013)Evolutionary Optimization on Continuous Dynamic Constrained Problems - An AnalysisEvolutionary Computation for Dynamic Optimization Problems10.1007/978-3-642-38416-5_8(193-217)Online publication date: 2013
https://doi.org/10.1007/978-3-642-38416-5_8
Pal KSaha CDas S(2013)Differential Evolution and Offspring Repair Method Based Dynamic Constrained Optimization4th International Conference on Swarm, Evolutionary, and Memetic Computing - Volume 829710.1007/978-3-319-03753-0_27(298-309)Online publication date: 19-Dec-2013
https://dl.acm.org/doi/10.1007/978-3-319-03753-0_27
Show More Cited By

Index Terms

A genetic algorithm for dynamic modelling and prediction of activity in document streams
1. Computing methodologies
  1. Artificial intelligence
    1. Search methodologies
      1. Heuristic function construction

Recommendations

An evolutionary algorithm for clustering data streams with a variable number of clusters

An evolutionary algorithm for clustering data stream is proposed.Our algorithm allows estimating k automatically from the data in an online fashion.It monitors eventual degradation in the quality of the induced clusters.Results show our algorithm ...
Clustering-based hierarchical genetic algorithm for complex fitness landscapes

We propose the use of a hierarchical genetic algorithm (GA) for optimisation in complex landscapes. While the slave GA tries to find the local optima in the restricted fitness landscape of low complexity, the master GA tries to identify interesting ...
Fuzzy genetic sharing for dynamic optimization

Recently, genetic algorithms (GAs) have been applied to multi-modal dynamic optimization (MDO). In this kind of optimization, an algorithm is required not only to find the multiple optimal solutions but also to locate a dynamically changing optimum. Our ...

Comments

Information & Contributors

Information

Published In

GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation

July 2007

2313 pages

ISBN:9781595936974

DOI:10.1145/1276958

General Chair:
Hod Lipson
Cornell University, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

GECCO07

Sponsor:

GECCO07: Genetic and Evolutionary Computation Conference

July 7 - 11, 2007

London, England

Acceptance Rates

GECCO '07 Paper Acceptance Rate 266 of 577 submissions, 46%;

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
230
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Pal KSaha CDas SCoello Coello C(2013)Dynamic Constrained Optimization with offspring repair based Gravitational Search Algorithm2013 IEEE Congress on Evolutionary Computation10.1109/CEC.2013.6557858(2414-2421)Online publication date: Jun-2013
https://doi.org/10.1109/CEC.2013.6557858
Nguyen TYao X(2013)Evolutionary Optimization on Continuous Dynamic Constrained Problems - An AnalysisEvolutionary Computation for Dynamic Optimization Problems10.1007/978-3-642-38416-5_8(193-217)Online publication date: 2013
https://doi.org/10.1007/978-3-642-38416-5_8
Pal KSaha CDas S(2013)Differential Evolution and Offspring Repair Method Based Dynamic Constrained Optimization4th International Conference on Swarm, Evolutionary, and Memetic Computing - Volume 829710.1007/978-3-319-03753-0_27(298-309)Online publication date: 19-Dec-2013
https://dl.acm.org/doi/10.1007/978-3-319-03753-0_27
Nguyen TYao X(2012)Continuous Dynamic Constrained Optimization—The ChallengesIEEE Transactions on Evolutionary Computation10.1109/TEVC.2011.218053316:6(769-786)Online publication date: 1-Dec-2012
https://dl.acm.org/doi/10.1109/TEVC.2011.2180533

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

An evolutionary algorithm for clustering data streams with a variable number of clusters

Clustering-based hierarchical genetic algorithm for complex fitness landscapes

Fuzzy genetic sharing for dynamic optimization

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

An evolutionary algorithm for clustering data streams with a variable number of clusters

Clustering-based hierarchical genetic algorithm for complex fitness landscapes

Fuzzy genetic sharing for dynamic optimization

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations