The document discusses the use of Elastic Search's percolator feature. The percolator allows storing queries in an index and then indexing documents to retrieve matching queries. This is opposite of traditional search where documents are indexed and queries retrieve them. The percolator works in real-time so queries can immediately be used. Examples are provided of adding percolator queries and using them to match documents. Additional use cases discussed include alerting, notifications, and monitoring systems.
1 of 19
Download to read offline
More Related Content
Use Cases for Elastic Search Percolator
1. Use Cases for Elastic Search Percolator
Search Meetup 2016
2. 2
Percolator
Search Meetup 2015
Traditionally you design documents based on your data, store them into an index, and then define queries
to retrieve these documents.
TheTheTheThe percolator works in the opposite direction.percolator works in the opposite direction.percolator works in the opposite direction.percolator works in the opposite direction.
First you store queries into an index and then you define documents in order to retrieve these queries.
The percolator and most of its features work in realtime - once a percolate query is indexed it can
immediately be used in the percolate API.
6. 6
Implementation details
ElasticElasticElasticElastic SearchSearchSearchSearch as real time Data Container with all events
PercolatorPercolatorPercolatorPercolator as real time Rule Engine
RedisRedisRedisRedis as queuing system
LogstashLogstashLogstashLogstash for event pushing
DropwizardDropwizardDropwizardDropwizard for RESTfull standalone application
HTML5 and AngularJSHTML5 and AngularJSHTML5 and AngularJSHTML5 and AngularJS for responsive GUI
Search Meetup 2015
Tools / frameworks
14. 14
Percolator details
stored per index in the .percolator type
active percolator queries are kept in memory
at percolate time, the document from the request gets parsed into a Lucene document and
is stored in a in-memory Lucene index
percolator queries are executed on single document in-memory index
multi percolate and percolation of existing documents is possible
Search Meetup 2015
Under the hood
17. 17
Alerting Use Case
easy with JSON structured logs
simplest use case: percolate on FATAL and ERROR log levels
alerts can be triggered for displaying on dashboards
also easy for enhanced use cases such as Monitoring of invariants: create alerts
whenever system goes outside of expected normal ranges
shift monitoring from reactive to proactive – be able to look for exceptional behavior
before it exhibits itself as a production problem
Search Meetup 2015
Using with ELK
18. 18
Other use cases
Search Meetup 2015
Alerting based notifications
Having corresponding data in elastic search you can notify customer about
• new product release
• price change of a product (is below threshold)
• weather changes (snow, rain etc)
• stock changes
• what do you use?