column

An overview of the deco system: data model and query language; query processing and optimization

Authors:

Aditya Parameswaran,

Hector Garcia-Molina,

Neoklis Polyzotis, and

Jennifer WidomAuthors Info & Claims

ACM SIGMOD Record, Volume 41, Issue 4

Pages 22 - 27

https://doi.org/10.1145/2430456.2430462

Published: 17 January 2013 Publication History

Abstract

Deco is a comprehensive system for answering declarative queries posed over stored relational data together with data obtained on-demand from the crowd. In this overview paper, we describe Deco's data model, query language, and system prototype, summarizing material from earlier papers. Deco's data model was designed to be general, flexible, and principled. Deco's query language extends SQL with simple constructs necessary for crowdsourcing, and has a precise semantics for arbitrary queries. Deco's query execution engine and cost-based query optimizer incorporate many novel techniques to address the limitations of traditional query processing techniques in the crowdsourcing setting. Query processing is guided by the objective of minimizing monetary cost and reducing latency.

References

[1]

Mechanical Turk. http://mturk.com.

[2]

S. Ahmad, A. Battle, Z. Malkani, and S. D. Kamvar. The jabberwocky programming environment for structured social computing. In UIST, 2011.

Digital Library

[3]

J. A. Blakeley, P. Larson, and F. W. Tompa. Efficiently updating materialized views. In SIGMOD, 1986.

Digital Library

[4]

A. Bozzon, M. Brambilla, and S. Ceri. Answering search queries with crowdsearcher. In WWW, 2012.

Digital Library

[5]

D. Deutch, O. Greenshpan, B. Kostenko, and T. Milo. Declarative platform for data sourcing games. In WWW, 2012.

Digital Library

[6]

A. Doan, R. Ramakrishnan, and A. Halevy. Crowdsourcing systems on the world-wide web. Communications of the ACM, 54(4):86--96, 2011.

Digital Library

[7]

M. J. Franklin, D. Kossmann, T. Kraska, S. Ramesh, and R. Xin. CrowdDB: Answering queries with crowdsourcing. In SIGMOD, 2011.

Digital Library

[8]

R. Goldman and J. Widom. WSQ/DSQ: A practical approach for combined querying of databases and the web. In SIGMOD, 2000.

Digital Library

[9]

S. R. Jeffery, L. Sun, M. DeLand, N. Pendar, R. Barber, and A. Galdi. Arnold: Declarative crowd-machine data integration. In CIDR, 2013.

[10]

A. Marcus, E. Wu, D. Karger, S. Madden, and R. Miller. Human-powered sorts and joins. In VLDB, 2012.

Digital Library

[11]

A. Marcus, E. Wu, S. Madden, and R. Miller. Crowdsourced databases: Query processing with people. In CIDR, 2011.

[12]

A. Parameswaran, H. Park, H. Garcia-Molina, N. Polyzotis, and J. Widom. Deco: Declarative crowdsourcing. In CIKM, 2012.

Digital Library

[13]

H. Park, R. Pang, A. Parameswaran, H. Garcia-Molina, N. Polyzotis, and J. Widom. Deco: A system for declarative crowdsourcing. In VLDB, 2012. Demonstration.

Digital Library

[14]

H. Park, A. Parameswaran, and J. Widom. Query processing over crowdsourced data, http://ilpubs.stanford.edu:8090/1052/. Technical report, Stanford InfoLab, 2012.

[15]

H. Park and J. Widom. Query optimization over crowdsourced data, http://ilpubs.stanford.edu:8090/1063/. Technical report, Stanford InfoLab, 2012.

Cited By

Rahman PNandi AHebert C(2020)Amplifying Domain Expertise in Clinical Data PipelinesJMIR Medical Informatics10.2196/196128:11(e19612)Online publication date: 5-Nov-2020
https://doi.org/10.2196/19612
Lukyanenko RParsons J(2019)Beyond Micro-TasksCrowdsourcing10.4018/978-1-5225-8362-2.ch076(1510-1535)Online publication date: 2019
https://doi.org/10.4018/978-1-5225-8362-2.ch076
Lukyanenko RParsons J(2019)Beyond Micro-TasksSocial Entrepreneurship10.4018/978-1-5225-8182-6.ch072(1403-1428)Online publication date: 2019
https://doi.org/10.4018/978-1-5225-8182-6.ch072
Show More Cited By

Index Terms

An overview of the deco system: data model and query language; query processing and optimization
1. Information systems
  1. Data management systems
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Database query processing and optimization (theory)

Recommendations

Deco: declarative crowdsourcing
CIKM '12: Proceedings of the 21st ACM international conference on Information and knowledge management

Crowdsourcing enables programmers to incorporate "human computation" as a building block in algorithms that cannot be fully automated, such as text analysis and image recognition. Similarly, humans can be used as a building block in data-intensive ...
Read More
Deco: a system for declarative crowdsourcing

Deco is a system that enables declarative crowdsourcing: answering SQL queries posed over data gathered from the crowd as well as existing relational data. Deco implements a novel push-pull hybrid execution model in order to support a flexible data ...
Read More
Demonstration of VerdictDB, the Platform-Independent AQP System
SIGMOD '18: Proceedings of the 2018 International Conference on Management of Data

We demonstrate VerdictDB, the first platform-independent approximate query processing (AQP) system. Unlike existing AQP systems that are tightly-integrated into a specific database, VerdictDB operates at the driver-level, acting as a middleware between ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM SIGMOD Record

ACM SIGMOD Record Volume 41, Issue 4

December 2012

62 pages

ISSN:0163-5808

DOI:10.1145/2430456

Issue’s Table of Contents

Copyright © 2013 Authors.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 January 2013

Published in SIGMOD Volume 41, Issue 4

Check for updates

Qualifiers

Column

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
339
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Rahman PNandi AHebert C(2020)Amplifying Domain Expertise in Clinical Data PipelinesJMIR Medical Informatics10.2196/196128:11(e19612)Online publication date: 5-Nov-2020
https://doi.org/10.2196/19612
Lukyanenko RParsons J(2019)Beyond Micro-TasksCrowdsourcing10.4018/978-1-5225-8362-2.ch076(1510-1535)Online publication date: 2019
https://doi.org/10.4018/978-1-5225-8362-2.ch076
Lukyanenko RParsons J(2019)Beyond Micro-TasksSocial Entrepreneurship10.4018/978-1-5225-8182-6.ch072(1403-1428)Online publication date: 2019
https://doi.org/10.4018/978-1-5225-8182-6.ch072
Rahman PHebert CNandi A(2019)ICARUSProceedings of the VLDB Endowment10.14778/3275366.328497011:13(2263-2276)Online publication date: 17-Jan-2019
https://dl.acm.org/doi/10.14778/3275366.3284970
Rahman PHebert CNandi A(2018)ICARUSProceedings of the VLDB Endowment10.5555/3275366.328497011:13(2263-2276)Online publication date: 1-Sep-2018
https://dl.acm.org/doi/10.5555/3275366.3284970
Parsons JLukyanenko R(2018)Beyond Micro-TasksJournal of Database Management10.4018/JDM.201801010129:1(1-22)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.4018/JDM.2018010101
Jain ASarma AParameswaran AWidom J(2017)Understanding workers, developing effective tasks, and enhancing marketplace dynamicsProceedings of the VLDB Endowment10.14778/3067421.306743110:7(829-840)Online publication date: 1-Mar-2017
https://dl.acm.org/doi/10.14778/3067421.3067431
Weng XLi GHu HFeng JLim EWinslett MSanderson MFu ASun JCulpepper SLo EHo JDonato DAgrawal RZheng YCastillo CSun ATseng VLi C(2017)Crowdsourced Selection on Multi-Attribute DataProceedings of the 2017 ACM on Conference on Information and Knowledge Management10.1145/3132847.3132891(307-316)Online publication date: 6-Nov-2017
https://dl.acm.org/doi/10.1145/3132847.3132891
Difallah DDemartini GCudré-Mauroux PBourdeau JHendler JNkambou RHorrocks IZhao B(2016)Scheduling Human Intelligence Tasks in Multi-Tenant Crowd-Powered SystemsProceedings of the 25th International Conference on World Wide Web10.1145/2872427.2883030(855-865)Online publication date: 11-Apr-2016
https://dl.acm.org/doi/10.1145/2872427.2883030
Chittilappilly AChen LAmer-Yahia S(2016)A Survey of General-Purpose Crowdsourcing TechniquesIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2016.255580528:9(2246-2266)Online publication date: 1-Sep-2016
https://dl.acm.org/doi/10.1109/TKDE.2016.2555805
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents