Query-Driven Data Profiling with OCEANProfile
Authors: 
Andreas M. Wahl, Christian Sauerhammer, Peter K. Schwab, 
Sebastian Herbst, 
Richard LenzAuthors Info & Claims
Andreas M. Wahl, Christian Sauerhammer, Peter K. Schwab, Sebastian Herbst, Richard LenzAuthors Info & ClaimsArticle No.: 7, Pages 1 - 3
Abstract
Complex data analysis scenarios often require discovering and combining multiple data sources. Data scientists usually formulate a series of SQL queries building on each other, also called a session, to iteratively derive results. However, due to a lack of familiarity with data sources or the complexity of query results, it can be a hard task to decide on the next query iteration solely based on the results of the last one.
While existing approaches provide mechanisms to assess the results of a specific query, support for analyzing results in the context of the respective session remains mostly absent. Such approaches do also not seamlessly integrate with established tools and workflows.
To overcome these problems, we introduce OCEANProfile, a framework for session-based profiling of query results. Query results are intercepted at driver level and streamed into our framework for automated data profiling. Result profiles can be compared with those of previous queries and visualized in a companion app compatible with existing analysis tools. Visualizations are automatically ranked according to their usefulness in the context of the respective session.
References
[1]
Abedjan et al. 2015. Profiling Relational Data: A Survey. The VLDB Journal 24, 4 (Aug. 2015).
[2]
Chapman et al. 2016. The Challenge of "Quick and Dirty" Information Quality. JDIQ 7, 1-2 (Feb. 2016).
[3]
Papenbrock et al. 2015. Data Profiling with Metanome. PVLDB 8, 12 (Aug. 2015).
[4]
Qin et al. 2018. DeepEye: An automatic big data visualization framework. Big Data Mining and Analytics 1, 1 (March 2018).
[5]
Vartak et al. 2015. SeeDB: efficient data-driven visualization recommendations to support visual analytics. PVLDB 8, 13 (2015).
[6]
Wahl et al. 2017. Query-Driven Knowledge-Sharing for Data Integration and Collaborative Data Science. In ADBIS'17.
[7]
Haas. 2017. Leveraging Data and People to Accelerate Data Science. In ICDE'17.
Index Terms
- Query-Driven Data Profiling with OCEANProfile
Recommendations
Scalable Query Profiling Employing Purging and Elimination Technique
ICIBE '18: Proceedings of the 4th International Conference on Industrial and Business EngineeringReusing Queries contributes in speeding up the performance of database in responding to future queries as it can reduce the number of database queries to be processed and sent back to the user. Profiling a query in a machine who requested a query in ...
View-Based Query Answering and Query Containment over Semistructured Data
DBPL '01: Revised Papers from the 8th International Workshop on Database Programming LanguagesThe basic querying mechanism over semistructured data, namely regular path queries, asks for all pairs of objects that are connected by a path conforming to a regular expression. We consider conjunctive two-way regular path queries (C2RPQc's), which ...
Query containment for data integration systems
Special issue on PODS 2000The problem of query containment is fundamental to many aspects of database systems, including query optimization, determining independence of queries from updates, and rewriting queries using views. In the data-integration framework, however, the ...
Comments
Information & Contributors
Information
Published In
August 2018
59 pages
ISBN:9781450366076
DOI:10.1145/3242153
Copyright © 2018 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
In-Cooperation
- NSF: National Science Foundation
- Google Inc.
- Microsoft: Microsoft
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 August 2018
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
BIRTE '18
BIRTE '18: International Workshop on Real-Time Business Intelligence and Analytics
August 27, 2018
Rio de Janeiro, Brazil
Acceptance Rates
Overall Acceptance Rate 12 of 21 submissions, 57%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 70Total Downloads
- Downloads (Last 12 months)2
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in