Automated Migration of Hierarchical Data to Relational Tables using Programming-by-Example
Authors:
Navid Yaghmazadeh,
Xinyu Wang,
Isil Dillig
Abstract:
While many applications export data in hierarchical formats like XML and JSON, it is often necessary to convert such hierarchical documents to a relational representation. This paper presents a novel programming-by-example approach, and its implementation in a tool called Mitra, for automatically migrating tree-structured documents to relational tables. We have evaluated the proposed technique usi…
▽ More
While many applications export data in hierarchical formats like XML and JSON, it is often necessary to convert such hierarchical documents to a relational representation. This paper presents a novel programming-by-example approach, and its implementation in a tool called Mitra, for automatically migrating tree-structured documents to relational tables. We have evaluated the proposed technique using two sets of experiments. In the first experiment, we used Mitra to automate 98 data transformation tasks collected from StackOverflow. Our method can generate the desired program for 94% of these benchmarks with an average synthesis time of 3.8 seconds. In the second experiment, we used Mitra to generate programs that can convert real-world XML and JSON datasets to full-fledged relational databases. Our evaluation shows that Mitra can automate the desired transformation for all datasets.
△ Less
Submitted 10 November, 2017;
originally announced November 2017.
Type- and Content-Driven Synthesis of SQL Queries from Natural Language
Authors:
Navid Yaghmazadeh,
Yuepeng Wang,
Isil Dillig,
Thomas Dillig
Abstract:
This paper presents a new technique for automatically synthesizing SQL queries from natural language. Our technique is fully automated, works for any database without requiring additional customization, and does not require users to know the underlying database schema. Our method achieves these goals by combining natural language processing, program synthesis, and automated program repair. Given t…
▽ More
This paper presents a new technique for automatically synthesizing SQL queries from natural language. Our technique is fully automated, works for any database without requiring additional customization, and does not require users to know the underlying database schema. Our method achieves these goals by combining natural language processing, program synthesis, and automated program repair. Given the user's English description, our technique first uses semantic parsing to generate a query sketch, which is subsequently completed using type-directed program synthesis and assigned a confidence score using database contents. However, since the user's description may not accurately reflect the actual database schema, our approach also performs fault localization and repairs the erroneous part of the sketch. This synthesize-repair loop is repeated until the algorithm infers a query with a sufficiently high confidence score. We have implemented the proposed technique in a tool called Sqlizer and evaluate it on three different databases. Our experiments show that the desired query is ranked within the top 5 candidates in close to 90% of the cases.
△ Less
Submitted 3 February, 2017;
originally announced February 2017.
