Deploying a Steered Query Optimizer in Production at Microsoft
Authors:
Wangda Zhang,
Matteo Interlandi,
Paul Mineiro,
Shi Qiao,
Nasim Ghazanfari Karlen Lie,
Marc Friedman,
Rafah Hosn,
Hiren Patel,
Alekh Jindal
Abstract:
Modern analytical workloads are highly heterogeneous and massively complex, making generic query optimizers untenable for many customers and scenarios. As a result, it is important to specialize these optimizers to instances of the workloads. In this paper, we continue a recent line of work in steering a query optimizer towards better plans for a given workload, and make major strides in pushing p…
▽ More
Modern analytical workloads are highly heterogeneous and massively complex, making generic query optimizers untenable for many customers and scenarios. As a result, it is important to specialize these optimizers to instances of the workloads. In this paper, we continue a recent line of work in steering a query optimizer towards better plans for a given workload, and make major strides in pushing previous research ideas to production deployment. Along the way we solve several operational challenges including, making steering actions more manageable, keeping the costs of steering within budget, and avoiding unexpected performance regressions in production. Our resulting system, QQ-advisor, essentially externalizes the query planner to a massive offline pipeline for better exploration and specialization. We discuss various aspects of our design and show detailed results over production SCOPE workloads at Microsoft, where the system is currently enabled by default.
△ Less
Submitted 24 October, 2022;
originally announced October 2022.
The Open Porous Media Flow Reservoir Simulator
Authors:
Atgeirr Flø Rasmussen,
Tor Harald Sandve,
Kai Bao,
Andreas Lauser,
Joakim Hove,
Bård Skaflestad,
Robert Klöfkorn,
Markus Blatt,
Alf Birger Rustad,
Ove Sævareid,
Knut-Andreas Lie,
Andreas Thune
Abstract:
The Open Porous Media (OPM) initiative is a community effort that encourages open innovation and reproducible research for simulation of porous media processes. OPM coordinates collaborative software development, maintains and distributes open-source software and open data sets, and seeks to ensure that these are available under a free license in a long-term perspective.
In this paper, we presen…
▽ More
The Open Porous Media (OPM) initiative is a community effort that encourages open innovation and reproducible research for simulation of porous media processes. OPM coordinates collaborative software development, maintains and distributes open-source software and open data sets, and seeks to ensure that these are available under a free license in a long-term perspective.
In this paper, we present OPM Flow, which is a reservoir simulator developed for industrial use, as well as some of the individual components used to make OPM Flow. The descriptions apply to the 2019.10 release of OPM.
△ Less
Submitted 4 October, 2019;
originally announced October 2019.
