Massachusetts Institute of Technology eBooks, Aug 1, 1988
... 2.2. Atomic Actions 17 Argus implements atomicity through the objects shared by actions; thes... more ... 2.2. Atomic Actions 17 Argus implements atomicity through the objects shared by actions; these objects are called atomic objects. Atomic objects, like ordinary objects, provide operations that can be used to access and manipulate them. In addition, however, their operations ...
Modern cluster management systems like Kubernetes and Openstack grapple with hard combinatorial o... more Modern cluster management systems like Kubernetes and Openstack grapple with hard combinatorial optimization problems: load balancing, placement, scheduling, and configuration. Currently, developers tackle these problems by designing custom application-specific algorithms---an approach that is proving unsustainable, as ad-hoc solutions both perform poorly and introduce overwhelming complexity to the system, making it challenging to add important new features. We propose a radically different architecture, where programmers drive cluster management tasks declaratively, using SQL queries over cluster state stored in a relational database. These queries capture in a natural way both constraints on the cluster configuration as well as optimization objectives. When a cluster reconfiguration is required at runtime, our tool, called Weave, synthesizes an encoding of these queries into an optimization model, which it solves using an off-the-shelf solver. We demonstrate Weave's efficacy ...
Cluster managers like Kubernetes and OpenStack are notoriously hard to develop, given that they r... more Cluster managers like Kubernetes and OpenStack are notoriously hard to develop, given that they routinely grapple with hard combinatorial optimization problems like load balancing, placement, scheduling, and configuration. Today, cluster manager developers tackle these problems by developing system-specific best effort heuristics, which achieve scalability by significantly sacrificing the cluster manager’s decision quality, feature set, and extensibility over time. This is proving untenable, as solutions for cluster management problems are routinely developed from scratch in the industry to solve largely similar problems across different settings. We propose DCM, a radically different architecture where developers specify the cluster manager’s behavior declaratively, using SQL queries over cluster state stored in a relational database. From the SQL specification, the DCM compiler synthesizes a program that, at runtime, can be invoked to compute policy-compliant cluster management de...
Research can rarely be performed on large-scale, distributed systems at the level of thousands of... more Research can rarely be performed on large-scale, distributed systems at the level of thousands of workstations. In this paper, we describe the motivating constraints, design principles, and architecture for an extensible, distributed system operating in such an environment. The constraints include continuous operation, dynamic system evolution, and integration with extant systems. The Information Bus , our solution, is a novel synthesis of four design principles: core communication protocols have minimal semantics, objects are self-describing, types can be dynamically defined, and communication is anonymous. The current implementation provides both flexibility and high performance, and has been proven in several commercial environments, including integrated circuit fabrication plants and brokerage/trading floors.
... 2.2. Atomic Actions 17 Argus implements atomicity through the objects shared by actions; thes... more ... 2.2. Atomic Actions 17 Argus implements atomicity through the objects shared by actions; these objects are called atomic objects. Atomic objects, like ordinary objects, provide operations that can be used to access and manipulate them. In addition, however, their operations ...
In a database to which data is continually added, users may wish to issue a permanent query and b... more In a database to which data is continually added, users may wish to issue a permanent query and be notified whenever data matches the query. If such continuous queries examine only single records, this can be implemented by examining each record as it arrives. This is very efficient because only the incoming record needs to be scanned. This simple approach does not work for queries involving joins or time. The Tapestry system allows users to issue such queries over a database of mail and bulletin board messages. The user issues a static query, such as “show me all messages that have been replied to by Jones,” as though the database were fixed and unchanging. Tapestry converts the query into an incremental query that efficiently finds new matches to the original query as new messages are added to the database. This paper describes the techniques used in Tapestry, which do not depend on triggers and thus be implemented on any commercial database that supports SQL. Although Tapestry is...
Massachusetts Institute of Technology eBooks, Aug 1, 1988
... 2.2. Atomic Actions 17 Argus implements atomicity through the objects shared by actions; thes... more ... 2.2. Atomic Actions 17 Argus implements atomicity through the objects shared by actions; these objects are called atomic objects. Atomic objects, like ordinary objects, provide operations that can be used to access and manipulate them. In addition, however, their operations ...
Modern cluster management systems like Kubernetes and Openstack grapple with hard combinatorial o... more Modern cluster management systems like Kubernetes and Openstack grapple with hard combinatorial optimization problems: load balancing, placement, scheduling, and configuration. Currently, developers tackle these problems by designing custom application-specific algorithms---an approach that is proving unsustainable, as ad-hoc solutions both perform poorly and introduce overwhelming complexity to the system, making it challenging to add important new features. We propose a radically different architecture, where programmers drive cluster management tasks declaratively, using SQL queries over cluster state stored in a relational database. These queries capture in a natural way both constraints on the cluster configuration as well as optimization objectives. When a cluster reconfiguration is required at runtime, our tool, called Weave, synthesizes an encoding of these queries into an optimization model, which it solves using an off-the-shelf solver. We demonstrate Weave's efficacy ...
Cluster managers like Kubernetes and OpenStack are notoriously hard to develop, given that they r... more Cluster managers like Kubernetes and OpenStack are notoriously hard to develop, given that they routinely grapple with hard combinatorial optimization problems like load balancing, placement, scheduling, and configuration. Today, cluster manager developers tackle these problems by developing system-specific best effort heuristics, which achieve scalability by significantly sacrificing the cluster manager’s decision quality, feature set, and extensibility over time. This is proving untenable, as solutions for cluster management problems are routinely developed from scratch in the industry to solve largely similar problems across different settings. We propose DCM, a radically different architecture where developers specify the cluster manager’s behavior declaratively, using SQL queries over cluster state stored in a relational database. From the SQL specification, the DCM compiler synthesizes a program that, at runtime, can be invoked to compute policy-compliant cluster management de...
Research can rarely be performed on large-scale, distributed systems at the level of thousands of... more Research can rarely be performed on large-scale, distributed systems at the level of thousands of workstations. In this paper, we describe the motivating constraints, design principles, and architecture for an extensible, distributed system operating in such an environment. The constraints include continuous operation, dynamic system evolution, and integration with extant systems. The Information Bus , our solution, is a novel synthesis of four design principles: core communication protocols have minimal semantics, objects are self-describing, types can be dynamically defined, and communication is anonymous. The current implementation provides both flexibility and high performance, and has been proven in several commercial environments, including integrated circuit fabrication plants and brokerage/trading floors.
... 2.2. Atomic Actions 17 Argus implements atomicity through the objects shared by actions; thes... more ... 2.2. Atomic Actions 17 Argus implements atomicity through the objects shared by actions; these objects are called atomic objects. Atomic objects, like ordinary objects, provide operations that can be used to access and manipulate them. In addition, however, their operations ...
In a database to which data is continually added, users may wish to issue a permanent query and b... more In a database to which data is continually added, users may wish to issue a permanent query and be notified whenever data matches the query. If such continuous queries examine only single records, this can be implemented by examining each record as it arrives. This is very efficient because only the incoming record needs to be scanned. This simple approach does not work for queries involving joins or time. The Tapestry system allows users to issue such queries over a database of mail and bulletin board messages. The user issues a static query, such as “show me all messages that have been replied to by Jones,” as though the database were fixed and unchanging. Tapestry converts the query into an incremental query that efficiently finds new matches to the original query as new messages are added to the database. This paper describes the techniques used in Tapestry, which do not depend on triggers and thus be implemented on any commercial database that supports SQL. Although Tapestry is...
Uploads
Papers by Brian Oki