Organizational patterns

Organizational patterns are inspired in large part by the principles of the software pattern community, that in turn takes it cues from Christopher Alexander's work on patterns of the built world. Organizational patterns also have roots in Kroeber's classic anthropological texts on the patterns that underlie culture and society. They in turn have provided inspiration for the Agile software development movement, and for the creation of parts of Scrum and of Extreme Programming in particular.

History

An early explicit citation to patterns of social structure can be found in the anthropological literature.

Patterns are those arrangements or systems of internal relationship which give to any culture its coherence or plan, and keep it from being a mere accumulation of random bits. They are therefore of primary importance.

Kroeber speaks of universal patterns that describe some overall scheme common to all human culture; of systemic patterns are broad but normative forms relating to beliefs, behaviors, signs, and economics; and total culture patterns that are local. Kroeber notes that systemic patterns can pass from culture to culture:

A second kind of pattern consists of a system or complex of cultural material that has proved its utility as a system and therefore tends to cohere and persist as a unit; it is modifiable only with difficulty as to its underlying plan. Any one such systemic pattern is limited primarily to one aspect of culture, such as subsistence, religion, or economics; but it is not limited areally, or to one particular culture; it can be diffused cross-culturally, from one people to another. . . . What distinguishes these systemic patterns of culture—or well-patterned systems, as they might also be called—is a specific interrelation of their component parts, a nexus that holds them together strongly, and tends to preserve the basic plan... As a result of the persistence of these systemic patterns, their significance becomes most evident on a historical view.

The pattern aspect of Kroeber's view fits very well the systems-thinking pattern view of Christopher Alexander in the field of architecture. Alexander's books became an inspiration for the software world, and in particular for the object-oriented programming world, in about 1993. Organizational patterns in the sense they are recognized in the software community today first made an appearance at the original Hillside Group workshop that would lead to the pattern community and its PLoP conferences.

The Hillside Group sent out a call for pattern papers and, in 1994, held the first pattern conference at Allerton Park in central Illinois in the United States. The second conference, also at Allerton, would follow a year later. These first two PLoP conferences witnessed a handful of organizational patterns:

• The RaPPEL pattern language (1995) by Bruce Whitenack that described organizational structures suitable to requirements acquisition;
• The Caterpillar's Fate pattern language (1995) by Norm Kerth that described organizational structures supporting evolution from analysis to design;
• A work by James Coplien (1995) describing several years of organizational research at Bell Laboratories;
• Episodes, a pattern language by Ward Cunningham (1996) describing key points of what today we would call Agile software development;
• A pattern language by Neil Harrison (1996) on the formation and function of teams.

A flurry of associated publications and follow-up articles followed quickly thereafter, including an extemporization of the organizational patterns approach in the Bell Labs Technical Journal, an invited piece in ASE, a CACM article by Alistair Cockburn and, shortly thereafter, a pattern-laden book by Alistair, as well as chapters by Benualdi and Janoff in the Patterns Handbook. It was also about this time that Michael A. Beedle et al. published patterns that described explicit extensions to existing organizational patterns, for application in projects using a then five-year-old software development framework called Scrum. A few more articles, such as the one by Brash et al. also started to appear.

Little more happened on the organizational patterns front until the publication of the book by Berczuk et al. on configuration management patterns; this was a break-off effort from the effort originally centered at Bell Labs.

In the meantime, Jim Coplien and Neil Harrison had been collecting organizational patterns and combining them into a collection of four pattern languages. Most of these patterns were based on the original research from Bell Laboratories, which studied over 120 organizations over the period of a decade. These empirical studies were based on subject role-play in software development organizations, reminiscent of the sociodramas of Moreno's original social network approach. However, the pattern language also had substantial input from other sources and in particular the works by Cockburn, Berczuk, and Cunningham. This collection was published as Organizational Patterns of Agile Software Development in 2004.

One of the most recent organizational pattern articles comes from an early pattern contributor and advocate, the object design pioneer Grady Booch.

Principles of discovery and use

Like other patterns, organizational patterns aren't created or invented: they are discovered (or "mined") from empirical observation. The early work on organizational patterns at Bell Laboratories focused on extracting patterns from social network analysis. That research used empirical role-playing techniques to gather information about the structure of relationships in the subject organization. These structures were analyzed for recurring patterns across organization and their contribution to achieving organizational goals. The recurring successful structures were written up in pattern form to describe their tradeoffs and detailed design decisions (forces), the context in which they apply, along with a generic description of the solution.

Patterns provide an incremental path to organizational improvement. The pattern style of building something (in this case, an organization) is:

1. Find the weakest part of your organization
2. Find a pattern that is likely to strengthen it
3. Apply the pattern
4. Measure the improvement or degradation
5. If the pattern improved things, go to step 1 and find the next improvement; otherwise, undo the pattern and try an alternative.

As with Alexander-style patterns of software architecture, organizational patterns can be organized into pattern languages: collections of patterns that build on each other.

A pattern language can suggest the patterns to be applied for a known set of working patterns that are present.

Organizational patterns, agile, and other work

The history of Agile software development and of organizational patterns have been entwined since the beginning. Kent Beck was the shepherd (interactive pattern reviewer) of the Coplien paper for the 1995 PLoP, and he mentions the influence of this work on extreme programming in a 2003 publication. The idea of daily Scrum meetings in fact came from a draft of an article for Dr. Dobb's Journal that described the organizational patterns research on the Borland QPW project. Beedle's early work with Sutherland brought the pattern perspective more solidly into the history of Scrum. More recently, the Scrum community has taken up newfound interest in organizational patterns and there is joint research going forward between the two communities. In this vein, the first ScrumPLoP conference took place in Sweden in May, 2010, sanctioned by both the Scrum Alliance and the Hillside Group.