The Method Radar: a way to organize methods for technology development with participation in mind

2.1 Models of participation

Arnstein [11] designed 1969 with the ‘ladder of participation’ a first-stage model of participation, which should serve to visualize an exchange between citizens and politicians and to recognize possible “empty forms of participation” [12]. This ladder divides the cooperation between decision-makers of urban projects and affected citizens into eight stages from active manipulation to a process steered by citizens. This is shown in Figure 2 and explained afterwards. These stages are not to be assumed as complete or comprehensive [11].

1. Manipulation

   describes a form of non-participation in which, however, the appearance of participation is to be created from the outside, e.g. through pseudo-committees with no real decision-making power.

2. Therapy

   describes a form of non-participation in which, the values/opinions of the “participations” is changed to match the desired result, instead of the other way round.

3. Informing

   describes the first and most important step towards participation. You can make decisions only when you know your rights, duties, and options.

4. Consultation

   describes the process of asking the target audience for their opinion. For example through surveys.

5. Placation

   describes the first level where some degree of influence from the target audience is visible. For example, by electing representatives to accompany the process or to provide feedback.

6. Partnership

   describes a power redistribution between all parties. Decision-making is made in collaborative groups.

7. Delegation

   describes a decision-making authority in the hand of the target audience.

8. Citizen Control

   describes that the whole process is in the hands of the individuals for which the decisions are relevant.

Figure 2:

Ladder of partizipation arnstein [13].

A later interpretation based on Arnstein’s [11] model was presented by Roger Hart [14]. This interpretation explicitly addresses the participation of children, who in many countries are excluded from state democratic processes (e.g. elections). Roger Hart considers this state of affairs to be insufficient and not in accordance with the applicable conventions of children’s rights. In order to provide a means of discussion, he has developed his form of ‘ladder of participation’. This is meant to serve as a linguistic metaphor to be able to talk about the involvement of children. The ladder is shown in Figure 3. The consideration of Hart’s model is interesting in that he himself wrote a critical reflection on his ladder of participation in 2008 [16]. This critique is discussed in more detail later on in the discussion section.

Figure 3:

Ladder of participation following hart [15] (an adaption from [14]).

There is also the participation pyramid from Straßburger and Rieger [17], which focuses on the following aspects:

1. Who initiated the process, and who bears responsibility for its further course?

2. How far do the possibilities of co-determination extend?

3. What enables and promotes co-determination or hinders or limits it?

The seven levels of the pyramid underline the spectrum between minimum participation and decision-making power, whereby the higher the level, the greater the degree of participation. The left side of the pyramid shows the view of the professionals on participation processes, and the right side the view of the citizens (or more concretely: addressees). As a special feature here, the half-pyramid tip is to be noted, since this degree of participation can only happen in civic self-direction. According to Straßer and Rieger [17], the central question for the classification of the degree of participation should be the following: “How are decisions made? With or without the citizens and addresses?” The answer to this question can be classified into different levels. It should also be mentioned that Chung and Lounsbury [18] have created a model, which represents the interaction between researchers and participants in eight levels. There are also bigger models like the twelve-stage model for citizen participation from Trojan [19]. Finally one of the most interesting models is the Kreis der Entscheidung [20], which can be translated as the circle of decision (this translation will be used in later references). The interesting aspect of it is the circular representation of the ladder model. This circular model fits perfectly into a radar representation and is there used later on as described in section 3. It can be seen in Figure 4. To give a comparison between the different levels of participation presented in the beginning by Arnstein [11], Hart [14], and Wright [20] a comparison is presented in Table 1. It shows the similarities between the models but also that there is a slight shift in the definition of each level. To make the comparison easier, the original names from the circle of decision, which are in German, can be found in Figure 4, but appear translated to English for the comparison table (Table 1). In comparison to the models of participation which evaluate methods after their usage, there is also a simpler approach used by Stickdorn et al. [22]They are categorizing methods beforehand in non-participative and participative approach. They just differentiate whether there is interaction between researchers and a target group. E.g. a method like cultural probe kits [22, p. 124] is listed as non-participative as the researchers do not face-to-face interacting with the target group. Within the ladder of participation, the probe kit can have participative aspects as the participants give feedback through the probe kit.

Figure 4:

Circles of decision (translated by the author) [21].

2.2 Technology radar

There are numerous established metrics in the IT industry that assess and classify trends, technologies, and products. “In a broader sense, the goal of a radar or map is to gain a competitive advantage by identifying opportunities and threats emerging from technological developments at an early stage and to provide the technological capabilities needed to face these challenges” defines Rohrbeck [5], who also classifies different goals and approaches in technology scouting.

Visualizations for such scouting studies appear in various standardized styles. Amongst the most well-known is the Gartner Magic Quadrant [23] and some of its competitors from other established consultancy companies. They use the so-called “Boston square” visualization, consisting of a 2 × 2 matrix, with technologies or products shown with symbols. This allows showing two main dimensions (x- and y-axis), plus additional information in the symbol shape, size, or color.

Also well known in the IT industry is the Gartner Hype Cycle [24], basically a graph with an upward slope, peak, trough, and then a plateau line in middle height. Trends, technologies, or products are shown as symbols on the >>hype cycle<<, visualizing the maturity and perception of each contestant.

The most versatile of these display styles is the radar chart. A radar chart in general has a spider-web structure, allowing it to show several related market areas as segments in the circle. In addition, concentric rings allow a classification according to maturity levels or similar categories. Trends, technologies, and products can then be displayed by symbols in varying shapes and colors. A more in-depth description of the fundamental function of the radar representation used in the technology sector can be found in the work from Schuh and Klappert [3], who explain it under the name “monitoring radar”. Although there are alternative visualizations (e.g. scored profiles [25] or technology curves [26]), this is an intuitive and well-established way of representing information.

The most prominent technology radar, according to Kroß et al. [27], is the Thoughtworks technology radar [7].^[1] The IT consulting company Thoughtworks evaluates every six months the technologies they are using and structures them in a so-called technology radar. It is a tool to get a broad overview of many of the currently used technologies, and in which state they are currently.

The basic representation that is used is a structure that looks similar to a radar screen, where the name comes from. This radar contains a set of technologies each represented in one dot in the system (later on only called a blip). This can be seen in Figure 5, which is a radar from 2016 that is still presented in a single image. The radar is primarily showing the maturity of a technology. This is done by using rings, in which each ring represents one maturity of a technology, from “hold” (technologies that should be avoided) over “assess” (could be interesting but was not tested yet by Thoughtworks) and “trial” (has been tested and is a good solution to consider in the right situation) to the highest level of “adopt” (should be used if appropriate). In Addition, to this maturity grade, the radar is split into quadrants to roughly divide the technologies into thematic areas. Finally, the blips themselves are highlighted by their shape when they are new on the radar.

Figure 5:

Full overview of technology radar from Thoughtworks [30].

In Figure 6 a detailed version of one quadrant is shown.

Figure 6:

Detail view of one quadrant in Thoughtworks technology radar [7].

Radar representations are discussed in many publications for a variety of reasons. Berndt and Mietzner [4] address how a technology radar can help facilitate knowledge and technology transfer. It shows a good example of what a radar visualization has to offer. Rohrbeck [5] also presents a radar similar to the cisco technology radar but for technology scouting. Boe-Lillegraven and Monterde [6] explore how and why foresight can influence ways of processing information based on the cisco technology radar. The paper conducted a survey and identified the most important reasons for using technology radar as the following:

1. “It helps us communicate”

2. “It is good for business”

3. “It brings new perspectives”

4. “We like the process”

5. “It helps carry out our task”

These reasons show the importance that a categorization system brings to any kind of process, as it shapes not only the personal perspective but also the communication between each other.

3.1 Transforming technology radar to Method Radar

Starting with the technology radar as shown in Figure 7 the four maturity grades (hold, assess, trial, and adopt) is replaced by the seven levels of participation (not involved – nicht beteiligt, information – information, consultation – anhörung, involvement – einbeziehung, codetermination – mitbestimmung, partial decision-making authority – teilw. entscheidungskompetenz, full decision-making authority – entscheidungsmacht) based on the circle of decision, as shown in Figure 4. The four quadrants (techniques, tools, platforms, and frameworks) are no longer appropriate categories for the content as methods need a different kind of categorization. For this categorization, the well-known phases of the double diamond [22, p. 88] (Discover, Define, Test, Deliver) are used. A translated version in form of the categories “Entdecken, Definieren, Testen, Entwickeln” is planned for the target (German) audience. These phases are exchangeable with different categories depending on the type of methods that are used. In this case, the design focuses on methods for participative technology development, which can be categorized by the double diamond.

Figure 7:

Technology radar from Thoughtworks (newer version) [7].

The concept art of the resulting method radar can be seen in Figure 8. It shows the radar separated in seven rings with the previously mentioned levels (in German) and four quadrants based on the double diamond (which will be translated to German in the implementation). The coloring of the radar itself and the blips are exchangeable to match the surrounding content. The different coloring blips is intended for existing methods (light green), newly developed methods (dark green), and currently selected element (red).

Figure 8:

Method Radar.

3.2 Adding methods to the radar

Adding methods to the radar requires categorization of them into a level of participation. The following question structure should aid in determining the right ring in the radar. Which is also visualized in Figure 9.

1. Is it clear who should be participating?

   Who should benefit from the usage of the method and should they be part of it?

   e.g. Is it about the participation of a social work professional, a young person in an institution, or a scientist?

   For technology development always choose the perspective of the target audience for which the technology is developed.

2. Is any type of exchange with the target group planned?

   Pure observing or watching footage for example does not create any exchange with the group itself.

3. Is the exchange direct?

   Exchanging through a third party filter the real opinion of the target audience through the third party.

   E.g. Asking professionals of social work what technology would help the juveniles most. It will always be what they think is the best for the juveniles.

4. Is the target group informed enough to make decisions?

   Participation can only happen when the participants can make informed decisions. Every decision without information can be interpreted as non-participative.

5. Is the target audience consulted for their opinion?

   The first step in participation is to ask. This could be starting a method by asking participants what technologies they think might be helpful to them.

6. Is the target group’s opinions consulted on findings/needs/plans found?

   The next level of participation is to not only ask for opinions but also get feedback on the findings. This ensures that the conclusions drawn are also accurate.

7. Are representatives of the target group involved in the process?

   Representatives from the group can become part of the process and help at least partly to get the target’s opinion.

8. Does the target group make decisions in at least one, not insignificant subarea?

   An important question is what decisions the target group is able to make. These decisions need to be important otherwise its false participation.

   E.g. Building a house and asking the future residents what color the hallway should have would be not significant.

9. Does the target group make all significant decisions?

   If this happens the target audience is in the primary driver role, while others are just supporters of them.

10. Does the target group initiate and takes responsibility for the action itself?

    The highest level of participation is when a group imitates something by themselves. This is not represented in the radar as it is rarely the case when applying methods.

Figure 9:

Flowchart for adding methods to the radar.

Besides the described level of participation which is presented as the rings on the radar, there are also the quadrants. To sort methods into the quadrants the primary focus of the method should be considered and sorted into one of the four categories They can be described as follows:

1. Discover

   Methods that focus on discovering and understanding a domain to get insights into it are placed here.

2. Define

   Methods that condense the discovered insights are listed here. These methods help to focus on the right things and define the steps to be taken to move on.

3. Develop

   Methods to take the steps that have been previously defined to create a solution. This can be ideation, prototyping, testing, and much more.

4. Deliver

   Methods that focus on wrapping things up, creating finished goods, and getting them out to the users.

To get a better understanding of the process of adding methods some examples are given. While participation is in focus here the sorting into the right quadrants is also done at the end of the section. The methods are taken from Stickdorn et al. [22]. The first example would be “preparatory research” [22, p. 144]. This is a desk research method that tries to get an understanding of what might be the right questions in a topic. It is intended to get a process started and can be considered non-involvement. If we follow the flowchart provided in Figure 9 it would look as follows:

1. Is it clear who should be participating?

   Depending on the topic that is researched, we may know the exact target group or have more of a rough idea. Although this question can certainly be discussed, it will become clear at the latest with the next questions that no participation is envisaged.

2. Is any type of exchange planned?

   No, this method does not include any exchange with people besides the researchers.

In comparison, “participant observation” [22, p. 146] directly targets the right audience as it involves understanding their environment. Depending on the exact definition it can be just watching or asking questions during the observation. It does not plan to bring the target to a level to make informed decisions and therefore it is in most cases non-involvement. Using the flowchart:

1. Is it clear who should be participating?

   Yes, the target group we are observing.

2. Is any type of exchange planned?

   As mentioned above it can be argued but again this is solved by looking further ahead. It is not intended by the method to inform the target group enough so that they can make informed decisions.

3. Is the exchange direct?

   Most likely yes.

4. Is the target group informed enough?

   Most likely no. They are regarded as objects of observation in the method.

A journey map [22, p. 152] on the other hand starts of with communicating the scope of the map and its usage. It invites the participants and not asks them but also discusses with them every aspect of the journey through multiple iterations. This can be considered to be partial decision-making authority.

1. Is it clear who should be participating?

   Yes, this is a requirement.

2. Is any type of exchange planned?

   Yes, as part of the method.

3. Is the exchange direct?

   Yes, the participants help form the journey.

4. Is the target group informed enough?

   Yes, the method intends that the participants take part in the discussions to form the journey, which requires sufficient information.

   With this knowledge, we know we are Rang one (information) on the radar or higher.

5. Is the target audience asked for their opinion?

   Yes, as mentioned they are not only asked they are enforced to take part in the discussion.

   We are now in Rang two (consultation) or higher.

6. Are the group consulted on findings/needs/plans?

   Yes, as mentioned above they are forming the findings together.

   We are now in Rang three (involvement) or higher.

7. Are representatives involved in the process?

   Yes, as mentioned above.

   We are now in Rang four (codetermination) or higher.

8. Does the group make significant decisions?

   Yes, they are taking part in shaping the whole journey which can be considered significant in regard to the method. We are now in Rang five (partial decision-making authority) or higher.

9. Does the group make all significant decisions?

   This can be discussed as depending on the execution the target audience can decide everything or can be restricted. Whether the method matches more full decision-making authority or particular decision-making authority depends on the strictness of the evaluation, which is to be determined by the user of the radar.

With these steps, the methods are categorized in regard to the participation level, which is represented through the rings in the radar. The Method Radar does offer a second dimension to sort the methods, which focuses on the phase in a process, based on the double diamond. The first two methods “preparatory research” and “participant observation” are intended to get a first understanding of a domain, its scope, and possible questions that need to be tackled. These two methods are part of the first quadrant which is “discover”. A journey map on the other hand helps get all the insights sorted and bring them into shape. It is part of the “define” step. Methods that focus on generating solutions, prototyping, and testing are sorted in the “develop” category, while the last quadrant “deliver” hosts methods that focus on creating a deliverable product and how to get it into the world. More examples can be seen in the implemented radar which is mentioned in chapter 4.3.

4.1 Usage

The Method Radar can be added to any standard HTML webpage. In the Listings used as an example, it is added to a GitHub page [34]. GitHub pages, by default, run a static HTML site generated by the tool Jekyll [35]. Whenever it is essential in the usage, the Listing mentions if anything is different from vanilla HTML. As the radar is based on D3.js [32] it is required to add the library as a script. In addition, it is necessary to import the radar script itself and add a style sheet. All files can be imported as external references and later copied into the project to customize everything.

After adding the required files, the radar must be placed on the website. This is done by adding a div element to the location where the radar should be later. This can be seen in Listing 1.

Through the id in the div the javascript locates the element and replaces it with the radar. This function expects three parameters that define the radar. The parameters are the configuration of the radar, its structure, and the entries. To allow the usage in any webpage, the expected format for the three parameters is JSON [36]. Listing 2 shows how the files can be added to the radar.

After these few steps, the page can be rebuilt, and the radar will be visible. The next step is to add entries to the radar and customize it to fit the own page.

4.2 Configuration

The radar can be configured through three files. The first file focuses on the radar’s configuration, e.g. coloring or naming. It contains four sections: One general, one for sectors, one for segments, and one for the blips. The second file that is needed defines the structure of the radar. That means it defines how many rings and sectors the radar has. It also defines the possible states that blips can have. It is also possible to visualize the movement of blips between different versions of the radar. The last JSON file is the list of entries in this case a list of methods.

4.3 Example

The final implementation of the Method Radar is viable under https://intia.de/werkzeuge/methoden/ [31]. It consists of an explanation of the Method Radar, its concept, and how to use it. Besides the radar, a list of the methods is provided, allowing a user to read short abstracts of the methods and hyperlinks to a detailed description of the method and its application. Currently, not all methods sorted in the radar have a detailed description of the method. The coloring of the blips is here used to indicate methods already on the website and are currently only categorized without description. A screenshot of the side is viable in Figure 10.

Figure 10:

Implementation of the method radar.

In this Figure, a radar with twelve methods is shown, which have been used in the INTIA research project. They are sorted according to the usage in the project and are mostly in German. Nevertheless, as an example, they are presented here. The darker-colored methods are described on the webpage, the lighter colors are not yet defined on the website. The list of the methods:

Analog prototyping for example is an open approach that intends to help give ideas a form and discuss possible solutions. It consists of a set of tiles with different (geometric) shapes. The tiles can be used to lay simple sequences. The tiles can be taken in hand and arranged according to rules. This makes it easy to come up with ideas together and map them out. It’s a method that focuses on helping a target group shape their ideas within the smart home domain (limited by the types of tiles). The intention is that people who are not developers can use it. The tiles are used to represent ideas in a simple programming language. To do this, the different tiles must be placed in a row so that a set is created. The programming tiles stand for corresponding actions. There are four types of programming tiles: tech (e.g. sensors), actions (e.g. open/close), and sentence structure (e.g. if/then). The tiles can be placed together in groups or individually. Here is an example: All sentences start with a sentence structure tile (in this example “If”). This is followed by a tech tile (“contact sensor”) and an action (“close”).

If the contact sensor is closed, then the smart lamp goes on.

Everything from the starting idea that is shaped to laying tiles is done by the target group. It does completely rely on the target group. Using the same approach as in chapter 3.2:

1. Is it clear who should be participating?

   Yes, this is a requirement.

2. Is any type of exchange planned?

   Yes, as part of the method.

3. Is the exchange direct?

   Yes, the participants exchange primarily with each other and the researchers are only on request part of the discussion.

4. Is the target group informed enough?

   Yes, the target group is briefed on the method beforehand and works on their own ideas.

   With this knowledge, we know we are Rang one (information) on the radar or higher.

5. Is the target audience for their opinion?

   Yes, as mentioned they are not only asked they are the primary part of the discussion.

   We are now in Rang two (consultation) or higher.

6. Are the group consulted on findings/needs/plans?

   Yes, as mentioned above they are forming the findings primarily by themself.

   We are now in Rang three (involvement) or higher.

7. Are representatives involved in the process?

   Yes, the whole process is shaped by the target group and only limited in the domain by the hexagonal plates.

   We are now in Rang four (codetermination) or higher.

8. Does the group make significant decisions?

   Yes, they are deciding which ideas are further developed, how they take shape and when a good solution is found. We are now in Rang five (partial decision-making authority) or higher.

9. Does the group make all significant decisions?

   This can be discussed (similar to the journey map) as depending on the execution the target audience can decide everything or can be restricted. Whether the method matches more full decision-making authority or particular decision-making authority depends on the strictness of the evaluation, which is to be determined by the user of the radar.