Keywords

1 Introduction

With the growing development of the internet and communication and data manipulation technologies, the amount of information that reaches users about these technologies has been increasing greatly. However, not all data that reach them are effectively useful for them, and this excess ends up becoming a problem in understanding the results obtained [18]. In order to facilitate the analysis of these data, Information Visualization techniques are introduced, since a visual representation of the data set makes understanding more simple and intuitive. This reduces the time spent on interpreting the information in addition to reducing the possibility of doing it incorrectly.

For Spence [15], visualization is something that takes place in the mind; in other words, it is a cognitive activity. This author also says that a determined data set can often hide, in a way, relationships that are fundamental for understanding, and a different perspective on this data set can reveal important information. In this case, information visualization is, according to Spence, the process of forming a mental model of a determined data set and through it, one can obtain some knowledge.

A source of data that could be better analyzed by means of visualization techniques is the one obtained through social networks, which accumulate daily a large amount of information. The information contained in these networks has been drawing the attention of common users and businesses, which use them in some way for some personal benefit. Their millions of adepts produce an excessive amount of data, useful or not, and these data are disseminated through the networks. Therefore, depending on the analysis that users need to carry out based on these data, they may need some way to filter the content that is provided to them.

Since information from social networks has been of great value for different areas, various visualization and interaction techniques have been developed and used to present this information using a computational interface. And today, the ubiquity of multi-touch technology [3] opens up a range of new possibilities of interaction.

In this context, the main research question of the work here presented is: Does the multi-touch interaction with information visualization techniques can improve the data analysis? And related to this: How the interaction could improve it? Is this kind of interaction better that the conventional keyboard plus mouse combination?

In order to answer these questions, the main goal of this work is to analyze the use of information visualization techniques allied to the interaction through multi-touch devices. To achieve this goal, we have used the following methodology: first we did a research about information visualization, social networks and multi-touch technology visualization. Then we implemented a prototype and we made a user study to analyze the mentioned interaction issues. Moreover, to implement the prototype, a process was applied to identify the appropriate multi-touch gestures to be implemented.

The next sections describe the research related to that presented in this paper, the process applied to study the gestures and the conception of the prototype developed. After, we analyze user interactions with the implemented prototype and we present some final consideration, concluding this paper.

2 Related Work

Due to the popularity of multi-touch interfaces, some studies have been carried out with the purpose of reconsidering information visualization techniques in order to enable interaction through these new interfaces. According to Isenberg et al. [7], “multi-touch interaction give people additional degrees of freedom to express their intentions and provide more direct access to their objects of interactions”.

For example, Kister, Reipschläger and Dachselt [8] discuss multi-touch manipulation using the magic lenses technique. Baur, Lee and Carpendale [1] developed a tool for analyzing stacked graphs that explore multi-touch gestures. Kristensson et al. [9] created a tool for search and photo visualization that uses information visualization techniques such as tag clouds and graphs, and in it, the user has the freedom to use all ten fingers for interaction.

There are also other studies that focus on tasks that are more complicated to carry out through these interfaces, as is the case with the Rizzo tool [17], or even the Touch2Annotate [2]. These studies focus on the weak points of multi-touch interfaces: precision and typing. Precision is also discussed by Thalmann et al. [16].

According to Lee et al. [10], information visualization techniques “do not take full advantage of the possibilities of evolving interaction technologies”. Rzeszotarski and Kittur [13] also explored how interactions different of WIMP “might improve exploratory data visualization through the introduction of physics-based affordances and multi-touch interaction techniques”.

3 Process for Elaborating Gestures

A process based on user opinions [4] was applied to define which gestures to use for interacting with the information visualization techniques to be implemented. This process consists of creating a focus group to discuss the data collected by means of a series of semi-structured individual interviews. Its five stages are described as follows.

3.1 System Definition

In the first stage of the process, the desired system to be used as the basis of study is chosen. Moreover, a set of initial commands is defined based on the purpose of the system with which one intends to produce the gesture interactions.

Considering the goal of analyze the use of information visualization techniques allied to the interaction through multi-touch devices, the main idea was to work with visualization techniques that could both correlate the gathered data and encourage users to interact with these data to get more information. In this context, the system would present interactive bubble charts, showing the correlation between distinct data and presenting more related information by the bubbles resizing.

To present to the users the idea to be developed, the application RipplesFootnote 1 was used. It consists of an information visualization tool in the social network Google +. This tool allows one to visualize the propagation of a post within this network. The visualization of post diffusion is composed of various circles and each one represents the profile of who shared the post. Circles within another circle, connected by arrows, represent the flow of the shares; i.e., the profiles that shares the post of another profile. As Ripples was not developed with a focus on multi-touch interfaces, interacting with it was symbolic, but allowed users to understand the idea to be discussed.

In order to determine user preferences, a set of initial commands was defined to execute manipulations of simple data. The commands in this process were: selecting and removing elements that are near to each other; selecting and removing elements that are far from each other; applying zoom; and translating elements. Moreover, a list of various illustrated multi-touch gestures [5] was presented to the users and they were invited to propose specific actions that could be generated based on each gesture.

3.2 Individual Interviews

The next step in the process recommends semi-structured interviews, in a qualitative approach with a restricted number of users (from five to ten), in order to advance the discussion.

Thus, the interviews were carried out with seven participantsFootnote 2 from different areas, such as law, economics and computer science. They were between 20 and 33 years of age and all had some type of multi-touch device, with which they interact daily.

Some common gestures were presented to the participants while they were questioned about their experience with the gestures. Users were asked to simulate gestures to interact with Ripples in the multi-touch device. They were also encouraged to use various fingers at the same time, in order to make the most out of this feature. While the participants answered the questions and interacted with the device, video and audio were recorded to register their interactions and expressions. In the second stage of the interview, a new list composed of simple and complex gestures was presented to the participants, in order to ask them about the possible commands that could be created based on these gestures.

3.3 Analysis of Interviews

In the next stage, based on the results of the interviews, an initial set of gestures of interaction should be created. For each command, the proposals can be categorized according to similarity. These categories are then filtered and only one subset of all of the gestures proposed in the interviews will be used in the following stage.

The interviews were extremely useful, since it was possible to bring together various multi-touch gestures based on user opinions. Though they were familiar with these devices, they were instructed to not restrict themselves to the gestures they already knew and were encouraged to be creative in devising new gestures. A set of gestures was created, composed of several different gestures that used different combinations of fingers and movements.

3.4 Focus Group

The next stage is the creation of a focus group to validate, reduce, broaden or improve the set of interaction techniques proposed in the interviews. This focus group was composed of six participants, undergraduate and graduate students from the areas of economics and computer science between 18 and 32 years old.

The moderator showed and simulated the set of gestures proposed in the interviews on a tablet. Therefore, it was possible to discuss each gesture and collect the users’ opinions, eliminating gestures that did not meet the requirements of the group.

3.5 Defining the Gestures for Interaction

Concluding this part of the process, we obtained a short list of gestures according to the opinion of the users for later implementation, starting with the interviews and refining it with the focus group. Some of them are:

  • Selection of Elements: To select more than one circle, one of the preferred options was to touch the screen with five fingers to create a circle (or four to create a square) and this defined shape acts as a selection area. Therefore, to select elements, the user can drag them to this shape using one or more fingers simultaneously. The gestures to deselect elements, according to the users, should be the opposite.

  • Application of Zoom: To apply the zoom to the elements, the users chose to execute a double touch on the element or to use the traditional zoom.

  • Viewing Information: To open information of the profile represented by the circle, users indicated a simple touch with two or three fingers. In this case, a touch with two fingers was used for viewing the text of the tweet and a touch with three fingers was defined for viewing the profiles of those who retweeted

4 Prototype Implementation

As previously mentioned, the main goal of the prototype called TweetStatus, is to allow the analysis of information visualization techniques that use two forms of interaction: multi-touch technology, and keyboard and mouse.

As application area the social networks were elected. The large amount of data created on these networks encourages the use of visualization techniques to their analysis [12]. In this field, the Twitter social network has been widely used due its characteristics of “following” (a friendship link doesn’t need to be accepted by both parts) and public data (the data available in this network could be easily gathered).

Because of these, Twitter was chosen as the source of data to be visualized in the prototype. Therefore the prototype is aimed at analyzing tweets, enabling a search for tweets through a keyword to then carry out various analyses on the results found. It is possible to evaluate the feeling expressed in the tweet, the person who created it, how many times it was retweeted, the repercussion and frequency of the topic researched and trends over time.

To use TweetStatus is no need to login, the user just type a keyword of interest, the prototype searches for tweets posted that have this word and then it generates a visualization of the information of these tweets. The user can use two types of information visualization to analyze the search results: a Bubble chart and a Timeline.

4.1 Bubble Chart

Firstly, TweetStatus search the tweets that contain the word typed by the user and, based on the data found, it generates a Bubble chart. In the Bubble chart, each circle represents a tweet and in it, it is possible to see the name of the profile that posted it. Moreover, the size of each circle represents the number of retweets, in other words, the number of shares that the tweet had in the social network. The search can also be done with written tweets in Portuguese and English.

Another feature of TweetStatus is to analyze the feeling expressed in the tweet. Each circle is colored according to the “feeling” contained in the text of the tweet; in other words, green circles represent good tweets or those with a positive message regarding the researched topic. Likewise, yellow represents neutral tweets and red indicates bad tweets or those with negative messages. To implement the characteristic of coloring the tweet according to the feeling expressed, it was necessary to use a lexicon of feelings: the Opinion Lexicon [14]. This lexicon is constituted, basically, of a list of words which were given a determined weight and this expressed the feeling transmitted by them. Each word of the tweet is evaluated according to this lexicon and then the sentimental value of the tweet is calculated by adding or subtracting the corresponding value of each word. For the purpose of evaluating the words, the final value of the feeling is obtained and then a color is attributed to the circle.

In Fig. 1, it is possible to observe the circles representing the tweets and each one has a size and a color according to the information represented by it. Moreover, it is also possible to read the text of some tweet selecting it. Then it is possible to see if it is in accordance with the feeling represented by color.

Fig. 1.
figure 1

Bubble chart before (left) and after a tweet selection (right). (Color figure online)

Full size image

In order for the user to select circles and know how many retweets each one represents or even how many a group of circles represents, it is possible to create a selection circle. Therefore, the circles that represent tweets can be grouped together and an analysis of the total good, bad and neutral retweets can be carried out. The result is presented in a text box above the Bubble chart. Also in the Bubble chart, the user can choose to view the names of those who retweeted the determined tweet, which are also represented by circles, hence the color grey. These two features are demonstrated in Fig. 2.

Fig. 2.
figure 2

Circle selection (left) and retweet presentation (right).

Full size image

4.2 Timeline

With the prototype, the user also has the option of visualizing the tweets on a Timeline, in which they are represented according to the date and time at which they were posted. As in the Bubble chart, the tweets on the Timeline are also colored according to the feeling expressed. In this type of visualization, the user can observe the repercussions of a determined topic over time, such that, for example, the different feelings expressed over the course of a soccer game can be seen.

The Timeline is interactive and the zoom can be applied to find out the exact moment of the post, as well as navigate between the dates. Figure 3 presents an example of Timeline visualization before and after applying the zoom.

Fig. 3.
figure 3

Timeline before (top) and after applying zoom (bottom).

Full size image

5 User Study

In the last phase, in order to discuss the ways of interacting and visualizing that were implemented in TweetStatus, a qualitative analysis was carried out with ten users, based on their use of the developed prototype. The analysis process was structured in four stages, which are detailed below. The interactions in all the stages were filmed and the data recorded in notes.

5.1 User Profiles

Ten users were interviewed, four women and six men with ages ranging from 14 to 54. Of these, two did not complete secondary education, four were undergraduate students, three were graduate students and one was a post-graduate student. Their major expertise areas varied between engineering, computer science, economics, pharmacy and languages. Regarding their skills in using the technology, only one claimed to be a beginner and the rest stated that they were at a level between intermediate and advanced. Questioned about whether they used the social network Twitter, five of them answered affirmatively (four use it daily and one sporadically). Their focus on Twitter is to read news, keep up to date on friends as well as express their opinions and feelings.

5.2 Using the Two Versions of TweetStatus

The participants were invited to use the two versions of the prototype (multi-touch and keyboard plus mouse). To this end, they received a set of tasks, which were posed to them without the intention of receiving “correct” answers as result. The main goal was to stimulate the use of the prototype and not evaluate the accuracy of the data.

The following tasks were carried out: identify the number of good, bad and neutral tweets and retweets; evaluate whether the color of the circle corresponded to the text of the tweets that it represented; evaluate whether the size of the circle corresponded to the number of times that the tweets had been retweeted; evaluate the frequency of the topic researched within the social network; evaluate whether there was, at some moment, a certain “emotional stability”, indicated by a sequence of tweets with the same feeling; and, evaluate the repercussion of the topic in general (good, bad or neutral).

Since the users were invited to use two versions of TweetStatus, its use was alternated with each new participant to avoid the possibility of already being familiar with the prototype during the second use, which could compromise the results. Therefore, half of the participants used the conventional interaction and then the multi-touch interaction and the other half did so in the inverse order.

After the tasks execution, a questionnaire was applied intended to collect data about the experience of using the two versions of the prototype: difficulties found, points to improve, user opinions on the use of multi-touch devices with information visualization, in addition to comparative data between the two ways of interaction.

5.3 Data Analysis

After the interviews, an analysis was carried out on the data with the purpose of discovering whether, in this case, using one multi-touch interface was capable of raising the quality of the information visualizations.

According to three users, being able “to touch” the information makes it easier to visualize the information as well as understand it. Some also claimed it was a way of triggering greater interest in visualization and consequently forcing themselves to be more attentive to what they were doing. During the interviews, some comments included “Touching generates more interest in interacting with the application…” (Participant 10) as well as “Normally things that you can interact with, modify, are more interesting than things that are finished products. That is the logic of the touchscreen that is how they win us over.” (Participant 3). Though five of the ten users said that the information perceived in the two interactions was the same, the positive statements suggest that a visualization technique that can be touched adds value to the information or its understanding.

When faced with the statement about the fact that they can drag elements on the screen and if this represented some advantage in information visualization, using the Likert scale, 80 % of users marked “agreed” or marked “totally agreed” with it. Specifically talking about the Bubble charts, all participants agreed that the circle’s dragging is an advantage in the visualization manipulation. As Participant 3 said “It gives an information control feel and encourages interaction”. However, regarding whether object manipulation was easier through the multi-touch interface, 20 % of users disagreed or totally disagreed with it; one of them reported having problems in manipulating the small objects. This shows that the data resulting from the searches carried out by the participants also influenced the interaction. The information that resulted in small elements on the screen certainly led to a less pleasant interaction for the users.

During the interviews, the users were also asked to give their opinion on the multi-touch version of the prototype being more intuitive than the conventional version and, in this case, there were diverging responses. Some agreed with this statement, though three users disagreed, one of them pointing out that, despite using the keyboard as an aid, with the click of the mouse, the interaction was more intuitive. However, these users also agreed that it was easier to manipulate the elements with the mouse since they were more used to using it. This leads to a reflection on the fact that they are more habituated to using the mouse and, perhaps for this reason, they believed that this version was more intuitive. Analyzing the data on the reported difficulties, five users reported it to be difficult to remember the commands in the traditional version (keyboard plus mouse), and none of them highlighted this in relation to the multi-touch commands.

The users were unanimous in agreeing that the precision of the mouse facilitates the manipulation of small objects. Despite this, alternatives to these techniques can be developed which together with the multi-touch interfaces can provide a positive experience for the user. One example of improving the precision of the multi-touch interaction is Rizzo [17], which simulates the precision of the mouse in multi-touch interfaces.

In using the Timeline, for example, it was claimed that “it is much easier to move the mouse here” (Participant 2). We believe that the demand for more precision at the moment of interacting may have caused this feeling, since the space for manipulating the zoom was considerably small. The mouse, despite having only one point of interaction, had a scroll wheel to apply the zoom, making it much more intuitive, since there are many applications that use this same form of interaction.

Concerning to the application domain, the manipulation of tweets and their sentimental analysis through the Bubble charts, the users could easily find information about the amount of tweets and retweets (good, bad and neutrals). They all agreed about the weight fidelity of the circles related to the number of tweets and/or retweets they represent. However, they could perceive that some color was not well employed – they didn’t represent the sentiment expressed by the tweet. For example, the tweet selected in Fig. 1 (right) was classified as neutral, but someone could consider it as good. As our main focus in these phases of our research was not in the precision of the sentimental analysis, this fact was saved for future prototype refinements. Considering the Timeline, it allows them to identify the frequency the topic researched appears in the network and also its repercussion. Using the Timeline, 8 of the 10 participants could verify the “stability” of certain sentiment during time (“some periods with several bad tweets and others neutrals”, as highlighted by Participant 5). Considering this results we observe that these visualizations, combined with the possibilities of multi-touch interaction, could improve the analysis made by users, indicating a research path probably fruitful.

6 Final Considerations

The great increase in the volume of information over the past few years, and the consequent emergence of various information visualization techniques, evidence the importance of this area. The techniques developed for this end are presented in several applications used on a daily basis with the most diverse types of users. However, they are, for the most part, developed for desktop computers and used through interactions with the mouse and keyboard. With the popularity of multi-touch devices, due to the massive use of smartphones and tablets, various elements related to interaction have begun to be reconsidered and adapted to these types of devices. Among them are information visualization techniques.

Some studies have questioned whether multi-touch devices are capable of improving the user experience, and thus carried out comparisons with traditional WIMP interfaces [6, 10, 11]. With respect to the information visualization techniques, their adaptation to these devices is considered a challenge to be overcome [1]. On the other hand, it is also considered a more natural way of interaction for data visualization [10, 13].

In this sense, this research aimed to investigate these issues, bringing more data to the discussion. This technology, as mentioned by some of the participants of the tests carried out, have the capacity to hold the attention of the users and thus provide better concentration on their part during the interactions. An information visualization technique on its own is a way of attracting the attention of the users, due to the way the information is presented. When one interacts with it through a multi-touch interface, one can say that value has been added, since this interaction helps to maintain the focus of the user on the visualization. Moreover, when well-executed, these interactions can make the process for understanding the information more pleasurable for the user. Thus, we can say that in some way it can improve the data analysis.

In this research we analyzed the multi-touch interaction combined with highly used visualization techniques, in this case, the Bubble chart and the Timeline. The results obtained up to this point indicate that multi-touch interactions are capable of making users more attentive to their activities, thus improving the quality of the information acquired during these interactions, which is better than using conventional keyboard and mouse combination. Consequently, one can say that information visualizations have also benefitted from this improvement and thus its quality has added value as well.

It is interesting that the information visualization techniques undergo a process of specifying gestures that should be used when they are implemented in multi-touch interfaces, similar to that applied in this study. This prior specification prevents the designer from using (and implementing) gestures that may not be well accepted by the users.

The research also revealed that in tasks that require precision, such as, for example, selecting and editing long texts, multi-touch should be avoided. On the other hand, since most tablets and smartphones users do not use mouse and keyboard to interact with these kinds of devices, a challenge for future research is to find gestures to allow a pleasure way to do it. Moreover, using multi-touch in devices in which the user is already used to other types of interactions should not be the only source of interaction; multimodal interfaces could be an option. And information visualization techniques that are being designed for these devices should be carefully analyzed before being implemented.

With respect to the prototype’ domain area, specifically, the participants were unanimous in saying that they would use it to carry out an analysis of the tweets. We can say that it provides better quality in exploring tweets, since some information that could not be seen through the original interface of the social network were contemplated in TweetStatus. Being able to analyze the feeling expressed can make the user focus on the tweets of interest to them, such as, for example, reading only good tweets regarding a product; or reading some bad ones with respect to a store in which one intends to buy some product. A business or public person, such as a politician, for example, can also analyze who was involved and at what time a bad repercussion began about them in the network. This type of information can be very important in a decision-making process.

Since the focus of the research in this paper was not aimed at new information generated by the analysis of the tweets, with future studies, we intend to advance this discussion. We intend to analyze the applicability of the prototype with its use by different kinds of users of Twitter in their daily routine. Users such as businesses and public people, who need to monitor their reputations on social media, can have profiles more interested in analyzing this information than that of users in general, adding value to the results. In this same scope, an improvement to be incorporated in the prototype is the creation of a configurable and personalized lexicon for each user. In this lexicon, each user of the tool could decide which words should be classified as good or bad, attributing their own weights to them.