Blogosphere: Research Issues, Tools, and Applications Nitin Agarwal Huan Liu Computer Science and Engineering Department Arizona State University Tempe, AZ 85287 {Nitin.Agarwal.2, ABSTRACT Weblogs, or Blogs, have facilitated people to express their thoughts, voice their opinions, and share their experiences and ideas. Individuals experience a sense of community, a feeling of belonging, a bonding that members matter to one another and their niche needs will be met through online interactions. Its open standards and low barrier to publication have transformed information consumers to producers. This has created a plethora of open-source intelligence, or “collective wisdom” that acts as the storehouse of overwhelming amounts of knowledge about the members, their environment and the symbiosis between them. Nonetheless, vast amounts of this knowledge still remain to be discovered and exploited in its suitable way. In this paper, we introduce various state-of-the-art research issues, review some key elements of research such as tools and methodologies in Blogosphere, and present a case study of identifying the influential bloggers in a community to exemplify the integration of some major aspects discussed in this paper. Towards the end, we also compare and contrast the blogosphere and social networks and the research therein. 1. INTRODUCTION TO BLOGOSPHERE Weblogs or Blogs are becoming one of the most popular media of communication and interaction among masses. A blog can be defined as a website that displays, in reverse chronological order, the entries by one or more individuals and usually has links to comments on specific postings. Each of these entries are called blog posts. A typical blog post can combine text, images, and links to other blogs, web pages, and other media related to its topic. Some blog posts provide a list of links to similar or related blog posts. Such a list of links is called blogroll. The ability for readers to leave comments in an interactive environment is an important part of blogging. People express their opinions, ideas, experiences, thoughts, wishes through these free-form writings. The individuals who author the blog posts are referred as bloggers. The websites that publish these blog posts are termed as blog sites or blogs. Blog sites often provide opinions, commentaries or news on a particular subject, such as food, politics, or local news; some function more like personal online diaries. The universe of all these blog sites is often referred as Blogosphere. There has been a tremendous increase in user-generated content in the past couple of years via the phenomenon of blog- SIGKDD Explorations Huan.Liu}@asu.edu ging. Acknowledging this fact, Times has named “You” as the person of the year 2006. This has created a considerable shift in the way information is assimilated by the individuals. This paradigm shift can be attributed to the low barrier to publication and open standards of content generation services like blogs, wikis, collaborative annotation, etc. These services have allowed the mass to contribute and edit articles publicly. Giving access to the mass to contribute or edit has also increased collaboration among the people unlike previously where there was no collaboration as the access to the content was limited to a chosen few. Increased collaboration has developed collective wisdom on the Internet. “We the media” [21], is a phenomenon named by Dan Gillmor: a world in which “the former audience”, not a few people in the back room, now decides what is important. The “former” consumer of the information becomes the new producer, transforming the lecture style of information consumption to conversation-based assimilation. Blogs have also made it easy for the content generators to author content independent of technical challenges of internet languages and scripts. Bloggers don’t need to worry about the low level programming details, rather they focus only on the content. This simplifies the content generation process to a great extent and attracts novice or even computer illiterates to participate in blogging activities. Blogs provide a platform where anyone can express himself or herself freely without being even restrained by their limited computer knowledge yet being able to publish content on the Internet. Publishing on the Internet also facilitates the readers to comment instantly, giving bloggers a feeling of satisfaction. For many years, psychologists, anthropologists and behavioral scientists have studied the societal capabilities of humans. They present studies and results that substantiate the fact that humans like engaging themselves in complex social relationships and yearn to be a part of social groups. People form communities and groups for the same reasons to quench the thirst for social interactions. Often these groups have like-minded members with similar interests who discuss various issues including politics, economics, technology, life style, entertainment, and what have you. These discussions could be between two members of the group or involve several members. Internet has virtually reduced the distance between any two points on Earth to zero. It has made possible for people to connect with each other beyond all geographical barriers. Blogs, on the top of it, has tremendously affected social interactions between people and communities. People not only Volume 10, Issue 1 Page 18 participate in regional matters but also international issues. They can connect to people sitting on exactly the other side of globe and discuss whatever they like, i.e., a flat world [18]. Communities can be spread across several time zones. This humongous mesh of social interactions is termed as social networks. Blogs can be considered as a type of social networks that encompass interactions between different people, members of a community or members across different communities. Each person in a social network is represented as a node and the communications represent the links or edges among these nodes. Blogosphere comprises of several focussed groups or communities that can be treated as subgraphs. These communities are highly dynamic in nature that have fascinated researchers to study its structural and temporal characteristics. There are myriad services offered under the umbrella of social networks along with Blogs. Other services include social friendship networks like Friendster1 , Facebook2 ; collaborative annotation like del.icio.us3 , StumbleUpon4 that constitute “folksonomy”; media sharing services like flickr5 , YouTube6 ; and wikis7 . All these services offer a fertile ground for research. In this paper we focus on the blogosphere. The popularity and widespread use of blogs can be attributed to the changes brought by Web 2.0 in the way users interact with the web. Blogs have been around for quite some time but it became unprecedently popular with the advent of Web 2.0. Although Web 2.0 may not be a technological shift, it changed the way now people interact through the Internet. People could not only consume information on the Internet but also contribute to it. Easier, more intuitive interfaces with desktop-like experience enticed users to stay connected and contribute their knowledge in terms of blog posts, wiki articles, developing folksonomies, etc. Wikis is an excellent example of Web 2.0 that slowly takes over online encyclopedias due to its sheer breadth of knowledge made possible by mass editing. Since more and more people are trying to be a part of Web 2.0, it has generated enormous amounts of information on the web which is also known as collective wisdom or open source intelligence. The basic differences between Web 1.0 (or, the way Web was accessed previously) and Web 2.0 can be listed as follows: • Former information consumers are now also producers. Web 2.0 has allowed the mass to contribute and edit articles through wikis and blogs. • Giving access to the mass to contribute or edit has also increased collaboration among the people unlike Web 1.0 where there was no collaboration as the access to the content was limited to a chosen few. • Increased collaboration has generated enormous open source intelligence or collective wisdom on the internet which was not there in Web 1.0. 1 http://www.friendster.com/ http://www.facebook.com/ 3 http://del.icio.us/ 4 http://www.stumbleupon.com/ 5 http://www.flickr.com/ 6 http://www.youtube.com/ 7 http://www.wikipedia.org/ 2 SIGKDD Explorations ✄ ✌✍ ✄ ✑ ✟✞✠✡ ☛☞ ✎✠✏✏✆✁☞ ✄ ✌✍ ✁✂✄☎ ✄ ✂✆✝✞ ✟✞✠✡ ☛☞ ✒✓✔✕✖ ✗✔✖ ✘✗✙✔✚✗✙✔✕✖ ✛✜ ✒✓✔✕✖ ✗✔✖ ✘✗✙✔✚✗✙✔✕✖ ✛✜ ✗ ✙✔ ✖✙✢✙✖✣ ✗✤ ✣✥✕✦✥✧ ★✦✩✣✪ ✩✫ ✤✙✬ ✕✭✘✙✔ ✖✕✖ ✣✥✕✦✥✧ ✮✩✦✕ ✤✙✬✕ ✪✕✦✥✩✔ ✗✤ ✗✯✯✩✣✔✚✥ ✰ ✮✩✦✕ ✤✙✬✕ ✖✙✥✯✣✥✥ ✙✩✔ ✫✩✦✣ ✘✥ ✲✩ ✩✦ ✗✤✘✩✥✚ ✔✕★✤✙★✙✛✤✕ ★✦✩✣✪ ✳✙★✴ ✖✕★✦✕✕ ✩✫ ★✦✩✣✪ ✙✔✚✕✦✗✯✚✙✩✔✧ ✖✙✥✯✣✥✥ ✙✩✔ ✗✔ ✖ ✯✩ ✤✤✗✛ ✩✦ ✗✚✙✩✔✧ ✱ ✩✣✦✔ ✗✤✥ ✩✦ ✖✙✗✦ ✙✕✥✧ ✲✩ ✩✦ ✗✤✘✩✥✚ ✔✕★✤✙★✙✛✤✕ ✤✤ ✙ ✓✙ ✖ ✘ ✗✔ ✖ ✖✙✥✯✣✥✥ ✙✩✔ ✛✩ ✗✦✖ ✥✧ ✵ ✔✩✦ ✘✩✣✥ ✯✩ ✤✤✕✯✚✙✢✕ ✓✙✥✖✩ ✘ ✗ ✖ ✙ ✤✤✙ ✥ ✩ ✧ ✯✩ ✕✯✚ ✢✕ ✔ ✩✪✕✔ ✥✩✣✦✯✕ ✔✚✕ ★✕✔✯✕✧ Table 1: Comparing Individual and Community Blog Sites. Blog sites can be categorized into individual blog sites or single-authored blog sites and community blog sites or multi-authored blog sites. Individual blog sites are the ones owned and maintained by an individual. Examples of individual blogs could be Sifry’s Alerts: David Sifry’s musings8 (Founder & CEO, Technorati), Ratcliffe Blog–Mitch’s Open Notebook9 , The Webquarters10 etc. On the other hand, community blog sites are owned and maintained by a group of like-minded users. Examples of community blogs could be Google’s Official Blog site11 , The Unofficial Apple Weblog12 , Engadget13 , Boing Boing: A Directory of Wonderful Things14 etc. We summarize the differences between individual and community blogs in Table 1. Such an interactive information delivery medium like blogs hosts a conducive ground for the virtual communities or communities that originate over the Internet. There has been a lot of ongoing research to mine knowledge in Blogosphere. This survey is organized as follows: Section 2 introduces various issues pertinent to the blogosphere. Section 3 reviews tools, general methodologies, datasets, and performance metrics that are useful for conducting research in Blogosphere. Section 4 presents a case study. Section 5 discusses the connection between Blogosphere and state of the art social networks. Section 6 concludes the paper with some possible future directions for research in the blogosphere. 2. RESEARCH ISSUES Here we study various research issues and challenges with potential applications. We discuss the research issues in terms of modeling, clustering, mining, community discovery and factorization, influence and propagation, trust and reputation, and spam blog filtering. 2.1 Modeling the Blogosphere The first and foremost challenge lies in developing an appropriate model for the blogosphere. Often researchers and practitioners ask, which is the model that best describes the structure and properties of the blogosphere. Such a model can help in gaining deeper insights into the relationships between bloggers, commenters, blog posts, comments, viewers/readers, and different blog sites in the blogosphere. This 8 http://www.sifry.com/alerts/ http://www.ratcliffeblog.com/ 10 http://webquarters.blogspot.com/ 11 http://googleblog.blogspot.com/ 12 http://www.tuaw.com/ 13 http://www.engadget.com/ 14 http://boingboing.net/ 9 Volume 10, Issue 1 Page 19 can help us in understanding and defining various concepts of the blogosphere at an abstract level. These type of models would also help in tackling several other challenges of the blogosphere. A model for the blogosphere would be useful in generating an artificial dataset, tuning the parameters to simulate a special scenario and compare different algorithms and studies. Such a model will also help in studying peculiarities in the blogosphere and infer latent patterns and structures that could explain certain phenomena like community discovery, spam blogs, information diffusion and influence, etc., to be discussed later in this section. Modeling the blogosphere is often associated with modeling the web. Researchers represent the web as a webgraph, where each webpage forms a node and hyperlinks between them as edges. This kind of representation results in a directed cyclic graph. Weights can be associated with these edges. Such a model that converts the web into a graphic model is extensively exploited. One prominent example is the search engine domain which relies on this graph based model of the web to rank webpages [10; 32]. Although the web models seem to be an appropriate choice for modeling the blogosphere but certain key differences prevent reusing the web models in the blogosphere domain. First, models developed for the web assumes a dense graph structure due to a large number of interconnecting hyperlinks within webpages. This assumption does not hold true in the blogosphere, since the hyperlink structure in the blogosphere is very sparse, as shown in [35]. Second, the level of interaction in terms of comments and replies to a blog post makes the blogosphere different from the web. Third, the highly dynamic and “short-lived” nature of the blog posts could not be simulated by the web models. Web models do not consider this dynamicity in the web pages. They assume web pages accumulate links over time. However, in a blog network, where blog posts are the nodes, it is impractical to construct a static graph like the one for the web. These differences necessitate the need for a model more towards the characteristics of the blogosphere. There are several models for the web like random graph [49], preferential attachment graph [6], hybrid graph [44], and random walk on graph [9]. A random graph constructs edges between each pair of nodes with some probability which fails to exhibit the power law degree distribution or scale-free graph structure. For this reason random graph models cannot be used to model the blogosphere. Preferential attachment graph models follow the phenomenon of “the rich gets richer”, where the probability of a new edge to a node to be added is based on its degree. The more the degree of a node the better the chances are for a different node to be connected with this node. These models exhibit the power law distribution. Hybrid graph models are basically a mixture of both random graphs and preferential attachment models, so as to give a lucky “poor” a chance to get “rich”. Blogosphere can be modeled using this model with some modifications. To solve the problem of irreducibility (strong connectedness with few isolated subgraphs), random walk on a graph model proposes a random jump with a fixed probability between 0.8 and 0.9 in addition to the preferential attachment model. The above models have been used to model the blogosphere with modifications, but these models could not explain the blogosphere precisely. This has motivated researchers to come up with models specific to the blogosphere. Leskovec et al. [38] studied the temporal patterns of the blogosphere SIGKDD Explorations like how often people create blog posts, burstiness and popularity, how these blog posts are linked, and what is the link density. They reported that these phenomena follow power law distributions. Based on their findings, they developed a cascade model similar to the SIS (susceptible-infectedsusceptible) model from the epidemiology. This way any randomly picked blog can infect its uninfected immediate neighbors probabilistically, which repeats the same process until no node remains uninfected. In the end, this gives a blog network. Kumar et al. [37] use the blogrolls given on a blog post to create a network of connected posts with the underlying assumption that blogrolls have links to related or similar blog posts. A lot of research has been conducted that posits a known network structure of the blogosphere to model the problem domain. Such models are specific to problem domains and are discussed next in reference to problem domains. 2.2 Blog Clustering Blogosphere is a storehouse of several publicly regulated media. Technorati15 reported that 175,000 blog posts were created daily which is 2 blog posts per second. This explosive growth makes it beyond human capabilities to look for interesting and relevant blog posts. Therefore a lot of research is going on to automatically cluster different blogs into meaningful groups such that readers can focus on interesting categories, rather than filtering out relevant blogs from the jungle. Often blog sites allow their users to provide tags to the blog posts. The human labeled tag information forms the so-called “folksonomy”. Brooks and Montanez [11] presented a study where the human labeled tags are good for classifying the blog posts into broad categories while they were less effective in indicating the particular content of a blog post. They used the tf-idf measure to pick the top three most famous words in every blog post and computed the pairwise similarity among all the blog posts and clustered them. They compared the results with the clustering obtained using the human labeled tags and reported significant improvement. In another research [39], authors tried to cluster blog posts by assigning different weights to title, body and comments of a blog post. However, these approaches rely on the keyword-based clustering which suffers from high-dimensionality and sparsity. Agarwal et al. [2] proposed WisClus that uses the collective wisdom of the bloggers to cluster the blogs. They have used the blog categories and construct the category relation graph to merge different categories and cluster the blogs that belong to these categories. Edges in the category relation graph represent the similarity between different categories which are the nodes in this graph. The similarity between two categories is computed using the number of blogs that simultaneously uses these categories as their blog labels. Experiments show that the collective wisdom based clustering performs better than keyword based clustering even after reducing the dimensionality and sparsity to the concept space using Latent Semantic Indexing (LSI) [14]. Clustering different blog posts would also help blog search engines like Technorati to narrow down the search space once the query context is clear. Websites like Blogcatalog16 organize blogs into a taxonomy that helps in focussed browsing of blogs. 15 16 http://www.technorati.com/ http://www.blogcatalog.com/directory Volume 10, Issue 1 Page 20 2.3 Blog Mining Blog mining as a technique is evolving and taking the form of qualitative research. Companies are using blogs as qualitative research tools. Historically, the interaction between marketers and consumers has been a closed loop. Marketers used to send out messages to consumers and sought their feedback through traditional research. Now, consumers can not only speak their mind but also broadcast their opinions. The surge of marketing messages combined with low consumer trust, has led to people relying on one another’s opinions to make informed decisions, prompting conversations between them. These interactions are found on the blogs and have attracted the attention of several companies. Blogs are immensely valuable resources to track consumers’ beliefs and opinions, initial reaction to a launch, understand consumer language, track trends and buzzwords, fine tune information needs. Blog conversations leave behind the trails of links, useful for understanding how information flows and how opinions are shaped and influenced. Tracking blogs also help in gaining deeper insights as bloggers share their views from various perspectives hence giving a ’context’ to the information collected. Mining sentiments from free text forms poses several challenges as compared to the historic feedback and surveys. A prototype system called Pulse [19] uses a Naı̈ve Bayes classifier trained on manually annotated sentences with positive/negative sentiments and iterates until all unlabeled data is adequately classified. Another system presented in [5] improves the blog retrieval by using opinionated words acquired from WordNet in the query proximity. Some wellknown opinion mining and sentiment analysis techniques [41] could also be borrowed from text mining domain due to high textual nature of blogs. 2.4 Community Discovery and Factorization Another important research which branched out from the blog-site clustering is determining and inferring communities. Several studies looked into identifying communities in Blogosphere. One method that researchers commonly use is content analysis and text analysis of the blog posts to identify communities in the blogosphere [7], [16], [37]. Kleinberg [32] used an alternative approach in identifying communities in web using a hub and authority based approach, clustering all the expert communities together by identifying them as authorities. Kumar et al. [36] extended the idea of hubs and authorities and included co-citations as a way to extract all communities on the web and used graph theoretic algorithms to identify all instances of graph structures that reflect community characteristics. While Chin and Chignell [12] proposed a model for finding communities taking the blogging behavior of bloggers into account, they aligned behavioral approaches in studying community with the network and link analysis approaches. They used a case study to first calibrate the measure to evaluate a community based on behavioral aspects using a behavioral survey which could be generalized later on, pruning the need of such surveys. Several researchers have also studied community extraction and social network formation using newsgroups and discussion boards. Although different from the blogosphere we include these here because discussion boards and newsgroups are also very similar to blogs in the sense that they do not have an explicit link structure, and the communication is SIGKDD Explorations not “person-to-person”, rather it is more “person-to-group”. Blanchard and Markus [8] studied “Virtual Settlement” - a Multiple Sport Newsgroup and analyzed the possibility of emerging virtual communities in it. They studied the characteristics of the newsgroup by conducting interviews with three different kinds of members: leaders (active and well respected), participants (active occasionally to events like triathlons) and lurkers (readers only). They reported that different virtual communities emerge between athletes and those who join the community to keep themselves informed of the latest developments. 2.5 Influence in Blogs and Propagation As communities evolve over time, so do the bellwethers or leaders of the communities who possess the power to influence the mainstream. According to the studies in [30], 83% people prefer consulting family, friends or an expert over traditional advertising before trying a new restaurant, 71% people prefer to do so before buying a prescription drug or visiting a place, 61% of people prefer to do so before watching a movie. This style of marketing is known as “wordof-mouth”. “Word-of-mouth” has been found to be more effective than the traditional advertising in physical communities. Studies from [30] show that before people buy, they talk, and they listen. Experts can influence decisions of people. For this reason these experts are aptly termed as the Influentials. Influential bloggers tend to submit influential blog posts that affect other members’ decisions and opinions. They accrue respect in the community over time. Other members tend to listen to what the influentials say before making decisions. Identification of these influential bloggers [4] could lead to several interesting applications. The influentials are potential market-movers. Since they can influence buying decisions of mainstream, companies can promote them as latent brand ambassadors for their products. Being such a highly interactive medium, blogs tend to host several vivid discussions on various issues including new products, services, marketing strategies and their comparative studies. Often this discussion also acts as “word-of-mouth” advertising of several products and services. A lot of advertising companies, approximately 64% [17] have acknowledged this fact and are shifting their focus towards blog advertising and identifying these influentials. The influentials could sway opinions in political campaigns, elections and reactions to government policies [15]. Because they know many people and soak up a large amount of information, the influentials stand out as knowledgeable, informed sources of advice and insight. Approximately, 84% of the influentials in physical communities are interested in politics and are sought out by others for their perspectives on politics and government, 55% on a regular basis. The influentials could help in customer support and troubleshooting. A lot of companies these days host their own customer blogs, where people could discuss issues related to a product. Often the influentials on these blogs troubleshoot the problems peer consumers are having, which could be trusted because of the sense of authority these influentials possess. Often the influentials offer suggestions to improve their products. These invaluable comments could be really helpful for companies and customers. Instead of going through each member’s blog posts, companies can focus Volume 10, Issue 1 Page 21 on the influentials’ blog posts. For instance, Macromedia17 aggregates, categorizes and searches the blog posts of 500 people who write about Macromedia’s technology. Some recent numbers from Technorati show a 100% increase in the size of the blogosphere every six months. It has grown over 60 times during the past three years. Approximately 2 new blog posts appear every second18 . New blog posts being generated with such a blazing fast rate, it is impossible to keep track of what is going on in the blogosphere. Many blog readers/subscribers just want to know the most insightful and authoritative stories before delving into the discussions. Blog posts from influential bloggers would exactly serve this purpose by standing out as representative articles of a blog site. The influentials can be the showcases of a group on the blogosphere. These interesting applications have attracted a surge of research in identifying influential blog sites as well as influential bloggers. Some try to find influential blog sites, in the entire blogosphere and study how they influence the external world and within the blogosphere [20]. The problem of ranking blog sites or bloggers differs from that of finding authoritative webpages. As pointed out in [35], blog sites in the blogosphere are very sparsely linked and it is not suitable to rank blog sites using Web ranking algorithms like PageRank [43] and HITS [32]. The Random Surfer model of webpage ranking algorithms [43] does not work well for sparsely linked structures. The temporal aspect is most significant in the blog domain. While a webpage may acquire authority over time (its adjacency matrix gets denser), a blog post or a blogger’s influence diminishes over time. Consequently, the adjacency matrix of blogs (considered as a graph) will get sparser as thousands of new sparsely-linked blog posts appear every day. Some recent work [35] suggests to add implicit links to increase the density of link information based on topics. If two blogs are talking about the same topic, an edge can be added between these two blogs based on the topic similarity or information epidemics. However, constructing links based on the topic models still remains an area of research. A similar strategy adopted by Adar et al. [1] is to consider the implicit link structure of blog posts. In their iRank algorithm, a classifier is built to predict whether or not two blogs should be linked. The objective in this work is to find out the path of infection (how one piece of information is propagated). iRank tries to find the blogs which initiates the epidemics. Note that an initiator might not be an influential as they might affect only limited blogs. Influentials should be those which play a key role in the information epidemics. Gruhl et al [24] study information diffusion of various topics in the blogosphere between different blog sites, drawing on the theory of infectious diseases. A general cascade model [23] is adopted. They derived their model from independent cascade model and generalized it to the general cascade model by relaxing the independence assumption. They associate ‘read’ probability and ‘copy’ probability with each edge of the blog graph indicating the tendency of a blog to be read and copied, respectively. They also parameterize the stickiness of a topic which is analogous to the virulence of a disease. An interesting problem related to viral market17 18 http://weblogs.macromedia.com/ http://www.sifry.com/alerts/archives/000436.html SIGKDD Explorations ing [46; 31] is how to maximize the total influence among the nodes (blog sites) by selecting a fixed number of nodes in the network. A greedy approach can be adopted to select the most influential node in each iteration after removing the selected nodes. This greedy approach outperforms PageRank, HITS and ranking by number of citations, and is robust in filtering splogs (spam blogs) [28]. Finding influential blog sites is perpendicular to the problem of identifying influential bloggers. Given the nature of the blogosphere, influential blog sites are few. A large number of non-influential sites belong to the long tail [3] where abundant new business, marketing, and development opportunities can be explored. Agarwal et al. [4] studied and modeled the influence of a blogger on a community blog site regardless of the site being influential or not. They modeled the blog site as a graph using inherent link structure, including inlinks and outlinks, as edges and treating different bloggers as nodes. Using the link structure the influence flow across different bloggers is observed, recursively. Other blog post level statistics like blog post quality and comments’ information were also used to achieve better results. The model used different weights to regulate the contribution of different statistics. These weights could be tuned to obtain different breeds of influential bloggers. Influential bloggers are not necessarily active bloggers at a blog site [4]. Many blog websites list top bloggers or top blog posts in some time frame (e.g., monthly). Those top lists are usually based on some traffic information (e.g., how many posts a blogger posted, or how many comments a blog post received) [20]. With the speedy growth of the blogosphere, it is increasingly difficult, if at all possible, to manually track the development and happenings in the blogosphere, in particular, at many blog sites where many bloggers enthusiastically participate in discussions, getting information, inquiring and seeking answers, and voicing their complaints and needs. 2.6 Trust and Reputation Open standards and low barrier to publishing has allowed anyone to submit blog posts and contribute to the participatory journalism. On one hand, it has created an overwhelming amount of collective wisdom; on the other hand, it has made difficult for readers to decide whom to trust or believe. This has been a great challenge since the inception of the World Wide Web which created the problem of authoritative webpages. Kleinberg [32] and Page et al [43] tried to give a solution for this problem by exploiting the link structure of webpages. But social networking sites and especially Blogosphere allow mass to create and edit content compromising (risking) the sanctity of the original content. Researchers anticipated this problem in social networking and recommender systems and conducted research in those areas. However, the potential of this research is still underestimated for the blogosphere domain and not much research is reported. Here we briefly point out the work already done in social networks to provide an insight to this problem and mention the current state of trust related research in Blogosphere. In social networks it is important not only to detect the influential members or experts in case of knowledge sharing in communities but also to assess to what extent some of the members are recognized as experts by their colleagues in the community. This leads to the estimation of trust and reputation of these experts. Some social friendship networks Volume 10, Issue 1 Page 22 like Orkut19 allow users to assign trust ratings implying a more explicit notion of trust. Whereas some websites have an implicit notion of trust where creating a link to a person on a webpage implies some amount of business trust for the person. In other cases, trust and reputation of experts could be typically assessed as a function of the quality of their response to other members’ knowledge solicitations. Pujol et al [45] proposed a NodeMatching algorithm to compute the authority or reputation of a node based on its location in the social friendship network. A node’s authority depends upon the authority of the nodes that refer to this node and also on the authority of other nodes that this node refers to. The basic idea is to propagate the reputation of nodes in the social friendship network. While Pujol et al. [45] proposed an approach to establish reputation based on the position of each member in the social friendship network, the authors of [50] developed a model for reputation management based on the DampsterShafer theory of evidence in the wake of spurious testimonies provided by malicious members of the social friendship network. Each member of a social friendship network is an agent. Each agent has a set of acquaintances a subset of which forms its neighbors. Each agent builds a model for its acquaintances to quantify their expertise and sociability. These models are dynamic and change based on the agent’s direct interactions with the given acquaintance, interactions with agents referred to by the acquaintance, and on the ratings this acquaintance received from other agents. The authors point out a significant problem with this approach which arises if some acquaintances or other agents generate spurious ratings or exaggerate positive or negative ratings, or offer testimonies that are outright false. Sabater and Sierra [47] propose a combination of reputation scores on three different dimensions. They combined reputation scores not only through social relations governed by a social friendship network (termed as social dimension) but also past experiences based on individual interactions (termed as individual dimension) and reputation scores based on other dimensions (termed as ontological dimension). For large social networks it is not always possible to get reputation scores based on just the individual dimension, so they can use the social dimension and ontological dimension that would enhance the reputation estimation by considering different contexts. The ontological dimension is very similar to the work proposed in [48], where the authors recommend collaboration in social networks based on several factors. They explain the importance of context in recommending a member of social network for collaboration. In [22], authors consider those social networking sites where users explicitly provide trust ratings to other members. However, for large social networks it is infeasible to assign trust ratings to each and every member so they propose an inferring mechanism which would assign binary trust ratings (trustworthy/non-trustworthy) to those who have not been assigned one. They demonstrate the use of these trust values in an email filtering application and report encouraging results. Authors also assume three crucial properties of trust for their approach to work: transitivity, asymmetry, and personalization. These trust scores are often transitive, meaning, if Alice trusts Bob and Bob trusts Charles then Alice can trust Charles. Asymmetry says that for two people 19 http://www.orkut.com SIGKDD Explorations involved in a relationship, trust is not necessarily identical in both directions. This is contrary to what was proposed in [50], who assume symmetric trust values in the social network between two members. Also, consolidating the trust scores for a member and computing a global trust score for each member might not give a reasonable estimation. Trust of a member is absolutely a personal opinion. Therefore, authors propose personalization of trust which means that a member could have different trust values with respect to different members. Guha et al [25] proposed another trust propagation scheme in social friendship networks based on a series of matrix operations, including the element of distrust along with the trust scores. Although there has been a lot of work that deals with trust in social networks and recommender systems, not many have considered trust in the blogosphere. Researchers have tried to transform the blogosphere domain to the problem domain considered in trust in social networks. Authors in [29] considers a window of words around the links in a blog post to mine the sentiments about the cited blog post. Using VoteLinks, these sentiments can be classified as positive, negative or neutral sentiments. These bags of sentiments can then be used to compute the link polarity between a pair of blog posts. Using Gruhl’s et al [25] trust propagation model, they compute the trust in the network of blog sites in the blogosphere. There is still a lot of information unexploited in this approach like comments from the readers on the blog post that can also be used to judge a blogger’s or a blog post’s trust. 2.7 Filtering Spam Blogs Spam blogs, often called splogs, is one of the major concerns in the blogosphere. Besides degrading search quality results it also wastes the network resources. So researchers are looking into this aspect of the blogosphere. Although it is a relatively new phenomenon, researchers have compared it with the existing work on web (link) spam detection. For web spam detection, authors in [42] distinguish between normal web pages and spam webpages based on the statistical properties like, number of words, average length of words, anchor text, title keyword frequency, tokenized URL. Some works [26; 27] also use PageRank to compute the spam score of a webpage. Some researchers consider splogs as a special case web spam. Authors in [33; 34] consider each blog post as a static webpage and use both content and hyperlinks to classify a blog post as spam using a SVM based classifier. However, there are some critical differences between web spam detection and splog detection. The content on blog sites is very dynamic as compared to that of web pages, so content based spam filters are ineffective. Moreover, spammers can copy the content from some regular blog posts to evade content based spam filters. Link based spam filters can easily be beaten by creating links pointing to the splogs. Authors in [40] consider the temporal dynamics of blog posts and propose a self similarity based splog detection algorithm based on characteristic patterns found in splogs like, regularities or patterns in posting times of splogs, content similarity in splogs, and similar links in splogs. 3. TOOLS AND OTHER RELATED ISSUES Having presented the status quo of the ongoing research in Blogosphere, we now discuss available tools to analyze the domain, methodologies across different disciplines, data Volume 10, Issue 1 Page 23 collection and pre-processing, and performance metrics. 3.1 Tools and APIs Several modeling tools are available to simulate the social networks that help study various characteristics of these networks and conduct experiments, including: • NetLogo20 : A multi-agent programming language and modeling environment designed in Logo programming language. Modelers can give instructions to hundreds or thousands of concurrently operating autonomous “agents”. This helps in exploring the connection between the individuals (micro-level) and the patterns that emerge from the interaction of many individuals (macro-level). • StarLogo21 : An extension of Logo programming language. It is used to model the behavior of decentralized systems like social networks. • Repast22 : Recursive Porous Agent Simulation Toolkit is an agent-based social network modeling toolkit. It has libraries for genetic algorithms, neural networks, etc. and allows users to dynamically access and modify agents at run time. • SocNetV29 : A Linux based SNA and visualizing utility. SocNetV can compute network and actor properties, such as distances, centralities, diameter etc. Furthermore, it can create simple random networks (lattice, same degree, etc.). Besides simulation and modeling toolkits there are APIs from Facebook, StumbleUpon, Technorati, del.icio.us, Digg, etc. These APIs could be used to download real-world data and study properties of social networks and concepts such as small worlds, random networks, scale-free networks, laws and distributions (normal distribution, Zipf’s law, power law), search in networks, computation/propagation of influence and trust, diffusion (epidemics), robustness in networks, collective wisdom, collaborative filtering, social decision making, social criminals, individual profiling and privacy, story construction, provenance, and the unique characteristics of Long Tail blogs/blog sites and Short Head blogs/blog sites. 3.2 Methodologies We now discuss the broad technical concepts that form the necessary background in conducting research in social network domain through centrality measures, network models, content analysis, link analysis, supervised learning, decision theoretic approach, and agent-based modeling. • Swarm23 : A multi-agent simulation package to simuNetwork centrality measures form an essential part of late the social or biological interaction of agents and social network analysis. Social network analysis is used to their emergent collective behavior. identify leaders, mavens, brokers, groups, connectors (bridges between groups), mavericks, etc. Researchers have used sev• UCINet24 : A comprehensive package for the analyeral centrality measures to gauge the information flow across sis of social network data including centrality meaa social network which could help in identifying different sures, subgroup identification, role analysis, elemenroles of nodes mentioned above. Centrality measures help tary graph theory, and permutation-based statistical in studying the structural attributes of nodes in a network. analysis. In addition, the package has strong matrix They help in studying the structural location of a node in analysis routines, such as matrix algebra and multithe network which could decide the importance, influence or variate statistics. prominence of a node in the network. Centrality measures 25 • Pajek : (Slovenian: spider) A software for analyzing help in estimating the extent to which the network revolves and visualizing large networks like social networks. around a node. Different centrality measures include Degree centrality, Closeness centrality, Betweenness centrality, and 26 • Network package in “R” : The network class can repEigenvector centrality. Degree centrality refers to the total resent a range of relational data types, and support number of connections or ties a node has in the network. arbitrary vertex/edge/graph attributes. This is used This could be imagined as a “hubness” value of that node. to create and/or modify the network objects and is Rows or column sums of an adjacency matrix would give the used for social network analysis (SNA). degree centrality for that node. Closeness centrality is de27 fined by the sum of all the geodesic distance of a node with • InFlow : Another integrated product for network analall other nodes in the network. This could be imagined as ysis and visualization. It has been used in the SNA the “nearest” node to the other nodes in the network. Bedomain. tweenness centrality refers to the extent a node is directly connected to nodes that are not directly connected, or the • NetMiner28 : A tool for exploratory network data analnumber of geodesic paths that pass through this node. This ysis and visualization. NetMiner allows to explore netevaluates how well a node can act as a “bridge” or intermework data visually and interactively, and helps in dediary between different subgraphs. A high betweenness centecting underlying patterns and structures of the nettrality node can become a “broker” between different subwork. graphs. Eigenvector centrality defines a node to be central 20 http://ccl.northwestern.edu/netlogo/ if it is connected to those who are central. This could be 21 http://education.mit.edu/starlogo/ gauged as the “authoritativeness” of a node. It is the prin22 http://repast.sourceforge.net/ cipal eigenvector of the adjacency matrix of the network. 23 http://www.swarm.org/wiki/Main Page Other SNA measures used for analyzing social networks 24 http://www.analytictech.com/ are clustering coefficient (the likelihood that associates of 25 http://vlado.fmf.uni-lj.si/pub/networks/pajek/ a nodes are associates among themselves to ensure greater 26 http://cran.r-project.org/src/contrib/Descriptions/network.htmlcliquishness), cohesion (extent to which the actors are con27 http://www.orgnet.com/inflow3.html 28 29 http://www.netminer.com/ http://socnetv.sourceforge.net/ SIGKDD Explorations Volume 10, Issue 1 Page 24 nected directly to each other), density (proportion of ties of a node to the total number of ties this node’s friends have), radiality (extent to which an individual’s network reaches out into the network and provides novel information), and reach (extent to which any member of a network can reach other members of the network). Useful concepts in modeling social networks demand a lucid understanding of various network models such as scale-free, random [49], preferential attachment [6], hybrid [44], cascade models, etc. The link structure could be modeled using the scale free power law distribution (P (k) ∝ k−γ ). There are two generic aspects of real networks (e.g., Social networks, Blog networks, World Wide Web, biological networks, etc.) that make scale-free power law models an appropriate choice as compared to random models. First, the number of nodes (N ) in the real networks is not static. N increases throughout the lifetime of the network and the new nodes attach to the vertices already present in the network. Second, the random network models assume that the probability that two vertices are connected is random and uniform. However, most real networks exhibit preferential connectivity. For example, a newly created webpage will be more likely to include links to well-known popular documents with already high connectivity. Thus the probability with which a new vertex connects to the existing vertices is not uniform; there is a higher probability that it will be linked to a vertex that already has a large number of connections. This property of scale free power law models is also known as preferential attachment models. Some works [44] have shown the relative importance of hybrid models in simulating social networks by determining the appropriate proportion of random and scale free networks. Information flow across the network could be studied with the help of cascade models. Information diffusion could be considered analogous to the spread of a viral disease. Models from epidemiology have been borrowed and studied to model diffusion aspect in social networks. The key is to exploit different properties of scale-free, random, preferential attachment, hybrid models, cascade models to efficiently and effectively model the social networks. Blogs have rich textual content. Not only people create new content, they also enrich the existing content by providing meta data such as labels and tags. These human-generated tags are also called “folksonomies”. State-of-the-art content analysis techniques could be used for basic clustering, classification of the blog posts/blog sites. Traditional text analysis approaches like tf-idf could be used for indexing the blog entries. Folksonomies could be considered as class labels and supervised machine learning could be performed, classification models could be learned on labeled dataset, and learned models could be used to predict the tags of unlabeled corpus. This forms an essential concept for semiautomatically generating “tag-clouds” with least human intervention. Link analysis helps in understanding several interesting phenomena of social networks. Text around the links give us knowledge about the linked blog posts. Based on the links, hubs and authorities could be discovered. This could be achieved exactly the same way as it is done for webpages. This approach could lead to the identification of expert communities. Several researchers have also pointed out the sparsity in the link structure of social networks which makes it different from the World Wide Web model. Many of them SIGKDD Explorations like Blogosphere assume implicit link information among bloggers. Links could be constructed using the topic analysis. For example, blog posts talking about same topic could be connected. Supervised learning algorithms could be used to predict topics of unlabeled blog posts, which helps achieve link construction. Several studies have been conducted to study decision theoretic approach for group-individual interaction and the effect of decision on an individual and/or a community as a whole. Decision theory studies what is the best possible decision to take given a fully informed decision maker. In the context of social networks this could be applied in finding the node in the network that is the best to make decisions with least possible side-effects and maximum possible gains for the rest of the nodes. This is a classic subject of study in microeconomics. Some decisions are difficult to make because of the need to reach a consensus among other members and the uncertainty in the response of different individuals. The analysis of such social decisions is dealt through game theory. Social networks are also studied from the perspective of agent-based modeling. Basically each node in a social network can be treated as an agent. This agent could be a blogger in the blogosphere domain. Then assuming the network follows some distribution, usually a scale-free model, we can model the decision making ability of the agent probabilistically. This can help us in studying the factors that affect his/her blogging behavior, what and how (s)he makes decisions, etc. Neural networks or genetic algorithms could also be used to train the model of these agents to closely simulate some real-world scenario, which means, iteratively tuning the model parameters and keep improving the model. 3.3 Data Collection Data collection is an essential part of studying and evaluating concepts in empirical research. Since social networking is a socio-psychological phenomenon and is more prevalent in the real world than the theoretical study, so enormous amounts of data exist on actual social networking websites. Moreover, since this involves user information, it is sensitive when the data is used in open research. Much of such data is unavailable due to privacy concerns. A few available datasets are: • Nielsen Buzzmetrics dataset: The dataset consists of about 14M blog posts from 3M blog sites collected by Nielsen BuzzMetrics30 in May 2006. The data is annotated with 1.7M blog-blog links. However, up to a half of the blog outlinks are missing. Only 51% of the total blog posts are in English. • Enron Email dataset: It contains data from about 150 users, mostly senior management of Enron. The corpus31 contains a total of about 0.5M messages. People have studied the social networks between users based on link construction. Links are constructed based on email senders and recipients. • APIs: APIs provided by Facebook, Digg, StumbleUpon, Technorati, del.icio.us etc. can be used to download data from a corresponding social networking website. Nevertheless, the API usage is often restricted 30 31 http://www.nielsenbuzzmetrics.com/ http://www.cs.cmu.edu/∼enron/ Volume 10, Issue 1 Page 25 of a post’s novelty. A large number of outlinks (θ) may suggest that a post refers to many other blog posts or articles, indicating that it is less likely to be novel. Correlation experiments in [4] have reported that the number of outlinks is negatively correlated with the number of comments which means more outlinks reduces people’s attention. to either last 30 days or top 100 results or in case of friendship networks like Facebook, one can only download data of his/her network of friends. There is another more challenging yet appropriate option to obtain datasets. People can write crawlers and parsers to download data from blog sites. These custom datasets can be downloaded and pre-processed to serve more specific needs. We discuss more in Section 4. • Eloquence - An influential is often eloquent [30]. This property is most difficult to approximate using some statistics. Given the informal nature of the blogosphere, there is no incentive for a blogger to write a lengthy piece that bores the readers. Hence, a long post often suggests some necessity of doing so. Therefore, we use the length of a post (λ) as a heuristic measure for checking if a post is influential or not. Correlation experiments in [4] have reported that the blog post length is positively correlated with number of comments which means longer blog posts attract people’s attention. 3.4 Experiments and Performance Metrics The fact that many concepts like, influence, trust, information propagation, identification of information routers, brokers, etc. in social network domain like Blogosphere are socio-psychological and highly subjective in nature, setting up experiments and evaluating the results is non-trivial. The absence of ground truth makes it even harder to compare different approaches available in the spectrum. Lack of ground truth makes an option to use search engines’ ranking algorithms as the baseline for most of the existing works, even though theoretically it has been proven that current search engines are not suited for social network data and link structure. Recent work like [4] have used another Web 2.0 application, i.e., Digg, to evaluate the influence in the blogosphere. Results and discussion are included in Section 4. 4. A CASE STUDY Here we present a study of identifying influential bloggers in a community [4]. We discuss model development, data collection and model tuning and verification through experiments. 4.1 Model Development Assuming the domain to be community or multi-authored blogs, the influential bloggers are defined as: A blogger can be influential if s/he has more than one influential blog post. The model assigns an influence score to each blog post of the blogger. These blog post level influence scores are used to calculate the influence of the blogger. An initial set of intuitive properties is proposed in [4] to approximately represent influential blog posts. • Recognition - An influential blog post is recognized by many. This can be equated to the case that an influential post p is referenced in many other posts, or its number of inlinks (ι) is large. The influence of those posts that refer to p can have different impact: the more influential the referring posts are, the more influential the referred post becomes. • Activity Generation - A blog post’s capability of generating activity can be indirectly measured by how many comments it receives, the amount of discussion it initiates. In other words, few or no comment suggests little interest of fellow bloggers, thus non-influential. Hence, a large number of comments (γ) indicates that the post affects many such that they care to write comments, and therefore, the post can be influential. There are increasing concerns over spam comments that do not add any value to the blog posts or blogger’s influence. Fighting spam is outside the scope of this work and recent research can be found in [33; 40]. • Novelty - Novel ideas exert more influence as suggested in [30]. Hence, the number of outlinks is an indicator SIGKDD Explorations 4.2 Data Collection Data collection is one of the critical tasks in this work. Since there are no available blog data sets for the purposes of our experiments, we need to collect real-world data. There exist many blog sites. Some like Google’s Official Blog site act as a notice board for important announcements rather than for discussions, sharing opinions, ideas and thoughts; some do not provide most of the statistics needed in our work, although they can be obtained via some additional work (more explanation later). A few publicly available blog datasets like the BuzzMetric dataset32 were designed for different research experiments so there is no way to obtain some key statistics required in this work. Therefore, we crawled a real-world blog site that provides the most statistics required in our experiments. The advantages of doing so include (1) minimizing our effort on figuring out ways to obtain the needed statistics, and (2) maximizing the reproducibility of our experiments independently. The Unofficial Apple Weblog (TUAW) site is such a site that satisfies these requirements. This blog site provides most needed information like blogger identification, date and time of posting, number of comments, and outlinks. The only missing piece of information at TUAW is the inlinks information, which we can obtain using Technorati API33 . We crawled the TUAW blog site and retrieved all the blog posts published since it was set up. We have collected over 10, 000 posts so far34 . We keep the complete history of the TUAW blog site and update it incrementally. All the statistics obtained after crawling is stored in a relational database for fast retrieval later35 . 4.3 Verification Many blog sites publish a list of top bloggers based on their activities on the blog site. The ranking is often made according to the number of blog posts each blogger submitted over a period of time. Using the number of posts of a blogger 32 http://www.nielsenbuzzmetrics.com/cgm.asp http://technorati.com/developers/api/cosmos.html 34 January 31, 2007. 35 This dataset will be made available upon request for research purposes. 33 Volume 10, Issue 1 Page 26 Top 5 TUAW Bloggers Erica Sadun Scott McNulty Mat Lu David Chartier Michael Rose Top 5 Influential Bloggers Erica Sadun Dan Lurie David Chartier Scott McNulty Laurie A. Duncan Table 2: Two lists of the top 5 bloggers according to TUAW and proposed model, respectively. Bloggers Influential Non-influential Active S1: 17 S3: 3 Inactive S2: 7 S4: 0/1 Table 4: Overlap between Top 20 blog posts at Digg and our model for last 6 months for different configurations. Table 3: Intersection of Digg and top 20 from our model. posted is obviously an oversimplified indicator, which basically says the most frequent blogger is an influential one. Such a status can be achieved by simply submitting many posts, as even junk posts are counted. Hence, an active blogger may not be an influential one; and in the same spirit, an influential blogger need not be an active one. In other words, the most active k bloggers are not necessarily the top influential one, and an inactive blogger can still be an influential one. Table 2 presents two lists of top 5 bloggers according to TUAW and based on the proposed model: the first column contains the top 5 bloggers published by TUAW and the second column lists the top 5 influential bloggers. Names in italics are the bloggers present in both lists. Three out of 5 TUAW top bloggers are also among the top 5 influential bloggers identified by our model. This set of bloggers suggests that some of the bloggers can be both active and influential. Some active bloggers are not influential and some influential bloggers are not active. For instance, ‘Mat Lu’ and ‘Michael Rose’ in the TUAW list, so they are active; and ‘Dan Lurie’ and ‘Laurie A. Duncan’ in the list of the influentials, but they are not active. In total, there could be four types of bloggers: both active and influential, active but non-influential, influential but inactive, inactive and non-influential. As we know, there is no training and testing data to evaluate the efficacy of the proposed model. The absence of ground truth about influential bloggers presents another challenge. The key issue is how to find a reasonable reference point for which four different types of bloggers can be evaluated so that we can observe their tangible differences. As an alternative to the ground truth, we resort to another Web2.0 site Digg36 to provide a reference point. According to Digg, “Digg is all about user powered content. Everything is submitted and voted on by the Digg community. Share, discover, bookmark, and promote stuff that’s important to you!”. As people read articles or blog posts, they can give their votes in the form of digg and these votes are recorded on Digg servers. This means, blog posts that appear on Digg are liked by their readers. The higher the digg score for a blog post is, the more it is liked. In a way, Digg can be considered as a large online user survey. Though only submitted blog posts are voted, Digg offers a way to evaluate 36 http://www.digg.com/ SIGKDD Explorations the blog posts of the four types. Digg provides an API to extract data from their database for a window of 30 days. Given the nature of Digg, a not-liked blog post will not be submitted thus will not appear in Digg. For January 2007, there were in total 535 blog posts submitted on TUAW. As Digg only returns top 100 voted posts, we use these 100 blog posts at Digg as our benchmark in evaluation. We take the four categories of bloggers, viz. 1. Active and Influential, 2. Inactive and Influential, 3. Active and Noninfluential, and 4. Inactive and Non-influential and categorize their posts into S1, S2, S3, and S4, respectively. We rank the blog posts of each category based on the influence score and pick top 20 blog posts from each of the first three categories. We randomly pick 20 blog posts from the last category in which bloggers are neither active nor influential. Next we compare these four sets of 20 blog posts with the Digg set of 100 blog posts to see how many posts in each set also appear in the Digg set. The results are shown in Table 3. From the table, we can see that S1 has 17 out of 20 in the Digg set, and S4 has 0 or 1 found in the Digg set depending on randomization. The results show the differences among the four categories of bloggers and our model identifies the influentials whose blog posts are more liked than others according to Digg. We also studied the contribution of different parameters and their relative importance. Since Digg assigns score to blog posts and not bloggers, the top influential blog posts from Digg37 are compared with those of our model. Different configurations of the proposed model was tried by considering 1. All-in i.e. all the four parameters, 2. No inlinks (outlinks, comments, and blog post length), 3. No comments (inlinks, outlinks, blog post length), 4. No outlinks (inlinks, comments, blog post length), and 5. No blog post length (inlinks, outlinks, comments). The overlap results for all these 5 configurations with digg were reported in Table 4. From the results in Table 4, it can be observed that configuration 2 (no inlinks) always performs the worst, configuration 3 (no comments) performs better, then comes configuration 4 (no outlinks) and then come configuration 5 (no blog post length). This gives the order of importance of all the four parameters, i.e. inlinks > comments > outlinks > blog post length, in the decreasing order of importance to influence estimation. For a blog site that has a reasonably long history, we can also study the temporal patterns of its influential bloggers. 37 This data was obtained using digg API. Volume 10, Issue 1 Page 27 The blog site TUAW provides blogging data since its inception February 2004. The proposed model is applied to identify top 5 influential bloggers with a moving 30-day window until January 2007, and there is no overlap between two consecutive windows. Details could be found in [4]. Based on the results obtained by studying the temporal patterns of influential bloggers we categorize these bloggers into one of the following: Long-term influentials They steadily maintain the status of being influential for a very long time. They can be considered “authority” in the community. Average-term influentials They maintain their influence status for 4-5 months. Transient influentials They are influential for a very short time period (only one or two months). Burgeoning influentials They are emerging as influential bloggers recently. They are the influentials worthy of more follow-up examinations. Detailed results, analysis, and experiments like weight and stability analysis, pairwise correlation study between different statistics, lesion study, and inclusion of other statistics could be found in [4]. 5. BLOGOSPHERE AND SOCIAL NETWORKS Social networks encompass services like the blogosphere, social friendship networks, media sharing (pictures e.g. Flickr, videos e.g. YouTube), collaborative annotation environment (del.icio.us), and wikis. Social Friendship Networks and Blogosphere share some commonalities like social collaboration, sense of community and experience sharing, yet there are some subtle differences. These nuances are worth pointing out here as they shed light on different ongoing research activities. There are some research areas that are specific to social friendship networks like collaborative recommendation, trust and reputation because they assume an explicit graph structure in the interaction among different members of the network. Similarly, some of the research areas are specific to blogosphere like blog post/blog site classification, spam blog identification because of the highly textual content nature of these articles. Unlike social friendship networks, the blogosphere does not have explicit links or edges between the nodes. These nodes could be friends in a social friendship network and bloggers in the blogosphere. We could still construct a graph structure in the blogosphere by assuming an edge from one blogger to another if a blogger has commented on other blogger’s blog post. This way we can represent the blogosphere with an equivalent directed graph. Social friendship networks already have predefined links or edges between the members in the form of a FOAF38 network, an undirected graph. This link/edge inference also poses major challenges in majority of research efforts going on in the blogosphere like identifying communities and influential bloggers. Although Adar et al. [1] proposed a model to infer links between different blog posts based on the propagation of the content in the blog 38 Friends of a friend SIGKDD Explorations posts, applicability of such techniques is limited if not much information epidemics is found. Another significant difference between social friendship networks and the blogosphere lies in the way influential members are perceived. Bloggers submit blog posts which are the main source of their influence. Influence score could be computed using blog posts through several measures like inlinks, outlinks, comments, and blog post length. This could give us an actual influential node based on the historical data of who influenced whom. Whereas members of a social friendship networks do not have such a medium through which they can assert their influence. The link information available on a social friendship network and other network centrality measures will just tell us the connectedness of a node which could be used to gauge the spread of influence rather than the influential node itself. Hence, there are works that measure the spread of influence through a node in social friendship networks [13; 31; 46]. A node that maximizes the spread of influence or who has a higher degree of connectivity is chosen for viral marketing. It is entirely possible that this node is connected to a lot of people but may not be the one who could influence other members. Bloggers spread their influence through blog posts. This information source could be tapped to compute influence of a blogger using several measures like inlinks, outlinks, comments, and blog post length. In a broader sense, influential nodes identified through the blogosphere are the ones who have “been influencing” fellow bloggers, whereas influential nodes identified through social friendship networks are the ones who “could influence” fellow members. The reason is trivial: in the blogosphere we have the history of who influenced whom through their blog posts, but in social friendship networks we don’t have such information. We only know who is linked to whom and the one who is the most linked could be used to spread the influence. But we don’t know whether he is the right person to do that job. Using [28], authors model the blogosphere as a social friendship network and then apply the existing works for mining influence in social friendship networks, but they lose essential statistics about the blog posts like inlinks, outlinks, comments, blog post quality, etc. A graph structure is strictly defined in social friendship networks whereas it is loosely defined in Blogosphere. Nodes are members or actors in a social friendship network but they could be bloggers, blog posts or blog sites. Social friendship networks are predominantly used for being in touch or making friends in society, while the main purpose of Blogosphere is to share ideas and opinions with other members of the community or other bloggers. This gives a more community experience to Blogosphere as compared to a more friendship oriented environment in social friendship networks. We could observe person-to-community interactions in the blogosphere, whereas one would observe more person-to-person interactions in social friendship networks. Another significant difference between the blogosphere and social friendship networks is in estimating the reputation/trust of members. Member’s reputation/trust in the blogosphere is based on the response to other member’s solicitations for advice. Member’s reputation/trust in a social friendship network is based on the network connections and/or locations in the network. We illustrate the differences mentioned above between social friendship networks and the blogosphere in Figure 1. The Volume 10, Issue 1 Page 28 7. Social N Networks Orkut, Facebook, LinkedIn, Linke Classmates.com,, etc. Social Friendship Networks 8. TUAW, Blogger,, Windows Windo Live Spaces, etc. complete set of social interacting services can be represented by Social Networks of which Social Friendship Networks and Blogosphere are focal parts. There are certain social friendship networking websites like Orkut, Facebook, LinkedIn39 , Classmates.com that strictly provide friendship networks. People join these networks to expand their social networks and keep in touch with colleagues. However, these websites enforce strict interaction patterns among friends and do not support flexible community structures as supported by blog sites. On the other hand blog sites like TUAW, Blogger40 , Windows Live Spaces41 allow members to express themselves and share ideas and opinions with other community members. However, these websites do not facilitate private friendship networks. Two members have to use other communication channels (e.g. email) to communicate between themselves, privately. But websites like LiveJournal and MySpace provide both social friendship networks and blogging capabilities to their members. Clearly based on the characteristics of social interactions one could observe overlap between social friendship networks and the blogosphere as depicted in Figure 1. CONCLUSIONS Blogosphere is one of the fastest growing, social networking media. The virtual communities in the blogosphere are not constrained by physical proximity and allow anytime, anywhere, and instant communications. In this paper we discuss current research issues in Blogosphere including modeling, blog clustering, blog mining, community discovery and factorization, influence and propagation, trust and reputation, and filtering spam blogs. We also present research methodologies including centrality measures, network models (scalefree, random, preferential attachment, hybrid, and cascade models), text and link analysis, decision theory, and agentbased modeling. We enlist ways to obtain datasets, visualize and study those using available tools. We also present a case study to exemplify how major aspects discussed in the paper are integrated to develop an application for identifying influential bloggers at a community blog site. Understanding the close relation of the blogosphere with the social networks, we compare and contrast these two environments and point out subtle yet significant differences that lead to different treatment for burgeoning researches in these environments. 39 http://www.linkedin.com/ http://www2.blogger.com/home 41 http://spaces.live.com/ 40 SIGKDD Explorations This work is in part supported by AFOSR and ONR grants to the second author. LiveJournal, MySpace ySpace, etc. Blogosp Blogosphere Figure 1: Social Friendship Networks and Blogosphere constitute part of Social Networks. 6. ACKNOWLEDGMENTS REFERENCES [1] E. Adar, L. Zhang, L. Adamic, and R. Lukose. Implicit structure and the dynamics of blogspace. In Proceedings of the 13th International World Wide Web Conference, 2004. [2] Nitin Agarwal, Magdiel Galan, Huan Liu, and Shankar Subramanya. Clustering blogs with collective wisdom. In Proceedings of the International Conference on Web Engineering, 2008. [3] Nitin Agarwal, Huan Liu, John J. Salerno, and Philip S. Yu. Searching for Familiar Strangers on Blogosphere: Problems and Challenges. In NSF Symposium on NextGeneration Data Mining and Cyber-enabled Discovery and Innovation (NGDM), 2007. [4] Nitin Agarwal, Huan Liu, Lei Tang, and Philip S. Yu. Identifying the influential bloggers. In Proccedings of the First ACM International Conference on Web Search and Data Mining (Video available at: http://videolectures.net/wsdm08 agarwal iib/), 2008. [5] G. Attardi and M. Simi. Blog mining through opinionated words. In Proceedings of the fifteenth Text REtrieval Conference (TREC), 2006. [6] A. L. Barabasi and R. Albert. Emergence of scaling in random networks. Science, 286(509), 1999. [7] A. Blanchard. Blogs as virtual communities: Identifying a sense of community in the julie/julia project. Into the Blogosphere: Rhetoric, Community and Culture.http://blog.lib.umn.edu/blogosphere, 2004. [8] A. Blanchard and M. Markus. The experienced sense of a virtual community: Characteristics and processes. The DATA BASE for Advances in Information Systems, 35(1), 2004. [9] A. Blum, T. H. C. Mugizi, and M. R. Rwebangira. A random-surfer web-graph model. In Third Workshop on Analytic Algorithmics and Combinatorics (ANALCO06), 2006. [10] Sergey Brin and Lawrence Page. The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems, 30(1–7):107–117, 1998. [11] Christopher H. Brooks and Nancy Montanez. Improved annotation of the blogosphere via autotagging and hierarchical clustering. In WWW ’06: Proceedings of the 15th international conference on World Wide Web, pages 625–632, New York, NY, USA, 2006. ACM Press. [12] Alvin Chin and Mark Chignell. A social hypertext model for finding community in blogs. In HYPERTEXT ’06: Proceedings of the seventeenth conference on Hypertext and hypermedia, pages 11–22, New York, NY, USA, 2006. ACM Press. Volume 10, Issue 1 Page 29 [13] Thayne Coffman and Sherry Marcus. Dynamic classification of groups through social network analysis and hmms. In Proceedings of IEEE Aerospace Conference, 2004. [28] Akshay Java, Pranam Kolari, Tim Finin, and Tim Oates. Modeling the spread of influence on the blogosphere. In Proceedings of the 15th International World Wide Web Conference, 2006. [14] Scott Deerwester, Susan T. Dumais, George W. Furnas, Thomas K. Landauer, and Richard Harshman. Indexing by latent semantic analysis. Journal of the American Society for information science, 1990. [29] Anubhav Kale, Amit Karandikar, Pranam Kolari, Akshay Java, Tim Finin, and Anupam Joshi. Modeling trust and influence in the blogosphere using link polarity. In International Conference on Weblogs and Social Media, 2007. [15] Daniel Drezner and Henry Farrell. The power and politics of blogs. In American Political Science Association Annual Conference, 2004. [16] L. Efimova and S. Hendrick. In search for a virtual settlement: An exploration of weblog community boundaries, 2005. [17] T. Elkin. Just an online minute... online forecast. http://publications.mediapost.com/index.cfm?fuseaction =Articles.showArticle art aid=29803. [18] Thomas L. Friedman. The World Is Flat: A Brief History of the Twenty-First Century. Farrar, Straus and Giroux, 2005. [30] Ed Keller and Jon Berry. One American in ten tells the other nine how to vote, where to eat and, what to buy. They are The Influentials. The Free Press, 2003. [31] David Kempe, Jon Kleinberg, and Eva Tardos. Maximizing the spread of influence through a social network. In Proceedings of the KDD, pages 137–146, New York, NY, USA, 2003. ACM Press. [32] J. Kleinberg. Authoritative sources in a hyperlinked environment. In 9th ACM-SIAM Symposium on Discrete Algorithms, 1998. [19] Michael Gamon, Anthony Aue, Simon Corston-Oliver, and Eric Ringger. Pulse: Mining Customer Opinions from Free Text. In Proceedings of the 6th International Symposium on Intelligent Data Analysis, 2005. [33] P. Kolari, T. Finin, and A. Joshi. SVMs for the blogosphere: Blog identification and splog detection. In AAAI Spring Symposium on Computational Approaches to Analyzing Weblogs, 2006. [20] Kathy E. Gill. How can we measure the influence of the blogosphere? In Proceedings of the WWW’04: workshop on the Weblogging Ecosystem: Aggregation, Analysis and Dynamics, 2004. [34] P. Kolari, A. Java, T. Finin, T. Oates, and A. Joshi. Detecting spam blogs: A machine learning approach. In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI), 2006. [21] Dan Gillmor. We the Media: Grassroots Journalism by the People, for the People. O’Reilly, 2006. [35] Apostolos Kritikopoulos, Martha Sideri, and Iraklis Varlamis. Blogrank: ranking weblogs based on connectivity and similarity features. In AAA-IDEA ’06: Proceedings of the 2nd international workshop on Advanced architectures and algorithms for internet delivery and applications, page 8, New York, NY, USA, 2006. ACM Press. [22] Jennifer Golbeck and James Hendler. Inferring binary trust relationships in web-based social networks. ACM Trans. Inter. Tech., 6(4):497–529, 2006. [23] Jacob Goldenberg, Barak Libai, and Eitan Muller. Talk of the network: A complex systems look at the underlying process of word-of-mouth. Marketing Letters, 12:211–223, 2001. [24] D. Gruhl, David Liben-Nowell, R. Guha, and A. Tomkins. Information diffusion through blogspace. SIGKDD Exploration Newsletter, 6(2):43–52, 2004. [25] R. Guha, Ravi Kumar, Prabhakar Raghavan, and Andrew Tomkins. Propagation of trust and distrust. In WWW ’04: Proceedings of the 13th international conference on World Wide Web, pages 403–412, New York, NY, USA, 2004. ACM Press. [26] Z. Gyongyi, P. Berkhin, Hector Garcia-Molina, and J. Pedersen. Link spam detection based on mass estimation. In Proceedings of the 32nd International Conference on Very Large Data Bases (VLDB), 2006. [27] Z. Gyongyi, H. Garcia-Molina, and J. Pedersen. Combating web spam with trustrank. In Proceedings of the 30th International Conference on Very Large Data Bases (VLDB), 2004. SIGKDD Explorations [36] R. Kumar, P. Raghavan, S. Rajagopalan, and A. Tomkins. Trawling the web for emerging cyber communities. In The 8th International World Wide Web Conference, 1999. [37] Ravi Kumar, Jasmine Novak, Prabhakar Raghavan, and Andrew Tomkins. On the Bursty Evolution of Blogspace. In Proceedings of the 12th international conference on World Wide Web, pages 568–576, New York, NY, USA, 2003. ACM Press. [38] J. Leskovec, M. McGlohon, C. Faloutsos, N. Glance, and M. Hurst. Cascading behavior in large blog graphs. In SIAM International Conference on Data Mining, 2007. [39] Beibei Li, Shuting Xu, and Jun Zhang. Enhancing clustering blog documents by utilizing author/reader comments. In ACM-SE 45: Proceedings of the 45th annual southeast regional conference, pages 94–99, New York, NY, USA, 2007. ACM Press. Volume 10, Issue 1 Page 30 [40] Yu-Ru Lin, Hari Sundaram, Yun Chi, Junichi Tatemura, and Belle L. Tseng. Splog detection using selfsimilarity analysis on blog temporal dynamics. In Proceedings of the 3rd international workshop on Adversarial information retrieval on the web (AIRWeb), pages 1–8, New York, NY, USA, 2007. ACM Press. [41] Bing Liu. Web Data Mining: Exploring Hyperlinks, Contents and Usage Data. Springer, 2006. [42] A. Ntoulas, M. Najork, M. Manasse, and D. Fetterly. Detecting spam web pages through content analysis. In Proceedings of the 15th international conference on World Wide Web (WWW), 2006. [43] Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. The pagerank citation ranking: Bringing order to the web. Technical report, Stanford Digital Library Technologies Project, 1998. [44] David M. Pennock, Gary W. Flake, Steve Lawrence, Eric J. Glover, and C. Lee Giles. Winners don’t take all: Characterizing the competition for links on the web. Proceedings of the National Academy of Sciences, 99(8):5207–5211, 2002. [45] Josep M. Pujol, Ramon Sangesa, and Jordi Delgado. Extracting reputation in multi agent systems by means of social network topology. In Proceedings of the first international joint conference on Autonomous agents and multiagent systems (AAMAS), pages 467–474, New York, NY, USA, 2002. ACM Press. [46] Matthew Richardson and Pedro Domingos. Mining knowledge-sharing sites for viral marketing. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge Discovery and Data mining, pages 61–70, New York, NY, USA, 2002. ACM Press. [47] Jordi Sabater and Carles Sierra. Reputation and social network analysis in multi-agent systems. In AAMAS ’02: Proceedings of the first international joint conference on Autonomous agents and multiagent systems (AAMAS), pages 475–482, New York, NY, USA, 2002. ACM Press. [48] Loren Terveen and David W. McDonald. Social matching: A framework and research agenda. ACM Trans. Comput.-Hum. Interact., 12(3):401–434, 2005. [49] D. J. Watts and S. H. Strogatz. Collective dynamics of ‘small-world networks. Nature, 393(6684):440442, 1998. [50] Bin Yu and Munindar P. Singh. Detecting deception in reputation management. In Proceedings of the second international joint conference on Autonomous Agents and Multiagent Systems (AAMAS), pages 73–80, New York, NY, USA, 2003. ACM Press. SIGKDD Explorations Volume 10, Issue 1 Page 31