Abstract

Literature review

As disease-causing microbes are always evolving, the appearance of new pathogens, the effects of existing pathogens on new populations, and the rise of drug-resistant bacteria (or superbugs) continue to pose a threat to global health in the form of EIDs.2 In addition to microbial adaptation and change, several other factors also contribute to the rise of EIDs. According to Lederberg et al,1 human demographics and behaviors, such as increased population density or individual behaviors (eg, sexual activities, substance abuse), can lead to the emergence of new infectious diseases. Modern medicine has benefited the human race at the price of millions acquiring nosocomial infections in hospitals. Food-borne illnesses, such as those associated with Escherichia coli, are caused by problems in food processing and handling. Economic development and land use can also lead to EIDs.1 One example is Lyme disease, a bacterium (Borrelia burgdorferi) carried by deer whose population and contact with humans increased with reforestation. The breakdown of basic public health and sanitation can also cause deadly attacks of EIDs (eg, Ebola outbreak in West Africa, cholera outbreak in Haiti).1 Finally, international travel and commerce spread local EIDs (eg, SARS) across national boundaries.1

The history of the 21st century is arguably a history of the rise of social media, computer technologies that allow the collaborative creation and sharing of information. Obar and Wildman4 define social media as services that are based on Web 2.0 technologies and largely rely on user-generated content, by which individuals and organizations create profiles and develop social networks online. According to a recent report of Pew Research Center, the percentage of U.S. adults using social media increased from 7% to 65% between 2005 and 2015.5 Globally, an estimated 2.34 billion people were social media users in 2016.6 Health communication researchers and practitioners have recognized the potential of social media in health education and promotion. A systematic review of literature7 shows that social media can be effectively used in health promotion in a number of ways, including providing information access, delivering health campaigns, and providing social support. Public health organizations such as the World Health Organization (WHO),8 the Centers for Disease Control and Prevention (CDC), and local public health departments have started to adopt social media in communicating with the public.

Social media are a potentially useful tool for the effective communication of EID outbreak updates and essential medical information to the public. EIDs represent unfamiliar risks to the public, who often turn to both traditional media and social media for information.9 How these EIDs are portrayed and communicated in media shapes people's perceptions of risks, which in turn have a significant impact on their decision-making process and risk management behaviors.10 Social media have been instrumental in informing the public about recent EID outbreaks such as the Ebola outbreak in 2014 and the H1N1 outbreak in 2009.3 Furthermore, social media users not only share EID-related information that they obtain from other sources (eg, traditional media), but also share their own personal experiences and understanding about EIDs. Because information about EIDs on social media are user-generated, such information is not always accurate or useful. It often contains rumors, misinformation, and conspiracy theories.11 As a result, the WHO calls for social media to be used more proactively in disseminating health messages to journalists, physicians, and the general public, particularly to counter misinformation about EIDs.8 To better understand the status of existing research on EID communication on social media, we posed the following research question: What is the current status of research on EID communication on social media?

Methods

Data sources and search strategy

Five medical and health science, psychology, social sciences, and communication databases were searched: PubMed/MEDLINE, PsycINFO, CINAHL Plus, Sociological Abstracts (ProQuest), and Communication Source (EBSCOhost). Based on an exploratory literature search and World Health Organization's “Disease outbreaks by year,”12 key words used in the search included terms for health crises (“epidemic” or “pandemic” or “influenza” or “virus” or “infectious disease” or “outbreak” or “Ebola” or “measles” or “Zika” or “Cholera” or “SARS” or “flu” or “H1N1” or “H5N1” or “H7N9” or “dengue” or “fever” or “plague” or “MERS” or “malaria” or “polio”) and terms for social media (“social media” or “social networking sites” or “SNS” or “Facebook” or “Twitter” or “YouTube” or “blog” or “chat room”). The initial search yielded 569 items. First, the titles of these articles were checked for duplications by one of the authors (B.B.), and 124 duplicates were removed. Next, the abstracts of the remaining 445 articles were screened by an author (B.B.) according to the criteria previously listed. Articles meeting one of the following criteria were excluded: (1) they were not written in English, (2) they were conceptual without empirical research, (3) they did not focus on EIDs, (4) they were only concerned with the contents of nonsocial media Web sites (eg, news Web sites), or (5) the social networking platforms under study (ie, discussion forums and online groups) were established by the research team specifically for teaching or research purposes. Studies on outbreak surveillance were also excluded from the current review. After screening, 71 articles remained in the sample. These articles could not be categorized as eligible based on the information provided in the abstract. Hence, the full texts of these articles were downloaded and read multiple times to ascertain eligibility. Two of the authors (L.T. and B.B.) discussed these articles to decide whether they should be included in the systematic review. Among these, 26 articles met the inclusion criteria. Reference lists from these articles were also screened, and 4 more articles were identified. In the end, a total of 30 articles were included in this systematic review. The earliest article was published in 2010 and the latest in 2016. Figure 1 shows the article inclusion flow diagram.

Open in a separate window

Fig1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram illustrating literature selection process.

Results

The 30 research articles were categorized by EIDs addressed, social media studied, and research approaches taken. In terms of the EIDs studied, H1N1 (or swine flu, n=15) received the most attention from researchers, followed by Ebola virus (n=10) and H7N9 (or avian flu/bird flu, n=2). A number of other EIDs, including West Nile virus, measles, Middle East respiratory syndrome coronavirus (MERS-CoV), and enterohemorrhagic Escherichia coli, only appeared in 1 article. (See Table 1 for information about articles studying each EID.)

In terms of the types of social media studied, Twitter was undoubtedly the most scrutinized social media platform and was studied in 16 articles. YouTube, Facebook, and blogs were each studied in 6 articles. Discussion forums were studied in 3 articles, and Flickr appeared in 2 articles. Finally, Instagram, Web site comments, Weibo (a Chinese microblogging platform), and Delicious were each studied in 1 article. (See Table 2 for information about articles studying each social media application.)

Table2

Types of social media studied (N = 43^*)

Social media	N	Studies
Twitter	16	Biswas,3 Chew and Eysenbach,14 Collier et al,15 Ding and Zhang,16 Freberg et al,9 Gaspar et al,39 Househ,27 Kim and Liu,18 Lazard et al,28 Liu and Kim,19 Mollema et al,40 Odlum and Yoon,30 Signorini et al,23 Towers et al,34 Vos and Buckner,37 Wong et al35
YouTube	6	Basch et al,26 Ding and Zhang,16 Dubey et al,38 Nagpal et al,29 Pandey et al,22 Pathak et al31
Facebook	6	Biswas,3 Ding and Zhang,16 Kim and Liu,18 Liu and Kim,19 Mollema et al,40 Strekalova33
Blogs	6	Ding and Zhang,16 Freberg et al,9 Gao et al,17 Mollema et al,40 Nerlich and Koteyko,21 Tausczik et al24
Discussion forums	3	Luoma-aho et al20 Mollema et al,40 Tirkkonen and Luoma-aho25
Flickr	2	Ding and Zhang,16 Seltzer et al32
Instagram	1	Seltzer et al32
Web site comments	1	Atlani-Duault et al13
Weibo	1	Fung et al36
Delicious	1	Freberg et al9

Open in a separate window

^*The total number is>30 because many articles studied multiple social media outlets.

To provide a systematic overview of these studies, we evaluated them in terms of the topic studied, theory used, method used, and major findings. We found that these studies typically take one of the following 3 approaches: (1) assessment of the public's interest in and responses to EIDs, (2) organizations' use of social media in EID communication, and (3) assessment of the accuracy of medical information about EIDs on social media. Only 1 article fell into 2 categories.16 Studies belonging to each of these approaches are subsequently reviewed in depth.

Approach 1: Assessment of the public's interest and responses

Most journal articles on this topic (n=17) used social media data to assess the public's interest in and responses to EIDs. They were typically based on manual content analysis with the occasional use of computer-assisted content analysis. Manual content analysis uses human coders to manually assign each text into a set of predetermined categories so that researchers can analyze a set of texts quantitatively.41 Because of the use of human coders, intercoder reliability needs to be calculated. Computer-assisted content analysis allows researchers to assess the topics, sentiments, and other contents of large amount of text through the use of computers. Common approaches used in computer-assisted content analysis include topical analysis, semantic network analysis, sentiment analysis, machine learning, and so forth.41 Occasionally, qualitative thematic analysis and discourse analysis methods were used. Thematic analysis allow researchers to inductively identify themes and patterns within data, whereas discourse analysis focuses on the structure of language.42 (See Table 3 for a summary of these studies in terms of EID studied, social media studied, theory applied, method used, and major findings.)

Table3

Approach 1: Assessment of public's interests and responses

Article	EID studied	Social media studied	Theory used	Method and intercoder reliability	Major findings
Atlani-Duault et al13	H1N1	Web site comments	Critical theory	Discourse analysis	A discourse about “geography of blame” was present in social media but not in traditional media. This discourse blamed government, pharmaceutic companies, and “figures of otherness” for the outbreak.
Chew and Eysenbach14	H1N1	Twitter	None	Manual content analysis for training	This study identified the frequencies of content (resource, personal experience, personal opinion, jokes, and marketing), qualifiers (humor, relief, downplayed risk, concern, frustration, and question), and links (news Web site, news blog, government, etc) and their longitudinal changes.
				k>0.70
				Computer-assisted content analysis based on key word queries using Structured Query Language
				Correlation, 0.70
Collier et al15	H1N1	Twitter	None	Manual content analysis for training	It studied 5 coping behaviors: avoidance, increased sanitation, seeking pharmaceutic intervention, wearing a mask, and self-diagnosis. These behaviors also correlated with the number of CDC-reported cases of flu.
				k=0.86
				Computer-assisted content analysis based on machine learning (SVM and Naive Bayes)
Ding and Zhang16	H1N1	Facebook, Twitter, e-cards, buttons and badges, Podcasts, Flickr, YouTube, widgets, Sina blog	Risk communication	Manual content analysis	Three types of messages on Sina blog—which represents the public's voice—were experiences and witnesses from grassroots bloggers, comments and criticisms from celebrity bloggers, and prevention information. The discourse on social media rejected the official discourses in traditional media.
				No intercoder reliability
Freberg et al9	H1N1	Delicious, Twitter, blogs	Crisis communication	Manual content analysis	It studied the following variables: source of information (CDC, UK Guardian, etc), types of documents (blogs, Web sites, news, videos, etc), tagged key words (H1N1, swine flu, flu, health, influenza, social media, CDC, etc), and sources bookmarked (Twitter, WebMD, CDC, Google Food Trends, etc).
				No intercoder reliability
					CDC was the most often bookmarked information source. Blog was the most often bookmarked document. H1N1, swine flu, flu, and health were the top 4 most bookmarked key words. Twitter was the most bookmarked source.
Fung et al36	MERS-CoV, H7N9	Weibo	None	Counted number of Weibo posts	Chinese Weibo users' interest in H7N9 doubled compared with the previous week (t test) in response to the Chinese government's announcement of a confirmed case, but their interest in MERS-CoV did not increase significantly after the WHO announcement. Overall, Chinese Weibo users responded more strongly to H7N9 than MERS-CoV because the former was closer to home.
Gao et al17	H1N1	Blogs	Framing	Manual content analysis	It identified 7 frames used in newspapers and health blogs: action, severity, conflict, new evidence, economic consequence, blame and responsibility, and reassurance. It also identified the dominant frame used in news articles or blogs. Blogs were more likely to use the new evidence frame. Sources used were also identified. Overall, newspapers used more sources than blogs.
				α>0.80
Gaspar et al39	EHEC	Twitter	Families of coping,43 crisis informatics	Manual content analysis	This study coded 12 coping strategies: self-reliance, support seeking, problem-solving, information seeking, accommodation, negotiation, delegation, isolation, helplessness, escape, submission, and oppression. The use of strategies differed in uncertain and certain periods.
				k=0.80
Househ27	Ebola	Twitter	None	Frequencies of tweets and Google News articles about Ebola	It compared the number of tweets and the number of Google News articles and concluded that the public interest as measured in the number of tweets was related to the number of Google News articles.
Luoma-aho et al20	H1N1	Discussion forums	Issue arena	Manual content analysis	It used social media data to represent the “citizens' view” in contrast with the organizational point of view presented in governmental media releases. Three topics were identified: symptoms of the flu; safety of the vaccine; and the epidemic, risk groups, and its victims.
				No intercoder reliability
Mollema et al40	Measles	Twitter, forums, blogs, Facebook, and others	None	Frequencies of messages about measles	It found stronger correlation between the weekly number of social media messages and the weekly number of online news articles than between the weekly number of social media messages and the weekly number of reported measles cases. Topics of social media discussion of measles were measles outbreak, measles prevention, perceived risks of measles, refusal of vaccination because of religious reasons, criticism toward vaccination, trust, and the role of institutions (in descending order).
					Sentiments expressed were frustration, humor, sarcasm, concern, and relief (in descending order).
				Manual content analysis
				α between 0.58 and 0.81
Nerlich and Koteyko21	H1N1	Blogs (cf. traditional newspapers in the UK)	Meta-communication	Thematic analysis (qualitative content analysis)	Three themes in print media stories were blame the official, blame the media, and “we are hooked on hype.”
					The interaction between traditional and digital media contributed to a heightened discourse of blame and counterblame, but also self-blame and reflection about the role of media in pandemic communication.
			Crisis communication
Odlum and Yoon30	Ebola	Twitter	None	Computer-assisted content analysis based on natural language processing	It identified 4 topics in the discussion about Ebola on Twitter: risk factors, prevention education, disease trend, and compassion. No frequency was reported.
Seltzer et al32	Ebola	Instagram and Flickr	None	Manual content analysis	Nine types of images were identified, including healthcare workers, West Africa, the Ebola virus, and the artistic rendering of Ebola.
				r=0.96 (Instagram)
				r=0.89 (Flickr)
					Types of texts identified were facts, fears, politics, and jokes.
					Instagram images were primarily coded as jokes or unrelated, whereas Flickr images primarily depicted healthcare workers providing care or other services.
Signorini et al23	H1N1	Twitter	None	Frequency counts	It used number of tweets to track public interest and disease development.
Tausczik et al24	H1N1	Blogs	Health belief model	Computer-assisted content analysis based on key words using LIWC	It used blog posts to monitor public anxiety by examining the language used in personal blogs. In comparison with control blogs, swine flu blog entries had significantly higher use of words related to health, death, and anxiety, and fewer words related to positive emotions. The use of language in blogs was similar to the language in newspaper articles.
Tirkkonen and Luoma-aho25	H1N1	Discussion forum	None	Manual content analysis	Civilians did not trust authorities and the protective actions taken in online forums. The authorities' intervention aimed at correcting false information and shaping opinions in the discussion forums seemed to fail.
				No
Towers et al34	Ebola	Search and Twitter, news videos	None	Frequency counts	Ebola-related news videos inspired tweets and Internet searches.
Vos and Buckner37	H7N9	Twitter	Crisis and emergency risk communication	Manual content analysis	A large proportion of messages contained sensemaking information, but few tweets contained efficacy information that would help individuals respond appropriately to the crisis.
				α>0.899
				Computer-assisted content analysis based on key words using KH Coder

Open in a separate window

CDC, Centers for Disease Control and Prevention; EHEC, enterohemorrhagic Escherichia coli; EID, emerging infection disease; MERS-CoV, Middle East respiratory syndrome coronavirus; WHO, World Health Organization.

Approach 2: Organizations' use of social media and public responses

Organizations, including both corporations and governmental agencies, have started to use social media to communicate EIDs to their different stakeholders. Five articles examined how different organizations used social media in EID communication through the perspective of crisis communication, and 2 articles focused on the public's responses to organizations' EID communication. Most of these studies were based on theories of risk and crisis communication. (For a summary, see Table 4 .)

Table4

Approach 2: Organizations' use of social media in communicating EIDs

Article	Type of EID	Type of social media	Organizations studied	Theoretical approach	Method and intercoder reliability	Major findings
Biswas3	H1N1	Twitter, Facebook	CDC, WHO	Outbreak communication	Manual content analysis	Types of messages: investigation or diagnosis, prevention, treatment, and update.
					No
						Facebook facilitated more interactivity because of its built-in features. However, both the CDC and WHO focused on 1-way communication instead of interacting with the public.
Ding and Zhang16	H1N1	All major social media	U.S. government (CDC and HHS)	Risk communication	Manual content analysis	CDC and HHS most frequently used Facebook in communicating with the public about H1N1, followed by Twitter, e-cards, buttons and badges, podcasts, Flickr, YouTube, and widgets. The functions served by these social media were updates, policies and guidelines, prevention topics, official actions and efforts, general information, and scientific research, in descending frequencies.
					No
Kim and Liu18	H1N1	Twitter, Facebook	13 government and corporate organizations	Situational crisis communication theory	Manual content analysis	Governmental organizations emphasized providing instructional information to their primary audience, such as guidelines about how to respond to a crisis, whereas corporations emphasized reputation management, frequently adopting denial, diminish, and reinforce response strategies.
					α=0.77-1.0
Liu and Kim19	H1N1	Twitter, Facebook	13 government and corporate organizations	Framing, crisis communication	Manual content analysis	Organizations were more likely to frame the crisis as a disaster, a health crisis, or a general health issue on traditional media and were more likely to frame it as a general crisis on social media. Organizations relied on traditional media more than social media to address emotions.
					α=0.77-1.0
Wong et al35	Ebola	Twitter	286 local health departments	None	Manual content analysis	78.6% tweets were information giving, 22.5% were on preparedness, 20.8% were news updates, and 10.3% were event promotion. Each wave of tweets corresponded with a major news event.
					r=0.54-1.0
Studies of the public's responses to organizations' use of social media
Lazard et al28	Ebola	Twitter	CDC	None	Computer-assisted content analysis based on unsupervised text mining (SAS Text Miner)	This study identified 8 major concerns of the public, such as expert opinions, prevention, and questions.
Strekalova33	Ebola	Facebook	CDC	None	Manual content analysis	Men wrote more comments per person than women on CDC posts about Ebola.
					No

Open in a separate window

CDC, Centers for Disease Control and Prevention; EID, emerging infection disease; HHS, U.S. Department of Health & Human Services; WHO, World Health Organization.

All 5 articles on organizations' social media messages about EIDs used manual content analysis. Among them, only 3 reported the types of social media used and the types of messages posted by organizations. For instance, Biswas3 studied how the CDC and WHO used Twitter and Facebook in communicating about H1N1 and identified the following 4 types of messages: investigation or diagnosis, prevention, treatment, and update. He also concluded that Facebook facilitated more interactivity because of its technical features, but both the CDC and WHO focused on 1-way communication instead of interacting with the public. Ding and Zhang16 studied how the CDC and the U.S. Department of Health & Human Services used social media during an H1N1 outbreak and found that Facebook was the most frequently used, followed by Twitter, e-cards, buttons and badges, podcasts, Flickr, YouTube, and widgets. The following types of messages were identified in descending frequencies: updates, policies and guidelines, prevention topics, official actions and efforts, general information, and scientific research, in descending frequencies. Wong et al35 studied how 286 local health departments in the United States communicated the risks of Ebola on Twitter and identified 4 major types of messages: information giving (78.6%), preparedness (22.5%), news update (20.8%), and event promotion (10.3%).

However, 2 studies offered more theoretical interest by comparing strategies used by different types of organizations in communicating about EIDs using traditional and social media. Adopting the situational crisis communication theory, Kim and Liu18 studied how 13 governmental organizations and corporations communicated H1N1 on social media (Twitter and Facebook) and concluded that governmental organizations emphasized risk communication (eg, providing guidelines about how to respond to the outbreak), whereas corporations paid more attention to reputation management. Using the same data set, Liu and Kim19 examined how governmental organizations and corporations framed the H1N1 outbreak on social media (Twitter and Facebook) and traditional media (eg, websites). They found that these organizations tended to frame the outbreak as a disaster, a health crisis, or a general health issue on traditional media, but tended to frame it as a general crisis on social media.

Two additional studies examined how the public responded to organizations' EID communication on social media and therefore were reviewed here as well. Lazard et al28 studied the public's response to CDC's Twitter live chat about Ebola using an unsupervised text mining tool (SAS Text Miner) and identified 8 major concerns expressed by the public, such as questions about the science around Ebola and fear of travel. However, they relied on a very small sample of tweets (N=2,155). Strekalova33 studied the characteristics of Facebook users commenting on CDC's Facebook posts about Ebola through manual content analysis and found that men wrote more posts per person than women. Neither of these studies was able to shed light on how people responded to the specific strategies or messages used by organizations.

Approach 3: Accuracy of medical information

Information about EIDs on social media is not always accurate or useful because it is user-generated. Five articles examined the accuracy of social media medical information about EIDs and interestingly all 5 studies in this category focused on YouTube. Among them, 3 studies were about Ebola virus, 1 was about West Nile virus, and 1 was about H1N1. Studies adopting this approach were not based on any specific communication or public health theories. These studies typically examined around 100 videos through manual content analysis. In general, approximately 70%-80% of videos contained useful information (either accurate medical information or outbreak updates) across different EIDs, such as West Nile virus,38 H1N1,22 and Ebola.31 Some studies further examined the types of medical information included in these YouTube videos, such as modes of transmission, screening, treatment, and symptoms.26, 29, 38 Very often, these studies tried to establish the relationship between content characteristics and popularity of videos. For instance, Dubey et al38 found that misleading videos had a significantly higher number of views per day, but Pandey et al22 concluded that there was no significant difference among useful videos, misleading videos, or updates in terms of views per day. Some of these studies also examined the relationship between the source and the quality of content and the number of views. Not surprisingly, Pathak et al31 found that independent users were more likely to post misleading videos, and news agencies were more likely to post useful videos. (See Table 5 for a summary of these 5 studies.)

Table5

Approach 3: Accuracy of medical information

Study	Type of EID	Sample	Reliability	Coding categories	Major findings
Basch et al26	Ebola	100 most watched videos	k=0.98	Video characteristics (source, year uploaded, length, total number of views)	One-third of videos discussed modes of transmission, but few mentioned treatment, and none mentioned the need for U.S. funding of disaster preparedness, coordination between governments on different levels, or beds ready for containment.
				Content (coded as yes or no: 19 items, such as modes of transmission, death toll in West Africa, number of cases in West Africa, quarantine, anxiety over infection, public fear, comedy skit, danger for healthcare personnel, conspiracy theory, and need for medical help and resources)
					There was no significant difference between consumer videos and commercial television videos in number of views.
Dubey et al38	West Nile	106 videos	k=0.95	Video characteristics (source, days on YouTube, length, total number of views)	Approximately 80% of videos contained useful information, among which 60% discussed prevention and 35% contained updates.
				Content (coded as useful or misleading/nonuseful; useful videos were further coded in terms of whether they contained the following items: prevention, symptom, and update)
					54% of videos were uploaded by individuals, 41% by news agencies, and 3% by healthcare agencies. Nonuseful videos had a significantly higher number of views per day.
Nagpal et al29	Ebola	100 most relevant videos	Third person arbitration	Source (individual or organization)	High relevance videos (ranked 1-50) had more views, likes, dislikes, shares, and subscriptions than low relevance videos (ranked 51-100). The difference was attributed to “clinical symptoms” only.
				Video information and quality index (5-point Likert scale regarding flow of information, information accuracy, quality, and precision)
				Medical information and content index (5-point Likert scale regarding components of medical information including prevalence, transmission, clinical symptoms, screening and testing, and treatment and outcome of Ebola infection)
Pandey et al22	H1N1	142 videos	k=0.97	Content (coded into 1 of 3 categories: useful, misleading, or update)	60% of videos were useful, 16% were misleading, and 23% were updates. No difference was found among these groups in terms of number of views.
				Source (CDC, UN, WHO, Red Cross, news agencies, and independent users)
				Video characteristics (total viewership, number of days since upload, length of video)
Pathak et al31	Ebola	108 most relevant videos	k=0.68	Content (coded as misleading or useful)	73% of videos were useful; the rest were misleading. Independent users were more likely to post misleading videos, and news agencies were more likely to post useful videos.
				Video characteristics (total number of views, likes, days on YouTube, length)
				Source (CDC, WHO, Red Cross, NGOs, academic or hospital, news agencies, independent users)

Open in a separate window

NOTE. All 5 articles studied YouTube and used the manual content analysis method.

CDC, Centers for Disease Control and Prevention; EID, emerging infection disease; NGO, nongovernmental organization; UN, United Nations; WHO, World Health Organization.

Discussion

Our systematic literature review of published journal articles on EID communication on social media shows increased academic interest in this topic. Researchers studied social media content about EIDs to understand (1) the public's interest in and responses to EIDs, (2) the use of social media by organizations during an EID outbreak, and (3) the accuracy of social media EID content. Although researchers are making strides in this new area of research, the field suffers from 2 main problems: a lack of theorization and a need for more methodologic rigor.

Footnotes

References