Original Article

Cephalalgia
2020, Vol. 40(14) 1633–1644
Googling migraine: A study of Google ! International Headache Society 2020
Article reuse guidelines:
as an information resource of sagepub.com/journals-permissions
DOI: 10.1177/0333102420942241
migraine management journals.sagepub.com/home/cep

Rachid Bojazar1 , Thien Phu Do1, Jakob Møller Hansen2,

David W Dodick3 and Messoud Ashina1

Abstract
Introduction: Health information is one of the most frequently searched topics on the internet. In this observational
cross-sectional study, we evaluated the content of the highest indexed Google search results related to migraine
management.
Methods: We identified the five most used search terms related to migraine management via Google Trends in the time
period 1 January 2004 to 2 October 2019. We entered each search term into Google’s search engine and retrieved the
search results from the first three pages from each search query. We stratified the recommended treatment options and
evaluated the websites using the DISCERN tool.
Results: In total, 73 unique websites recommended a total of 77 different migraine treatment options, consisting of 35
(45%) acute and 42 (55%) preventive treatments. For acute treatments, pharmacological options were more frequently
recommended (88% of websites), whereas non-pharmacological options were more frequent among preventive treat-
ments (67%). Evaluation of the consumer health information using the DISCERN tool showed that non-governmental
organizations had the highest mean total score with 3.8 (
0.19).
Conclusions: Googling when looking for migraine treatments reveals a multitude of management proposals of varying
quality. Non-governmental organizations provide the overall highest quality of written consumer health information on
migraine treatments among search results. We encourage stakeholders to optimize and distribute high-quality and peer-
reviewed information on migraine management.

Keywords
Online, patient information, internet, headache, health information, consumer
Date received: 14 January 2020; revised: 6 April 2020; accepted: 23 May 2020

Introduction
The general migraine literacy appears to be poor in
spite of a high prevalence (1). In both America and
Europe, health information is one of the most frequent- 1
Danish Headache Center and Department of Neurology, Rigshospitalet
ly searched topics on the internet (2). Today, many Glostrup, Faculty of Health and Medical Sciences, University of
patients view the internet as a valuable and reliable Copenhagen, Glostrup, Denmark
2
source of heath related information and consult the Danish Headache Knowledge Center, Rigshospitalet Glostrup, Glostrup,
Denmark
internet before seeking a professional diagnosis (2). 3
Mayo Clinic, Scottsdale, AZ, USA
Despite the benefit of easily accessible health informa-
tion, the possibility of misinformation leading to unfor- Corresponding author:
tunate actions or psychological distress should be a Messoud Ashina, Danish Headache Center, Department of Neurology,
Rigshospitalet Glostrup, Faculty of Health and Medical Sciences,
concern. To ensure the best possible treatments for
University of Copenhagen, Valdemar Hansen Vej 5, DK-2600 Glostrup,
patients, health care providers ought to familiarize Denmark.
themselves with patterns in patients’ self-care (3). Email: ashina@dadlnet.dk
1634 Cephalalgia 40(14)

Google is both the most world’s most visited website 10-operated computer using Mozilla Firefox in
and most used search engine making this the first (and Glostrup, Greater Copenhagen, Denmark. All cookies,
sometimes) only source of health information. previous search history and temporary internet files
Google provides indexation by sorting through were deleted prior to search entry. In Private Mode,
internet websites and data via multiple proprietary we set Google Trends to capture worldwide non-real-
algorithms (4). These algorithms consider a variety of time search data from 1 January 2004 (earliest possible
variables when ranking different search results. In part, date) to 2 October (date of search query), capturing all
Google’s algorithms favour newly uploaded content as categories of web searches. We then identified the five
well as websites that many users seem to appreciate most frequent (top) related queries to migraine treat-
when conducting searches, as well as references from ment. Only the top five search queries in relation to
other websites. According to Google, this provides an migraine treatment were eligible for further explora-
opportunity for assessment of reliability and authority. tion. These five search terms were then entered in
Computer software determines which websites to visit Google. We repeatedly cleared cookies, web browsing
and index, including the frequency and volume of history, and temporary internet files before and
indexed data (5). between search queries. The search results (e.g.
Given the accessibility of the internet, multiple Uniform Resource Locators – URLs) from the first
attempts have been made to elucidate the available three Google pages were retrieved for each search
tools for assessing quality and reliability of online query. The rationale for a cutoff of three Google
health information (6). These attempts include website pages is that most people doing an online search will
readability scoring systems and quality certification. look no further than the first three pages generated
Besides these approaches, the utilization of standard- (11). We excluded duplicate websites and defective
ized instruments to evaluate the framework and con- website links across search terms.
tent of written consumer health information has been
used. The DISCERN instrument is an example of a Content analysis
validated questionnaire to assess consumer health
information. The DISCERN instrument was formed For each search query, we screened the content of each
on systematic input from an expert panel including website for information related to migraine manage-
clinical specialists, general practitioners and self-help ment. We recorded whether there were recommenda-
group members (7). It facilitates standardized but yet tions of headache diary usage and cautions against
flexible appraisal of online consumer health informa- medication overuse headache. Furthermore, we
tion (8). assessed the nature of recommended treatments (e.g.
The aim of this observational cross-sectional study is acute, preventative, non-pharmacological, pharmaco-
to evaluate the content of the highest indexed websites logical interventions) and stratified the recommended
about migraine treatment on Google. treatment options accordingly. Finally, we grouped the
websites by information provider type based on their
self-reported purpose on the website’s “About us”
Methods sections.
Two authors (RB, TPD) independently used the
Search strategy DISCERN tool to evaluate the contents of each web-
We used Google Trends to identify the most frequent site. The DISCERN instrument is a validated tool
migraine-related search query on Google (9). Google which was developed to assess the quality for written
Trends tracks Google searches and provides rankings consumer health information on treatment choices. It
for past search queries since 2004. Search terms are consists of a brief questionnaire of 16 questions divided
scored relative to the total amount of searches per- into three sections. Questions 1 through 8 are named
formed on Google. This is quantified by Relative Section 1, where the reliability of the information is
Search Volume (RSV), ranging from 0–100, where assessed – for example, the framework of the conveyed
100 is the most frequently searched term at the given message. Section 2 consists of questions 9 through 15,
time (10). By entering “migraine” as a search term, where the quality of information is evaluated. Question
Google Trends suggests the highest indexed related 16 addresses the rater’s overall impression of the
search terms to the search term “migraine”. This func- source. The DISCERN score scale ranges from 1 to
tion is used to determine which search terms to use 5, where 1 is low quality, 3 is partial quality, and 5 is
when screening for information related to treatment. high quality information (7). We defined agreement as
On 2 October 2019, at approximately 16:15, we entered consensus within one score. In case of disagreement, we
each search term into Google’s Trends search engine. resolved any discrepancy by re-assessing the website in
The searches were performed on a Windows question.
Bojazar et al. 1635

Most used search terms on Google in relation to “Migraine” identified

through Google Trends

“Migraine “Excedrin” “Excedrin “Migraine “Migraine

treatment” (n = 30) migraine” relief” medicine”
(n = 30) (n = 30) (n = 29) (n = 30)

Website exclusion (n = 76)

• Duplicates (n = 75)
• Defective link (n = 1)

Pool total number of search results on the first

three pages of Google (n = 73)

Website exclusion (n = 7)

• Articles in peer-
reviewed journals
(n = 7)

Stratification of Evaluation of
recommended websites using the
treatments DISCERN tool
(n = 73) (n = 66)

Figure 1. Flowchart of the search strategy, web searches and processing of results.

Approval of a scientific ethics committee is not pharmaceutical product containing aspirin, paraceta-
required as the data is publicly available. We collected, mol and caffeine.
stratified and evaluated the data using Microsoft Excel Table 1 and Figure 2 shows the search terms and
Version 1908, Microsoft Office 365 ProPlus (Microsoft, relative search volume. Historically, “excedrin” had the
2019) and IBM SPSS Statistics Version 26 (IBM, 2019). largest change in popularity over time, with peak
change in interest around January of 2012. The other
Data availability four search terms showed sporadic change in interest in
All data for this article is available per request. search volume over time. Of note, the curves in
Figure 2 shows the change in search volume of a
search term relative to itself.
Results
A flowchart of the search strategy and results is pre-
Search results
sented in Figure 1. We identified the following search
terms from Google Trends, which were the highest Input of these search terms into Google yielded a total
indexed migraine treatment-related terms on Google of 149 search hits from the first three pages. We
(in ranked order): “migraine treatment”, “excedrin”, removed 75 duplicates and one defective link. The
“excedrin migraine”, “migraine relief”, and “migraine remaining 73 websites were pooled, and the content
medicine”. ExcedrinVR is an over-the-counter of these websites were subsequently analyzed by
1636 Cephalalgia 40(14)

stratification of the recommended treatment options. treatment, 45 out of 73 (62%) websites caution against
We evaluated 66 of these 73 websites using the medication overuse headache.
DISCERN tool, as seven websites were excluded due The 73 included websites recommended 77 different
to them being articles in peer-reviewed journals. treatment options (Table 2). Sixty-seven websites
(92%) recommended at least one of 35 acute migraine
treatment options. The majority, 64 websites (88%)
Migraine management recommended at least one of 15 acute pharmacological
treatments. Thirty-seven websites recommended at
Thirty-two websites out of 73 (44%) websites recom-
least one of 20 non-pharmacological treatments.
mended that patients track their symptoms using a
Regarding preventive treatment, 49 websites (67%) rec-
headache diary. In the context of pharmacological
ommended at least one of 42 different options. Forty-
six websites (63%) recommended at least one of 18
Table 1. Overview of the most popular search terms related to
preventive non-pharmacological treatments and 40
migraine. Relative search volume is the quantity of search inter-
est relative to the highest point of the chart for the given time. A websites (55%) recommended at least one of 24 phar-
value of 100 is peak popularity for that term. A value of 50 macological treatments. Thirty-six out of 73 websites
denotes that the term is half as popular as the term with the (49%) recommends only pharmacological treatments,
value of 100. A value of 0 means that there was insufficient data whereas six out of 73 websites (8%) strictly recom-
for this term. mends non-pharmacological options.
Search term Relative search volume (RSV)
Information provider type
Migraine treatment 39
Excedrin 32 Figure 3 shows websites grouped by website category
Excedrin migraine 32 per treatment provider type. Grouping of websites by
Migraine relief 28 information provider type revealed eight different cat-
Migraine medicine 27 egories: 16 e-commerce websites (22%), 11 websites by
medical professionals (15%), 11 health information

Figure 2. Popularity of search terms over time. Figure 2 depicts the worldwide popularity of the five mentioned search queries over
time, compared to itself. Google Trends data were available from 1 January 2004 until 2 October 2019. In the upper bar of Figure 2,
each search term is marked with colours corresponding to the curve for that specific search term. Note that a curve represents the
term’s change in search volume over time compared to itself. The change in volume is not normalized to the four other search terms.
Therefore, a higher value over time depicts a change of number of searches for that specific search term. The used search terms and
their relative search volume values are listed in Table 1. Google and the Google logo are registered trademarks of Google LLC, used
with permission. Data source: Google Trends https://www.google.com/trends.
Bojazar et al. 1637

Table 2. Characteristics of migraine treatment options as recommended in internet search results. The table depicts the recom-
mended treatment options and the number/percentage of websites which recommend them. The table is organised by nature of
treatment option. Treatment options are divided into acute/preventive and further into modality of treatment, e.g. pharmacological/
non-pharmacological.

Number of websites
Recommendation n ¼ 73 (100%)

Headache diary 32 (44%)

Caution against medication overuse headache 45 (62%)
Acute treatment 67 (92%)
Non-pharmacological 37 (51%)
Relaxation 25 (34%)
Sleep 22 (30%)
Place wrapped ice pack on head/neck 19 (26%)
Drink fluids 13 (18%)
Transcranial magnetic stimulation 12 (16%)
Biofeedback 11 (15%)
Acupuncture 10 (14%)
Mindfulness 9 (12%)
Massage 9 (12%)
GammaCore nVS 8 (11%)
Yoga 8 (11%)
Meal 7 (10%)
Herbs 6 (8%)
Acupressure 5 (7%)
Essential oils 5 (7%)
Cognitive behavioural therapy 4 (5%)
Transcutaneous supraorbital nerve stimulation 4 (5%)
Remote electrical neuromodulation (REN) 3 (4%)
Ginger 3 (4%)
Marijuana 1 (1%)
Pharmacological treatment 64 (88%)
NSAID 42 (58%)
Excedrin 40 (55%)
Triptans 37 (51%)
Paracetamol (Acetaminophen) 36 (49%)
Anti-emetics 24 (33%)
Ergot alkaloids 22 (30%)
Opioids 14 (19%)
Barbiturates 9 (12%)
Anti CGRP small molecules 9 (12%)
Vitamins and minerals 7 (10%)
Isometheptene mucate 7 (10%)
Lasmiditan 7 (10%)
Steroid injection (prednisone or dexamethasone) 7 (10%)
Sphenopalatine ganglion block 3 (4%)
Pseudoephedrine 1 (1%)
Preventive treatment 49 (67%)
Non-pharmacological 46 (63%)
Avoid triggers 38 (52%)
Sleep 29 (40%)
Meal 28 (38%)
Relaxation 26 (36%)
Exercise 26 (36%)
Drink fluids 22 (30%)
Herbs 17 (23%)
Acupuncture 16 (22%)
Biofeedback 14 (19%)
(continued)
1638 Cephalalgia 40(14)

Table 2. Continued.
Number of websites
Recommendation n ¼ 73 (100%)

Transcranial magnetic stimulation 12 (16%)

Transcutaneous electrical nerve stimulation 12 (16%)
Cognitive behavioral therapy 7 (10%)
Surgical treatment 5 (7%)
Massage 5 (7%)
GammaCore nVS 3 (4%)
Marijuana 2 (3%)
Focused ultrasound 1 (1%)
Migraine glasses 1 (1%)
Pharmacological treatment 40 (55%)
Botox 30 (41%)
Beta blockers 26 (36%)
Anti-CGRP monoclonal antibodies 25 (34%)
Tricyclic antidepressants 24 (33%)
Anticonvulsants 24 (33%)
Vitamins and minerals 19 (26%)
L-type calcium channel blockers 18 (25%)
Angiotensin II blockers 10 (14%)
Anti-CGRP small molecules 7 (10%)
Serotonin receptor antagonists 7 (10%)
Triptans* 7 (10%)
NSAIDs* 6 (8%)
Hormonal contraceptives* 5 (7%)
SNRI 5 (7%)
SSRI 4 (5%)
Melatonin 3 (4%)
NSAIDs 2 (3%)
MAOIs 1 (1%)
Quetiapine 1 (1%)
Tonabersat 1 (1%)
Transient receptor potential vanilloid type 1 antagonists 1 (1%)
Nitric oxide synthase inhibitors 1 (1%)
NMDA receptor antagonists 1 (1%)
Gap junction blockers 1 (1%)
*These were specific recommendations for menstrual-related migraine.

corporation websites (15%), 10 non-profit organiza- Health information corporations also cover the
tions (14%), nine news websites (12%), seven articles largest fraction of total acute treatments with 34
from peer-reviewed journals (10%), five governmental out of 35 options (97%). Health information corpora-
information service websites (7%), and four biotech tion and non-profit organizations cover an
company websites (5%). equal number of acute pharmacological treatments
In total, 77 different migraine treatment options with 14 out of 15 options (93%). Regarding preventive
were recommended across all eligible websites. Thirty- treatment recommendations, health information
five options (45%) were acute treatment recommenda- corporations covers the largest total number of preven-
tions and 42 (55%) were preventive treatments. Of the tive treatments with 41 out of 42 (98%). Furthermore,
acute treatments, 20 (57%) were non-pharmacological health information corporations cover the largest
and 15 (43%) were pharmacological. Out of the 42 fraction of both non-pharmacological preventive
preventive treatment options, 18 (43%) were non- treatments with 17 out of 18 (94%) options
pharmacological, whereas the remaining 24 (57%) and pharmacological preventives with 24 out of
were pharmacological. 24 (100%).
Health information corporations cover the largest Table 3 is an overview of the respective information
fraction of acute non-pharmacological treatments provider types and contains examples of websites in
with 20 out of 20 (100%) recommended options. each category.
Bojazar et al. 1639

Distribution of health information providers

Biotech company, n = 4, 5%
Governmental information service, n = 5, 7%
E-commerce website, n = 16, 22%

Peer-reviewed journal, n = 7, 10%

News, n = 9, 12%
Medical professionals, n = 11, 15%

Non-profit organization, n = 10, 14%

Health information corporation, n = 11, 15%

Figure 3. Distribution of health information providers. This pie chart showing the distribution of health information providers found
in the processed web search results. The total number of web search results after removal of duplicates and a defective link is 73.

Table 3. Overview of information provider types. Table 3 shows the defined information provider types with examples of websites.
The information provider types of webpages are derived from the websites’ “About us” sections, where websites describe their
purpose. Websites with identical purposes are grouped together.

Website category Example of websites

Biotech company Allergan, eNeura, etc.

E-commerce website Amazon, Walmart, etc.
Governmental information Food and Drug Administration, National Health Service, etc.
service
Health information corporation Healthline, WebMD, etc.
Medical professionals MayoClinic, Southern Cross, etc.
News ABC News, The Washington Post, etc.
Non-profit organization The Migraine Trust, Headache Australia, etc.
Peer-reviewed journal Articles in The Journal of Headache and Pain, Headache, The Journal of Head and Face Pain etc.

Table 4 lists the remaining website categories 4.1 (
0.22). Medical professionals had the highest
and corresponding percentage of treatment mean DISCERN score for Section 2, which rates qual-
recommendations. ity of information, with a rating of 3.8 (
0.17).
Statistical significance of DISCERN scores were estab-
lished between groups with a one-way ANOVA.
Quality of websites on treatment options
Differences between categories were significant in
Figure 4 shows the quantification of reliability and Section 1, regarding reliability (p < 0.001); Section 2,
quality of conveyed health information on treatment regarding quality of content (p ¼ 0.029), and mean
using the DISCERN instrument. The figure reports total DISCERN score (p < 0.001).
consensus results of DISCERN between raters. Non- We quantified the level of agreement between raters
profit organizations had the highest mean total by weighted Cohen’s kappa coefficient due to the ordi-
DISCERN score of 3.8 (
0.19). Non-profit organiza- nal nature of the DISCERN scale. The estimated
tions had the highest mean DISCERN score for agreement between raters was kappa ¼ 0.489, 95% CI
Section 1, which addresses reliability, with a rating of (0.448, 0.530), p < 0.001.
1640 Cephalalgia 40(14)

Table 4. Stratification of internet search results by provider type. The table shows website categories and how many treatment
options they cover per type of treatment. Treatment types entails acute/preventive and pharmacological/non-pharmacological. Data
are presented as number of treatments covered and percentage of total number of treatment options – for example, n (percentage).
Website categories are listed in alphabetical order.

Acute Preventive

Non- Non-
Website category Total pharmacological Pharmacological Total pharmacological Pharmacological

Biotech company 10 (29%) 9 (45%) 1 (7%) 9 (21%) 7 (39%) 2 (8%)

E-commerce website 16 (46%) 7 (35%) 9 (60%) 19 (45%) 12 (67%) 7 (29%)
Governmental information service 12 (34%) 7 (35%) 5 (33%) 20 (48%) 10 (56%) 10 (42%)
Health information corporation 34 (97%) 20 (100%) 14 (93%) 41 (98%) 17 (94%) 24 (100%)
Medical professionals 27 (77%) 15 (75%) 12 (80%) 28 (67%) 13 (72%) 15 (63%)
News 14 (40%) 8 (40%) 6 (40%) 14 (33%) 9 (50%) 5 (21%)
Non-profit organisation 30 (86%) 16 (80%) 14 (93%) 30 (71%) 16 (89%) 14 (58%)
Peer-reviewed journal 18 (51%) 5 (25%) 13 (87%) 22 (52%) 8 (44%) 14 (58%)

(a) (b) Mean DISCERN score for Section 2

Mean DISCERN score for Section 1
Biotech company 3.1 (±0.43)
Biotech company 2.4 (±0.35)
E-commerce website 2.6 (±0.24)
E-commerce website 2.2 (±0.26)
Governmental information service 3.7 (±0.26)
Governmental information service 3.8 (±0.26)
Health information corporation 3.5 (±0.15)
Health information corporation 3.5 (±0.2)
Medical professionals 3.8 (±0.17)
Medical professionals 3.7 (±0.21)
News 3.2 (±0.15)
News 3 (±0.11)
Non-profit organization 3.6 (±0.12)
Non-profit organization 4.1 (±0.22)
1 2 3 4 5
1 2 3 4 5

(c) Mean total DISCERN score

Biotech company 2.7 (±0.41)

E-commerce website 2.3 (±0.24)

Governmental information service 3.7 (±0.24)

Health information corporation 3.5 (±0.17)

Medical professionals 3.7 (±0.18)

News 3.1 (±0.13)

Non-profit organization 3.8 (±0.19)

1 2 3 4 5

Figure 4. DISCERN scores of consumer health information grouped by website category. These figures provide an overview of
mean DISCERN scores according to each section of the questionnaire. (a) Scores for Section 1 (reliability; questions 1 through 8), (b)
scores for Section 2 (quality of content; questions 9 through 15), (c) total DISCERN score (all 16 questions). Data shows mean (

standard error of the mean) to the right side of each column for category of information provider type. The reported results
are means of consensus scores between raters. The DISCERN tool rates information quality, scoring from 1 to 5, where 1 is low
quality, 3 is partial quality, and 5 is high quality information. Columns are arranged in alphabetical order. Statistical significance of
DISCERN scores were established between groups with a one-way ANOVA. Differences between categories were significant in
Section 1, regarding reliability (p < 0.001); Section 2, regarding quality of content (p ¼ 0.029); and mean total DISCERN score
(p < 0.001).
Bojazar et al. 1641

Discussion Botox is the most frequently covered preventive

pharmacological treatment, followed by beta blockers
The current study investigated migraine treatment
and anti-CGRP monoclonal antibodies. Concurrently,
information on the internet. The highest indexed web-
conventional preventive therapeutics like beta blockers
sites recommended a total of 77 acute and preventive
and tricyclic antidepressants are still frequently recom-
treatment options, both non-pharmacological and
mended, which may be because they are first choice
pharmacological. Health information corporation is
preventives in medical practices (15).
the health information provider type which covers the
Online health information on migraine treatments
broadest number of treatments. Non-profit organiza-
covers an extensive number of treatment options,
tions comprise the most reliable provider type, whereas which can be problematic for patients seeking health
medical professionals provide the highest quality of information online. Considering the estimated
information. Non-profit organizations also had the level of health literacy (17), most online seekers
highest mean total DISCERN score. of health care information may not have the tools
to evaluate and compare different treatment
Migraine management options, therefore the high number of treatments rec-
Ranking of treatment recommendations shows that ommended by websites may complicate patient’s self-
treatment modalities of anecdotal evidence rank high- care (18).
est among acute non-pharmacological options. Furthermore, the high number of non-validated
Relaxation, sleep, and placing ice packs on the head treatment recommendations can disrupt the overlap
are ranked highest among acute non-pharmacological between health care professionals’ and patients’ scope
treatments. These treatments are cheap and easily of interpretation of optimal migraine treatment.
accessible compared to for example medical Inability to eliminate discrepancies between the treat-
devices. In addition, the nature of the aforementioned ment expectations of patients and healthcare professio-
treatments is based on anecdotal evidence, which may nals can lead to suboptimal treatments (18).
be why they are highly indexed on Google, as
these treatments are generally favoured by patients Information provider type
(12). E-commerce websites constitute the largest group of
Among acute pharmacological treatments, the most health information providers of migraine treatments.
frequently reported treatments are non-migraine spe- This is perhaps due to ExcedrinVR being frequently
cific treatments such as non-steroidal anti-inflammato- entered in migraine-related searches on the internet,
ry drugs (NSAIDs) rather than migraine-specific likely derived from non-migraine specific over-the-
therapeutics like triptans. NSAIDs are available as counter analgesics being widely used as acute
over-the-counter analgesics and cheaper in most migraine relief medication (19). The high search
countries, which may make them preferred over volume index of e-commerce websites can be suggestive
migraine-specific drugs (13,14). Fourteen websites of migraineurs using Google in search of acute relief.
(19%) mention opioids as a treatment option, and The abrupt increase in searches on ExcedrinVR
despite specifying opioids as a last-resort option, the around January of 2012 is likely due to the recall of
high number of websites covering opioids in the con- the drug by the manufacturer at the time (20). The
text of migraine is problematic. Opioids are potentially manufacturer at the time recalled all lots because of
addictive and not recommended in any guidelines reports of chipped pills and faulty packaging, which
(15,16). Considering that 64 websites (88%) recom- could result in bottles containing foreign tablets, cap-
mend pharmacological treatment, as well as 49% of lets or capsules (20).
websites only recommends pharmacological options, Furthermore, Google is often used as a gateway to
it is noteworthy that only 45 websites (62%) caution e-commerce websites across website types (21). Over-
against medication overuse headache, highlighting a the-counter treatment options are therefore readily
major challenge in online-based patient education. available for patients to buy directly from the internet
Among preventive treatments, non-pharmacological without a prescription (22). Restricted access to health-
treatments are most frequently covered, with avoidance care, for example in a private healthcare setting, could
of triggers as the most frequent recommendation, fol- incite patients to seek available over-the-counter treat-
lowed by sleep, meals and relaxation. As with acute ments instead of consulting a professional. Taken
treatments, cheap and easily applicable interventions together, this might explain the high indexation of
are favoured over migraine-specific products. This e-commerce websites.
may underline the previously described anecdotal Stratification analysis of treatment options with
nature of internet treatment recommendations. respect to provider types revealed that health
1642 Cephalalgia 40(14)

information corporations are providing the broadest for epidemiological data, but rather a depiction of
coverage of information on migraine treatment queries. We do not use Google searches as a surrogate
options. Health information corporations are for- for epidemiological data, therefore this limitation does
profit organizations, which entails a financial incite- not necessarily apply to our study. Furthermore,
ment for providing a large volume of information Google uses proprietary methods to determine index-
with opportunities for both advertisements and spon- ation of websites. The degree of variability, and thereby
sored content on websites. Though this can be found generalisability, was not assessed in this study, due to
on such websites, it is specified per advertisement. the searches being conducted on a single computer in
Furthermore, high information volume can favour Greater Copenhagen, Denmark. Although we cannot
websites in terms of high Google search result ranking, exclude regional differences in a Google search, we
as there is potentially a broader coverage of search believe the clearance of temporary internet files and
term matches (10). strict use of English search terms will limit potential
differences.
Quality of websites This is a cross-sectional study of internet
websites while the content is constantly indexed.
Non-profit organizations had the highest mean total
However, this is an inherent limitation in studies of
DISCERN score. When solely observing Section 1 of
online content.
the DISCERN tool, which addresses the reliability of
The DISCERN tool for evaluation of written con-
the conveyed message, non-profit organizations score
sumer health information does not require an assess-
highest with a mean score of 4.1 (
0.22).
ment of interrater reliability. Nevertheless, we
The DISCERN instrument favours information that
quantified the level of agreement between raters by
is conveyed with clear purpose, framework, and multi-
weighted Cohen’s kappa coefficient due to the ordinal
ple solutions. This might explain why non-profit organ-
nature of the DISCERN scale. The estimated agree-
isations obtain the highest score in Section 1 and why
ment between raters was kappa ¼ 0.489, 95% CI
medical professionals did not, despite obtaining the
(0.448, 0.530), p < 0.001. The estimated kappa can be
highest mean score on quality of information. Non-
interpreted as moderate strength of agreement (26,27)
profit organisations scored highest in Section 1 which,
and is similar to the strength of agreement of the
when added with scores from Section 2, resulted in the
founding expert panel of the DISCERN tool (7).
highest mean total DISCERN score. Our results may
Estimations of interrater reliability on online informa-
suggest that, among the highest indexed websites, med-
tion is elusive as the content is heterogeneous. This
ical professionals do not prioritise the framework of a
message despite being an important factor in patient emphasised the need for the two raters to settle
education (23). any disagreement in rating by re-assessing the website
The discrepancy of health information quality in question.
between different websites is similar to previous find- When assessing the content of websites, the authors
ings in other medical conditions with analysis and eval- were not blinded to the publisher of the websites, which
uation of content (24,25). This suggests that some can constitute a source of bias. However, extraction of
challenges in online health information is similar data from the website in a blinded manner (e.g. web-
across medical conditions. scraping) is considered a violation of the terms of use
Websites recommending treatment options based of the websites (28,29). Furthermore, the purpose
on anecdotal evidence rank relatively high in of the study is to assess the results from googling
DISCERN scoring. This is consistent with health migraine, therefore content presentation is included in
information consumers heavily weighing anecdotal this scope.
information for treatment choices (12). Websites with
information on migraine treatment can obtain a high Conclusion
DISCERN score regardless of the evidence behind rec-
Googling when looking for migraine treatments reveals
ommendation of the treatment, which might explain
a myriad of management recommendations of varying
why conventional evidence-based approaches are not
quality. Patients use the internet in the context of self-
favoured.
treatment before consulting healthcare professionals,
which should incentivise health care professionals, gov-
Strengths and limitations ernmental service providers and other stakeholders to
Google Trends is a search string compositor based on provide reliable information of high quality. We
monitoring of Google entries. It is not a proxy marker encourage stakeholders to optimise the distribution of
Bojazar et al. 1643

high-quality and peer-reviewed information on scientific society; for example, The International
migraine management, possibly in the context of a Headache Society.

Key findings
• Among acute treatments, pharmacological treatments are most frequently covered and most commonly
non-migraine specific treatments such as non-steroidal anti-inflammatory drugs (NSAIDs) rather than
migraine-specific therapeutics like triptans.
• Among preventive treatments, non-pharmacological treatments are most frequently covered, with avoid-
ance of triggers as the most frequent recommendation, followed by sleep, meals and relaxation.
• E-commerce websites dominate the highest-ranked indexed migraine-related websites on Google.

Declaration of conflicting interests Migraine Foundation, International Headache Society,

The authors disclosed receipt of the following financial sup- Canadian Headache Society.
port for the research, authorship, and/or publication of this MA reports personal fees from Allergan, Amgen, Alder,
article: RB, TPD, JMH report no conflicts of interest. DWD Eli Lilly, Novartis and Teva. He participated in clinical trials
reports the following conflicts: Personal fees: Amgen, AEON, as the principal investigator for Alder, Amgen, ElectroCore,
Association of Translational Medicine, University Health Novartis and Teva trials; he has no ownership interest and
Network, Daniel Edelman Inc., Autonomic Technologies, does not own stocks of any pharmaceutical company, and
Axsome, Aural Analytics, Allergan, Alder also serves as associated editor of Cephalalgia, Headache,
BioPharmaceuticals, Biohaven, Charleston Laboratories, and co-editor of the Journal of Headache and Pain. MA is
Clexio, Dr Reddy’s Laboratories/Promius, Electrocore the President of the International Headache Society and
LLC, Eli Lilly, eNeura, Neurolief, Novartis, Ipsen, Impel, General Secretary of the European Headache Federation.
Satsuma, Supernus, Sun Pharma (India), Theranica, Teva,
Vedanta, WL Gore, Nocira, PSL Group Services, Funding
University of British Columbia, XoC, Zosano, ZP Opco, The authors disclosed receipt of the following financial sup-
Foresite Capital, Oppenheimer; Upjohn (Division of Pfizer), port for the research, authorship, and/or publication of this
Pieris, Revance, Equinox, Salvia, Amzak Health. Speaking article: The Lundbeck Foundation funded the publication of
fees: Eli Lilly, Novartis Canada, Amgen. Speakers Bureaus: this article.
None. CME fees or royalty payments: HealthLogix,
Medicom Worldwide, MedLogix Communications, Mednet, ORCID iD
Miller Medical, PeerView, WebMD Health/Medscape, Rachid Bojazar https://orcid.org/0000-0001-9565-483X
Chameleon, Academy for Continued Healthcare Learning,
Universal Meeting Management, Haymarket, Global References
Scientific Communications, Global Life Sciences, Global 1. Viana M, Khaliq F, Zecca C, et al. Poor patient aware-
Access Meetings, UpToDate (Elsevier), Oxford University ness and frequent misdiagnosis of migraine: Findings
Press, Cambridge University Press, Wolters Kluwer Health; from a large transcontinental cohort. Eur J Neurol
Stock options: Precon Health, Aural Analytics, Healint, 2019; 27: 536–541.
Theranica, Second Opinion/Mobile Health, Epien, Nocira, 2. McMullan M. Patients using the Internet to obtain health
Matterhorn/Ontologics, King-Devick Technologies; information: How this affects the patient-health profes-
Consulting without fee: Aural Analytics, Healint, Second sional relationship. Patient Educ Couns 2006; 63: 24–28.
Opinion/Mobile Health, Epien; Board of Directors: Epien, 3. Emanuel EJ and Emanuel LL. Four models of the
Matterhorn/Ontologics, King-Devick Technologies. Patent: physician-patient relationship. JAMA J Am Med Assoc
1992; 267: 2221–2226.
17189376.1-1466:vTitle: Botulinum Toxin Dosage Regimen
4. Google. How Google Search Works | Search Algorithms,
for Chronic Migraine Prophylaxis without fee; Research
https://www.google.com/search/howsearchworks/algo
funding: American Migraine Foundation, US Department rithms/ (2019, accessed 23 March 2020).
of Defense, PCORI, Henry Jackson Foundation; 5. Google. How Google Search Works | Crawl and
Professional society fees or reimbursement for travel: Indexing, https://www.google.com/search/howsearch
American Academy of Neurology, American Brain works/crawling-indexing/ (2013, accessed 23 March
Foundation, American Headache Society, American 2020).
1644 Cephalalgia 40(14)

6. Kim P, Eng TR, Deering MJ, et al. Review of published 19. Seng EK, Robbins MS and Nicholson RA. Acute
criteria for evaluating health-related websites. West J migraine medication adherence, migraine disability and
Med 1999; 170: 329–332. patient satisfaction: A naturalistic daily diary study.
7. Charnock D, Shepperd S, Needham G, et al. DISCERN: Cephalalgia 2017; 37: 955–964.
An instrument for judging the quality of written consum- 20. US Food & Drug Administration. Safety alerts for human
er health information on treatment choices. J Epidemiol medical products > Novartis consumer health over-the-
Commun Health 1999; 53: 105–111. counter products: Recall – potential presence of foreign
8. Charnock D and Shepperd S. Learning to DISCERN tablets or chipped or broken tablets or gelcaps, https://
online: Applying an appraisal tool to health websites in wayback.archive-it.org/7993/20170112131013/http://www.
a workshop setting. Health Educ Res 2004; 19: 440–446. fda.gov/Safety/MedWatch/SafetyInformation/SafetyAlert
9. Google. Google Trends, https://trends.google.com/ sforHumanMedicalProducts/ucm286265.htm (2012, acces-
trends/?geo=US (2006, accessed 25 November 2019). sed 29 November 2019).
10. Google. Trends Help, https://support.google.com/trends/ 21. Stucke ME and Grunes AP. Big data and competition
#topic=4365530 (2020, accessed 25 November 2019). policy, 1st ed. Oxford University Press, 2016, pp. 1–11.
11. Eysenbach G and K€ ohler C. How do consumers search 22. Bessell TL, Silagy CA, Anderson JN, et al. Quality of
for and appraise health information on the world wide global e-pharmacies: Can we safeguard consumers? Eur
web? Qualitative study using focus groups, usability tests, J Clin Pharmacol 2002; 58: 567–572.
and in-depth interviews. Br Med J 2002; 324: 573–577. 23. Cady R, Farmer K, Beach ME, et al. Nurse-based edu-
12. Fagerlin A, Wang C and Ubel PA. Reducing the influ- cation: An office-based comparative model for education
ence of anecdotal reasoning on people’s health care deci- of migraine patients. Headache 2008; 48: 564–569.
sions: Is a picture worth a thousand statistics? Med Decis 24. Fisher JH, O’Connor D, Flexman AM, et al. Accuracy
Making 2005; 25: 398–405. and reliability of internet resources for information on
13. Von Seggern RL and Adelman JU. Cost considerations idiopathic pulmonary fibrosis. Am J Respir Crit Care
in headache treatment part 2: Acute migraine treatment. Med 2016; 194: 218–225.
Headache 1996; 36: 493–502. 25. Hirsch M, Aggarwal S, Barker C, et al. Googling endo-
14. Tfelt-Hansen P and Steiner TJ. Over-the-counter triptans metriosis: A systematic review of information available
for migraine: What are the implications? CNS Drugs on the Internet. Am J Obstet Gynecol 2017; 216: 451–458.
2007; 21: 877–883. 26. Cohen J. Weighted kappa: Nominal scale agreement pro-
15. Steiner TJ, Jensen R, Katsarava Z, et al. Aids to man- vision for scaled disagreement or partial credit. Psychol
agement of headache disorders in primary care (2nd edi- Bull 1968; 70: 213–220.
tion). J Headache Pain 2019; 20: 57. 27. Fleiss JL. The measurement of interrater agreement. In

16. Evers S, Afra J, Frese A, et al. EFNS guideline on the Statistical methods for rates and proportions, 2nd edn.
drug treatment of migraine – Revised report of an EFNS New York: John Wiley, 1981, pp.212–236.
task force. Eur J Neurol 2009; 16: 968–981. 28. Healthline Media Inc. Terms of Use – Healthline, https://
17. Goodhew SC. Migraine literacy and treatment in a uni- www.healthline.com/terms-of-use (2020, accessed 24
versity sample. SN Compr Clin Med 2019; 1: 749–757. March 2020).
18. Bussey LG and Sillence E. The role of internet resources 29. WebMD. Terms and conditions of use,https://www.
in health decision-making: A qualitative study. Digit webmd.com/about-webmd-policies/about-terms-and-con
Heal 2019; 5; 1–5. DOI: 10.1177/2055207619888073. ditions-of-use (2019, accessed 24 March 2020).