article

Quality of health information for consumers on the web: A systematic review of indicators, criteria, tools, and evaluation results

Authors:

Bo XieAuthors Info & Claims

Journal of the Association for Information Science and Technology, Volume 66, Issue 10

Pages 2071 - 2084

https://doi.org/10.1002/asi.23311

Published: 01 October 2015 Publication History

Abstract

The quality of online health information for consumers has been a critical issue that concerns all stakeholders in healthcare. To gain an understanding of how quality is evaluated, this systematic review examined 165 articles in which researchers evaluated the quality of consumer-oriented health information on the web against predefined criteria. It was found that studies typically evaluated quality in relation to the substance and formality of content, as well as to the design of technological platforms. Attention to design, particularly interactivity, privacy, and social and cultural appropriateness is on the rise, which suggests the permeation of a user-centered perspective into the evaluation of health information systems, and a growing recognition of the need to study these systems from a social-technical perspective. Researchers used many preexisting instruments to facilitate evaluation of the formality of content; however, only a few were used in multiple studies, and their validity was questioned. The quality of content i.e., accuracy and completeness was always evaluated using proprietary instruments constructed based on medical guidelines or textbooks. The evaluation results revealed that the quality of health information varied across medical domains and across websites, and that the overall quality remained problematic. Future research is needed to examine the quality of user-generated content and to explore opportunities offered by emerging new media that can facilitate the consumer evaluation of health information.

References

[1]

Abbott, V.P. 2000. Web page quality: can we measure it and what do we find? A report of exploratory findings. Journal of Public Health Medicine, Volume 22 Issue 2, pp.191-197.

[2]

Arora, N.K., Hesse, B.W., Rimer, B.K., Viswanath, K., Clayman, M.L., &Croyle, R.T. 2008. Frustrated and confused: The American public rates its cancer-related information-seeking experiences. Journal of General Internal Medicine, Volume 23 Issue 3, pp.223-228.

[3]

Bailey, M.A., Coughlin, P.A., Sohrabi, S., Griffin, K.J., Rashid, S.T., Troxler, M.A. ' 2012. Quality and readability of online patient information for abdominal aortic aneurysms. Journal of Vascular Surgery, Volume 56 Issue 1, pp.21-26.

[4]

Barnes, C., Harvey, R., Wilde, A., Hadzi-Pavlovic, D., Wilhelm, K., &Mitchell, P.B. 2009. Review of the quality of information on bipolar disorder on the Internet. Australian & New Zealand Journal of Psychiatry, Volume 43 Issue 10, pp.934-945.

[5]

Bernstam, E.V., Sagaram, S., Walji, M., Johnson, C.W., &Meric-Bernstam, F. 2005. Usability of quality measures for online health information: Can commonly used technical quality criteria be reliably assessed? International Journal of Medical Informatics, Volume 74 Issue 7-8, pp.675-683.

[6]

Bernstam, E.V., Shelton, D.M., Walji, M., &Meric-Bernstam, F. 2005. Instruments to assess the quality of health information on the World Wide Web: What can our patients actually use? International Journal of Medical Informatics, Volume 74 Issue 1, pp.13-19.

[7]

Brunswik, E. 1952. The conceptual framework of psychology. Chicago: University of Chicago Press.

[8]

Car, J., Lang, B., Colledge, A., Ung, C., &Majeed, A. 2011. Interventions for enhancing consumers' online health literacy. The Cochrane Database of Systematic Reviews 2011, Issue 6. Article No. CD007092.

[9]

Charnock, D. 1998. The DISCERN handbook: Quality criteria for consumer health information on treatment choices. Retrieved from: "http://www.discern.org.uk/discern.pdf"

[10]

Chestnutt, I.G. 2002. The nature and quality of periodontal related patient information on the world-wide web. British Dental Journal, Volume 193 Issue 11, pp.657-659; discussion 643.

[11]

Chou, W., Prestin, A., Lyons, C., &Wen, K. 2013. Web 2.0 for health promotion: Reviewing the current evidence. American Journal of Public Health, Volume 103 Issue 1, pp.e9-18.

[12]

Cline, R.J.W., &Haynes, K.M. 2001. Consumer health information seeking on the Internet: the state of the art. Health Education Research, Volume 16 Issue 6, pp.671-692.

[13]

Craigie, M., Loader, B., Burrows, R., &Muncer, S. 2002. Reliability of health information on the Internet: an examination of experts' ratings. Journal of Medical Internet Research, Volume 4 Issue 1, pp.e2. Available at: "http://www.jmir.org/2002/1/e2"/

[14]

Crocco, A.G., Villasis-Keever, M., &Jadad, A.R. 2002. Analysis of cases of harm associated with use of health information on the internet. JAMA, Volume 287 Issue 21, pp.2869-2871.

[15]

Eysenbach, G. 2008. Medicine 2.0: Social networking, collaboration, participation, apomediation, and openness. Journal of Medical Internet Research, Volume 10 Issue 3. Available at: "http://www.jmir.org/2008/3/e22"/

[16]

Eysenbach, G., &Diepgen, T.L. 1998. Towards quality management of medical information on the internet: evaluation, labelling, and filtering of information. BMJ, Volume 317 Issue 7171, pp.1496-1502.

[17]

Eysenbach, G., &Kohler, C. 2002. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ, Volume 324 Issue 7337, pp.573-577.

[18]

Eysenbach, G., Powell, J., Kuss, O., &Sa, E.-R. 2002. Empirical studies assessing the quality of health information for consumers on the World Wide Web. JAMA, Volume 287 Issue 20, pp.2691-2700.

[19]

Feufel, M.A., &Stahl, S.F. 2012. What do Web-use skill differences imply for online health information searches? Journal of Medical Internet Research, Volume 14 Issue 3, pp.e87. Available at: "http://www.jmir.org/2012/3/e87"/

[20]

Fogg, B.J. 2003. Persuasive technology: Using computers to change what we think and do. San Franciso: Morgan Kaufmann.

Digital Library

[21]

Fox, S. 2011. The social life of health information. Retrieved from: "http://www.pewinternet.org/files/old-media/Files/Reports/2011/PIP_Social_Life_of_Health_Info.pdf"

[22]

Gagliardi, A., &Jadad, A.R. 2002. Examination of instruments used to rate quality of health information on the internet: Chronicle of a voyage with an unclear destination. BMJ, Volume 324 Issue 7337, pp.569-573.

[23]

Griffiths, K.M., Tang, T.T., Hawking, D., &Christensen, H. 2005. Automated assessment of the quality of depression Websites. Journal of Medical Internet Research, Volume 7 Issue 5, pp.e59. Available at: "http://www.jmir.org/2005/5/e59"/

[24]

Hamm, M.P., Chisholm, A., Shulhan, J., Milne, A., Scott, S.D., Given, L.M. ' 2013. Social media use among patients and caregivers: A scoping review. BMJ open, Volume 3 Issue 5, e002819.

[25]

Harland, J.P. 2007. Assessing the quality of websites providing information on multiple sclerosis: Evaluating tools and comparing sites. Health Informatics Journal, Volume 13 Issue 3, pp.207-221.

[26]

Hesse, B.W., &Shneiderman, B. 2007. eHealth research from the user's perspective. American Journal of Preventive Medicine, Volume 32 Issue 5 Suppl, pp.S97-103.

[27]

Health On the Net Foundation. 2013. The HON Code of Conduct for medical and health web sites HONcode. Accessed from: "http://www.hon.ch/HONcode/Conduct.html"

[28]

Hedman, A.S. 2008. Using the SMOG formula to revise a health-related document. American Journal of Health Education, Volume 39 Issue 1, pp.61-64.

[29]

Hendrick, P.A., Ahmed, O.H., Bankier, S.S., Chan, T.J., Crawford, S.A., Ryder, C. R. ' 2012. Acute low back pain information online: An evaluation of quality, content accuracy and readability of related websites. Manual Therapy, Volume 17 Issue 4, pp.318-324.

[30]

Hogarth, R.M. 1987. Judgement and choice 2 ed. New York: John Wiley & Sons.

[31]

Hwang, K.O., Farheen, K., Johnson, C.W., Thomas, E.J., Barnes, A. S., &Bernstam, E.V. 2007. Quality of weight loss advice on Internet forums. The American Journal of Medicine, Volume 120 Issue 7, pp.604-609.

[32]

Irwin, J.Y., Thyvalikakath, T., Spallek, H., Wali, T., Kerr, A. R., &Schleyer, T. 2011. English and Spanish oral cancer information on the Internet: A pilot surface quality and content evaluation of oral cancer Web sites. Journal of Public Health Dentistry, Volume 71 Issue 2, pp.106-116.

[33]

Jadad, A.R., &Gagliardi, A. 1998. Rating health information on the Internet: Navigating to knowledge or to Babel? JAMA, Volume 279 Issue 8, pp.611-614.

[34]

Kaicker, J., Wu, K., &Athreya, S. 2012. Assessing the reliability and quality of online uterine fibroid embolization resources. CardioVascular and Interventional Radiology, Volume 36 Issue 2, pp.385-394.

[35]

Khazaal, Y., Chatton, A., Cochand, S., Coquard, O., Fernandez, S., Khan, R. ' 2010. Quality of web-based information on alcohol dependence. Drugs: Education, Prevention, and Policy, Volume 17 Issue 3, pp.248-260.

[36]

Kim, P., Eng, T.R., Deering, M.J., &Maxfield, A. 1999. Published criteria for evaluating health related web sites: Review. BMJ, Volume 318 Issue 7184, pp.647-649.

[37]

Kincaid, J., Fishburne, R.P., Rogers, R.L., &Chissom, B.S. 1975. Derivation of new readability formulas Automated Readability Index, Fog Count, and Flesch Reading Ease formula for Navy enlisted personnel. Research Branch Report 8-75. Chief of Naval Technical Training: Naval Air Station, Memphis, TN.

[38]

Lam, C.G., Roter, D.L., &Cohen, K.J. 2013. Survey of quality, readability, and social reach of websites on osteosarcoma in adolescents. Patient Education and Counseling, Volume 90 Issue 1, pp.82-87.

[39]

Meric, F., Bernstam, E.V., Mirza, N.Q., Hunt, K.K., Ames, F.C., Ross, M.I. ' 2002. Breast cancer on the world wide web: cross sectional survey of quality of information and popularity of websites. BMJ, Volume 324 Issue 7337, pp.577-581.

[40]

Mousiolis, A., Michala, L., &Antsaklis, A. 2012. Polycystic ovary syndrome: double click and right check. What do patients learn from the Internet about PCOS? European Journal of Obstetrics & Gynecology and Reproductive Biology, Volume 163 Issue 1, pp.43-46.

[41]

Nasser, S., Mullan, J., &Bajorek, B. 2012. Assessing the quality, suitability and readability of internet-based health information about warfarin for patients. Australasian Medical Journal, Volume 5 Issue 3, pp.194-203.

[42]

Ni Riordain, R., &McCreary, C. 2009. Head and neck cancer information on the internet: Type, accuracy and content. Oral Oncology, Volume 45 Issue 8, pp.675-677.

[43]

O'Grady, L. 2006. Future directions for depicting credibility in health care web sites. International Journal of Medical Informatics, Volume 75 Issue 1, pp.58-65.

[44]

O'Keefe, D.J. 2002. Persuasion: Theory and research 2nd ed. Thousand Oaks, CA: Sage Publications.

[45]

Perez-Lopez, F.R., &Roncero, G.R.P. 2006. Assessing the content and quality of information on the treatment of postmenopausal osteoporosis on the World Wide Web. Gynecological Endocrinology, Volume 22 Issue 12, pp.669-675.

[46]

Pew Research Center. December 16, 2013. Health fact sheet. Accessed from: "http://www.pewinternet.org/fact-sheets/health-fact-sheet/"

[47]

Post, R.E., &Mainous, A.G., 3rd. 2010. The accuracy of nutrition information on the Internet for type 2 diabetes. Archives of Internal Medicine, Volume 170 Issue 16, pp.1504-1506.

[48]

Risk, A., &Dzenowagis, J. 2001. Review of Internet health information quality initiatives. Journal of Medical Internet Research, Volume 3 Issue 4, pp.e28. Available at: "http://www.jmir.org/2001/4/e28"/

[49]

Sandvik, H. 1999. Health information and interaction on the internet: a survey of female urinary incontinence. BMJ Clinical Research ed., Volume 319 Issue 7201, pp.29-32.

[50]

Silberg, W.M., Lundberg, G.D., &Musacchio, R.A. 1997. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor-Let the reader and viewer beware. JAMA, Volume 277 Issue 15, pp.1244-1245.

[51]

Stanovich, K.E. 1999. Who is rational?: Studies of individual differences in reasoning. New York: Psychology Press.

[52]

Stinson, J.N., White, M., Breakey, V., Chong, A.L., Mak, I., Low, K.K. ' 2011. Perspectives on quality and content of information on the internet for adolescents with cancer. Pediatric Blood & Cancer, Volume 57 Issue 1, pp.97-104.

[53]

Tallgren, M., &Backlund, M. 2009. Patient information about general anaesthesia on the internet. Anaesthesia, Volume 64 Issue 4, pp.408-415.

[54]

Tan, B.H.L., Kostapanagiotou, K., &Jilaihawi, A.N.A. 2009. A review of mesothelioma information on the World Wide Web. Journal of Thoracic Oncology, Volume 4 Issue 1, pp.102-104.

[55]

Tavare, A.N., Alsafi, A., &Hamady, M.S. 2012. Analysis of the quality of information obtained about uterine artery embolization from the Internet. Cardiovascular and Interventional Radiology, Volume 35 Issue 6, pp.1355-1362.

[56]

Tseng, S., &Fogg, B.J. 1999. Credibility and computing technology. Communications of the ACM, Volume 42 Issue 5, pp.39-44.

Digital Library

[57]

Tsunehara, S.T., &Gaster, B. 2002. A critical assessment of diabetes information on the Internet. Diabetes Educator, Volume 28 Issue 2, pp.186.

[58]

Wallace, L.S., Turner, L.W., Ballard, J.E., Keenum, A.J., &Weiss, B.D. 2005. Evaluation of web-based osteoporosis educational materials. Journal of Women's Health, Volume 14 Issue 10, pp.936-945.

[59]

Wang, Y.L., &Liu, Z.K. 2007. Automatic detecting indicators for quality of health information on the Web. International Journal of Medical Informatics, Volume 76 Issue 8, pp.575-582.

[60]

Williams, J. 2010. Social networking applications in health care: Threats to the privacy and security of health information. In Proceedings of the 2010 ICSE Workshop on Software Engineering in Health Care SEHC, pp. pp.39-49.

Digital Library

[61]

Wilson, P. 2002. How to find the good and avoid the bad or ugly: A short guide to tools for rating quality of health information on the internet. BMJ, Volume 324 Issue 7337, pp.598-602.

[62]

Wyatt, J.C. 1997. Commentary: Measuring quality and impact of the world wide web. BMJ, Volume 314 Issue 7098, pp.1879-1879.

[63]

Yeo, H., Roman, S., Air, M., Maser, C., Trapasso, T., Kinder, B. ' 2007. Filling a void: Thyroid cancer surgery information on the internet. World Journal of Surgery, Volume 31 Issue 6, pp.1185-1191.

[64]

Zermatten, A., Khazaal, Y., Coquard, O., Chatton, A., &Bondolfi, G. 2010. Quality of web-based information on depression. Depression and Anxiety, Volume 27 Issue 9, pp.852-858.

[65]

Zhang, Y. 2012. Consumer health information searching process in real life settings. In Proceedings of the 75th Annual Conference of the American Society for Information Science & Technology ASIST'12, pp. pp.1-10.

[66]

Zhang, Y. 2014. Beyond quality and accessibility: Source selection in consumer health information searching. Journal of the Association for Information Science and Technology, Volume 65 Issue 5, pp.911-927.

Digital Library

[67]

Zun, L.S., Downey, L., &Brown, S. 2011. Completeness and accuracy of emergency medical information on the Web: Update 2008. Western Journal of Emergency Medicine, Volume 12 Issue 4, pp.448-454.

Cited By

Urman AHannak AMakhortykh M(2024)User Attitudes to Content Moderation in Web SearchProceedings of the ACM on Human-Computer Interaction10.1145/36374238:CSCW1(1-27)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637423
Xie WKang CXu LCheng HDai P(2024)Study on the quality evaluation of mobile social media health information and the relationship with health information disseminationInformation and Management10.1016/j.im.2024.10392761:3Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.im.2024.103927
Liu JZhang YKim Y(2023)Consumer Health Information Quality, Credibility, and Trust: An Analysis of Definitions, Measures, and Conceptual DimensionsProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578331(197-210)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578331
Show More Cited By

Recommendations

A model for online consumer health information quality

This article describes a model for online consumer health information consisting of five quality criteria constructs. These constructs are grounded in empirical data from the perspectives of the three main sources in the communication process: health ...
Quality of Antenatal Care at Rural and Urban Primary Health Centre in Jeneponto Regency
ICHSM '18: Proceedings of the International Conference on Healthcare Service Management 2018

This research aims to determine the different quality of antenatal care at urban and rural puskemas (public health center), Jeneponto regency. The type of the research was observational analysis with cross sectional design. There were 139 fixed samples ...
Validation of the Arizona Governor's Quality Award criteria: a test of the Baldrige criteria
Abstract
Tens of thousands of firms have used the Malcolm Baldrige National Quality Award (MBNQA) criteria to improve the quality of their products and processes. Because the MBNQA data are confidential, there has been no published assessment of the ...

Comments

Information & Contributors

Information

Published In

cover image Journal of the Association for Information Science and Technology

Journal of the Association for Information Science and Technology Volume 66, Issue 10

October 2015

212 pages

ISSN:2330-1635

EISSN:2330-1643

Issue’s Table of Contents

Publisher

John Wiley & Sons, Inc.

United States

Publication History

Published: 01 October 2015

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Urman AHannak AMakhortykh M(2024)User Attitudes to Content Moderation in Web SearchProceedings of the ACM on Human-Computer Interaction10.1145/36374238:CSCW1(1-27)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637423
Xie WKang CXu LCheng HDai P(2024)Study on the quality evaluation of mobile social media health information and the relationship with health information disseminationInformation and Management10.1016/j.im.2024.10392761:3Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.im.2024.103927
Liu JZhang YKim Y(2023)Consumer Health Information Quality, Credibility, and Trust: An Analysis of Definitions, Measures, and Conceptual DimensionsProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578331(197-210)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578331
Schultheiß S(2023)How search engine marketing influences user knowledge gainProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578297(475-478)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578297
Zhang YLiu JSong S(2023)The design and evaluation of a nudge‐based interface to facilitate consumers' evaluation of online health information credibilityJournal of the Association for Information Science and Technology10.1002/asi.2475974:7(828-845)Online publication date: 5-Jun-2023
https://dl.acm.org/doi/10.1002/asi.24759
Peng R(2022)How online searches fuel health anxietyComputers in Human Behavior10.1016/j.chb.2022.107384136:COnline publication date: 16-Aug-2022
https://dl.acm.org/doi/10.1016/j.chb.2022.107384
Chang TKao HWu JHsiao KChan T(2022)Integrated ontology-based approach with navigation and content representation for health care website designComputers in Human Behavior10.1016/j.chb.2021.107119128:COnline publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1016/j.chb.2021.107119
Zhang YTrace C(2022)The quality of health and wellness self‐tracking dataJournal of the Association for Information Science and Technology10.1002/asi.2459173:6(879-891)Online publication date: 26-Apr-2022
https://dl.acm.org/doi/10.1002/asi.24591
Ma DZuo MLiu L(2021)The Information Needs of Chinese Family members of Cancer Patients in the Online Health CommunityInformation Processing and Management: an International Journal10.1016/j.ipm.2021.10251758:3Online publication date: 1-May-2021
https://dl.acm.org/doi/10.1016/j.ipm.2021.102517
Chang YZhang YGwizdka J(2021)The effects of information source and eHealth literacy on consumer health information credibility evaluation behaviorComputers in Human Behavior10.1016/j.chb.2020.106629115:COnline publication date: 1-Feb-2021
https://dl.acm.org/doi/10.1016/j.chb.2020.106629
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents