Web Search or Conversation with an Artificial Intelligence? Analysis of Misinformation and Relevance in the Case of Radon Gas

  1. Noel Pascual-Presa 1
  2. Marcos Fernández-Pichel 2
  3. David E. Losada 2
  4. Berta García-Orosa 1
  1. 1 Universidade de Santiago de Compostela, Facultade de Ciencias da Comunicación
  2. 2 Universidade de Santiago de Compostela, Centro Singular de Investigacion en Tecnoloxías Intelixentes (CiTIUS)
Revista:
El profesional de la información

ISSN: 1386-6710,  1699-2407

Año de publicación: 2024

Volumen: 33

Número: 2

Tipo: Artículo

DOI: 10.3145/EPI.2024.0220 DIALNET GOOGLE SCHOLAR lock_openAcceso abierto editor

Otras publicaciones en: El profesional de la información

Resumen

Health-related information plays a crucial role in public health management, empowering individuals to make informed decisions and adopt behaviours that mitigate the effects of potential risks. The internet and the emergence of new technologies, such as conversational models equipped with Artificial Intelligence, present opportunities, and challenges in this field. This research focuses specifically on the risk of radon, a natural radioactive gas recognised worldwide as one of the leading causes of lung cancer and a persistent threat over time. The aim of this study is to analyse the information provided for thisspecific risk by two key information access tools: web search engines and AI-based conversational agents (ChatGPT). To carry out this interdisciplinary research (journalism-communication-computer science) we employ a mixed methodological design (quantitative and qualitative) and apply methods from the areas of Information Retrieval (IR), Big Data, and Artificial Intelligence (AI). The results of this study show that information on the internet about the risk of radon often lacks relevance and does not meetthe information needs of users. We also found that some websites provide a significant amount of good quality information but there are often some misleading contents. ChatGPT proves to be more accurate in providing relevant and good quality information but contains a higher proportion of misinformation. Consequently, this raises concerns about the integrity of the information provided and emphasises the need to monitor and improve the accuracy of these computational tools.

Referencias bibliográficas

  • Amante, Daniel J; Hogan, Timothy P; Pagoto, Sherry L; English, Thomas M; Lapane, Kate L(2015). "Access to care and use of the Internet to search for health information: results from the US National Health Interview Survey". Journal of Medical Internet Research,v. 17, n. 4, pp. e106. https://doi.org/10.2196/jmir.4126
  • Arslan, Sedat(2023). "Exploring the Potential of Chat GPT in Personalized Obesity Treatment". Annals of Biomedical Engineering,v. 51, n. 9, pp. 1887-1888. https://doi.org/10.1007/s10439-023-03227-9
  • Baeza-Yates, Ricardo; Ribeiro-Neto, Berthier (1999). Modern Information Retrieval. ACM Press New York. https://people.ischool.berkeley.edu/~hearst/irbook/print/chap10.ps.gz
  • Biswas, Som S. (2023a). "Potential Use of Chat GPT in Global Warming". Annals of Biomedical Engineering, v. 51, n. 6, pp. 1126-1127. https://doi.org/10.1007/s10439-023-03171-8
  • Biswas, Som S. (2023b). "Role of Chat GPT in Public Health". Annals of Biomedical Engineering, v. 51, n. 5, pp. 868-869. https://doi.org/10.1007/s10439-023-03172-7
  • Bujnowska-Fedak, Maria M.; Waligóra, Joanna; Mastalerz-Migas, Agnieszka (2019). "The Internet as a Source of Health Information and Services". In: Advancements and Innovations in Health Sciences. Pokorski, Mieczyslaw (Ed.), pp. 1-16. Springer International Publishing. https://doi.org/10.1007/5584_2019_396
  • Calixte, Rose; Rivera, Argelis; Oridota, Olutobi; Beauchamp, William; Camacho-Rivera, Marlene (2020). "Social and Demographic Patterns of Health-related Internet Use Among Adults in the United States: A Secondary Data Analysis of the Health Information National Trends Survey". International Journal of Environmental Research and Public Health, v. 17, n. 18, pp. 6856. https://doi.org/10.3390/ijerph17186856
  • ChatGPT (2023). "Differences between GPT-3.5 and GPT-4". ChatGPT Blog. https://chatgpt.ch/en/differences-between-gpt-3-5-and-gpt-4/
  • Chen, Yen-Yuan; Li, Chia-Ming; Liang, Jyh-Chong; Tsai, Chin-Chung (2018). "Health Information Obtained From the Internet and Changes in Medical Decision Making: Questionnaire Development and Cross-sectional Survey". Journal of Medical Internet Research, v. 20, n. 2, pp. e47. https://doi.org/10.2196/jmir.9370
  • Cheng, Christina; Dunn, Matthew (2015). "Health Literacy and the Internet: A Study on the Readability of Australian Online Health Information". Australian and New Zealand Journal of Public Health, v. 39, n. 4, pp. 309-314. https://doi.org/10.1111/1753-6405.12341
  • Costa-García, Álvaro; Okajima, Shotaro; Yang, Ningjia; Ueda, Sayako; Shimoda, Shingo (2022). "Current Trends and Challenges Towards the Digital Health Era". In: 2022 IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO), pp. 1-5. IEEE. https://doi.org/10.1109/ARSO54254.2022.9802962
  • Croft, W Bruce; Metzler, Donald; Strohman, Trevor (2010). Search Engines: Information Retrieval in Practice. Addison-Wesley Reading. https://ciir.cs.umass.edu/irbook
  • Eysenbach, Gunther; Köhler, Christian (2002). "How Do Consumers Search for and Appraise Health Information on the World Wide Web? Qualitative Study Using Focus Groups, Usability Tests, and In-depth Interviews". BMJ, v. 324, n. 7337, pp. 573-577. https://doi.org/10.1136/bmj.324.7337.573
  • Fallis, Don (2015). "What is Disinformation?". Library Trends, v. 63, n. 3, pp. 401-426. https://doi.org/10.1353/lib.2015.0014
  • Fernández-Pichel, Marcos; Meyer, Selina; Bink, Markus; Frummet, Alexander; Losada, David E; Elsweiler, David (2023). "Improving the Reliability of Health Information Credibility Assessments". In: CEUR Workshop Proceedings, pp. 43-50. https://ceur-ws.org/Vol-3406/paper4_jot.pdf
  • Fernández Villar, Alicia (2017). "El Mapa Español de Radón: Análisis de Factores Indicativos del Riesgo de Exposición al Radón". Tesis doctoral, Universidad de Cantabria. https://repositorio.unican.es/xmlui/handle/10902/11553
  • Feshina, Stella S.; Konovalova, Oksana V.; Sinyavsky, Nikolai G. (2019). "Industry 4.0—Transition to New Economic Reality". In: Industry 4.0: Industrial Revolution of the 21st Century. Popkova, Elena G.; Ragulina, Yulia V.; Bogoviz, Aleksei V. (Eds.), pp. 111-120. Springer International Publishing. https://doi.org/10.1007/978-3-319-94310-7_11
  • Finney Rutten, Lila J; Blake, Kelly D; Greenberg-Worisek, Alexandra J; Allen, Summer V; Moser, Richard P; Hesse, Bradford W (2019). "Online Health Information Seeking Among US Adults: Measuring Progress Toward a Healthy People 2020 Objective". Public Health Reports, v. 134, n. 6, pp. 617-625. https://doi.org/10.1177/0033354919874074
  • García-Orosa, B.; Forja-Pena, T. (2024). "Post COVID-19 Risk Communication in Digital Media: A Case Study of Radon Gas". In Press
  • García-Orosa, Berta (2021). "Disinformation, Social Media, Bots, and Astroturfing: The Fourth Wave of Digital Democracy". Profesional de la Información, v. 30, n. 6, pp. e300603. https://doi.org/10.3145/epi.2021.nov.03
  • Gutiérrez-Caneda, Beatriz; Vázquez-Herrero, Jorge; López-García, Xosé (2023). "AI Application in Journalism: ChatGPT and the Uses and Risks of an Emergent Technology". Profesional de la Información, v. 32, n. 5, pp. e320514. https://doi.org/10.3145/epi.2023.sep.14
  • Gutteling, Jan M; Wiegman, Oene (2013). Exploring Risk Communication. Springer Dordrecht. https://doi.org/10.1007/978-94-017-1523-2
  • Hermes-DeSantis, Evelyn R; Hunter, Robert T; Welch, Julie; Bhavsar, Roma; Boulos, Daniel; Noue, Marie-Ange (2021). "Preferences for Accessing Medical Information in the Digital Age: Health Care Professional Survey". Journal of Medical Internet Research, v. 23, n. 6, pp. e25868. https://doi.org/10.2196/25868
  • Hunsaker, Amanda; Hargittai, Eszter (2018). "A Review of Internet Use Among Older Adults". New Media & Society, v. 20, n. 10, pp. 3937-3954. https://doi.org/10.1177/1461444818787348
  • Jacobs, Wura; Amuta, Ann O; Jeon, Kwon Chan (2017). "Health Information Seeking in the Digital Age: An Analysis of Health Information Seeking Behavior Among US Adults". Cogent Social Sciences, v. 3, n. 1, pp. 1302785. https://doi.org/10.1080/23311886.2017.1302785
  • Jiang, Shaohai; Liu, Piper Liping (2020). "Digital Divide and Internet Health Information Seeking Among Cancer Survivors: A Trend Analysis from 2011 to 2017". Psycho-Oncology, v. 29, n. 1, pp. 61-67. https://doi.org/10.1002/pon.5247
  • Karako, Kenji; Song, Peipei; Chen, Yu; Tang, Wei (2023). "New Possibilities for Medical Support Systems Utilizing Artificial Intelligence (AI) and Data Platforms". BioScience Trends, v. 17, n. 3, pp. 186-189. https://doi.org/10.5582/bst.2023.01138
  • Klemm, Celine; Das, Enny; Hartmann, Tilo (2019). "Changed Priorities Ahead: Journalists’ Shifting Role Perceptions When Covering Public Health Crises". Journalism, v. 20, n. 9, pp. 1223-1241. https://doi.org/10.1177/1464884917692820
  • Krippendorff, Klaus (2018). Content Analysis: An Introduction to its Methodology. Sage Publications. https://doi.org/10.4135/9781071878781
  • Lagoe, Carolyn; Atkin, David (2015). "Health Anxiety in the Digital Age: An Exploration of Psychological Determinants of Online Health Information Seeking". Computers in Human Behavior, v. 52, pp. 484-491. https://doi.org/10.1016/j.chb.2015.06.003
  • Laugesen, John; Hassanein, Khaled; Yuan, Yufei (2015). "The Impact of Internet Health Information on Patient Compliance: A Research Model and an Empirical Study". Journal of Medical Internet Research, v. 17, n. 6, pp. e143. https://doi.org/10.2196/jmir.4333
  • Lin, Carolyn A; Atkin, David J; Cappotto, Celeste; Davis, Carrie; Dean, Julie; Eisenbaum, Jennifer; House, Kenyetta; Lange, Robert; Merceron, Alexandra; Metzger, Jacqueline (2015). "Ethnicity, Digital Divides and Uses of the Internet for Health Information". Computers in Human Behavior, v. 51, pp. 216-223. https://doi.org/10.1016/j.chb.2015.04.054
  • Masilamani, Vaageessan; Sriram, Arulchelvan; Rozario, Ann-Maria (2020). "eHealth Literacy of Late Adolescents: Credibility and Quality of Health Information Through Smartphones in India". Comunicar, v. 28, n. 64, pp. 85-95. https://doi.org/10.3916/C64-2020-08
  • McHugh, Mary L (2012). "Interrater Reliability: The Kappa Statistic". Biochemia Medica, v. 22, n. 3, pp. 276-282. https://hrcak.srce.hr/89395
  • Medlock, Stephanie; Eslami, Saeid; Askari, Marjan; Arts, Derk L; Sent, Danielle; De Rooij, Sophia E; Abu-Hanna, Ameen (2015). "Health Information–Seeking Behavior of Seniors Who Use the Internet: A Survey". Journal of Medical Internet Research, v. 17, n. 1, pp. e10. https://doi.org/10.2196/jmir.3749
  • Menachemi, Nir; Rahurkar, Saurabh; Harle, Christopher A; Vest, Joshua R (2018). "The Benefits of Health Information Exchange: An Updated Systematic Review". Journal of the American Medical Informatics Association, v. 25, n. 9, pp. 1259-1265. https://doi.org/10.1093/jamia/ocy035
  • Mitu, Bianca (2016). "Health in the Digital Era: Searching Health Information Online". In: The Power of the Media in Health Communication. Marinescu, Valentina; Mitu, Bianca (Eds.), pp. 145-156. Routledge. https://www.taylorfrancis.com/chapters/edit/10.4324/9781315554068-15
  • Montoro-Montarroso, Andrés; Cantón-Correa, Javier; Rosso, Paolo; Chulvi, Berta; Panizo-Lledot, Ángel; Huertas-Tato, Javier; Calvo-Figueras, Blanca; Rementeria, M José; Gómez-Romero, Juan (2023). "Fighting Disinformation with Artificial Intelligence: Fundamentals, Advances and Challenges". Profesional de la Información, v. 32, n. 3, pp. e320322. https://doi.org/10.3145/epi.2023.may.22
  • Moreno, Megan A (2017). "Seeking Health Information Online". JAMA Pediatrics, v. 171, n. 5, pp. 500. https://doi.org/10.1001/jamapediatrics.2016.3109
  • Morgan, M. G.; Fischhoff, B.; Bostrom, A.; Atman, C. J. (2001). Risk Communication. Cambridge University Press. https://doi.org/10.1017/CBO9780511814679
  • Nguyen, Amy; Mosadeghi, Sasan; Almario, Christopher V (2017). "Persistent Digital Divide in Access to and Use of the Internet as a Resource for Health Information: Results from a California Population-based Study". International Journal of Medical Informatics, v. 103, pp. 49-54. https://doi.org/10.1016/j.ijmedinf.2017.04.008
  • Nogueira, Rodrigo; Jiang, Zhiying; Lin, Jimmy (2020). "Document Ranking with a Pretrained Sequence-to-Sequence Model".
  • Model". arXiv preprint arXiv:2003.06713. https://doi.org/10.48550/arXiv.2003.06713
  • O’Connor, Siobhan; Hanlon, Peter; O’Donnell, Catherine A.; Garcia, Sonia; Glanville, Julie; Mair, Frances S. (2016). "Understanding Factors Affecting Patient and Public Engagement and Recruitment to Digital Health Interventions: A Systematic Review of Qualitative Studies". BMC Medical Informatics and Decision Making, v. 16, n. 1, pp. 120. https://doi.org/10.1186/s12911-016-0359-3
  • OpenAI (2023a). "Introducing ChatGPT." OpenAI Blog. https://openai.com/blog/chatgpt
  • OpenAI (2023b). "What is Chat GPT and How Does It Work?" Chat GPT de OpenAI. https://help.openai.com/en/articles/6783457-what-is-chatgpt
  • Osei Asibey, Benedict; Agyemang, Seth; Boakye Dankwah, Augustina (2017). "The Internet Use for Health Information Seeking Among Ghanaian University Students: A Cross‐Sectional Study". International Journal of Telemedicine and Applications, v. 2017, n. 1, pp. 1756473. https://doi.org/10.1155/2017/1756473
  • Perko, Tanja; Turcanu, Catrinel (2020). "Is Internet a Missed Opportunity? Evaluating Radon Websites From a Stakeholder Engagement Perspective". Journal of Environmental Radioactivity, v. 212, pp. 106123. https://doi.org/10.1016/j.jenvrad.2019.106123
  • Raffel, Colin; Shazeer, Noam; Roberts, Adam; Lee, Katherine; Narang, Sharan; Matena, Michael; Zhou, Yanqi; Li, Wei; Liu, Peter J (2020). "Exploring the Limits of Transfer Learning With a Unified Text-to-Text Transformer". Journal of Machine Learning Research, v. 21, n. 140, pp. 1-67. https://www.jmlr.org/papers/v21/20-074.html
  • Robertson, Stephen; Zaragoza, Hugo (2009). "The Probabilistic Relevance Framework: BM25 and Beyond". Foundations and Trends® in Information Retrieval, v. 3, n. 4, pp. 333-389. https://doi.org/10.1561/1500000019
  • Rotman, David (2023). "ChatGPT is About to Revolutionize the Economy. We Need to Decide What That Looks Like." MIT Technology Review. https://www.technologyreview.com/2023/03/25/1070275/chatgpt-revolutionize-economy-decide-what-looks-like
  • Ruano-Raviña, A; Quindós-Poncela, L; Barros-Dios, J M (2014). "Indoor Radon and Public Health in Spain. Time for Action". Gaceta Sanitaria, v. 28, n. 6, pp. 439-441. https://doi.org/10.1016/j.gaceta.2014.09.003
  • Samet, Jonathan M (1989). "Radon and Lung Cancer". JNCI: Journal of the National Cancer Institute, v. 81, n. 10, pp. 745-758. https://doi.org/10.1093/jnci/81.10.745
  • Sarfraz, Zouina; Sarfraz, Azza; Iftikar, Hamza Mohammad; Akhund, Ramsha (2021). "Is COVID-19 Pushing Us to the Fifth Industrial Revolution (Society 5.0)?". Pakistan Journal of Medical Sciences, v. 37, n. 2, pp. 591-594. https://doi.org/10.12669/pjms.37.2.3387
  • Schwab, K. (2020). "La Cuarta Revolución Industrial". Futuro Hoy, v. 1, n. 1, pp. 06-10. https://doi.org/10.52749/fh.v1i1.1
  • Schwab, Klaus (2016). La Cuarta Revolución Industrial. Debate. https://www.mintur.gob.es/Publicaciones/Publicacionesperiodicas/EconomiaIndustrial/RevistaEconomiaIndustrial/429/LIBROS%202.pdf
  • Stanfill, Mary H; Marc, David T (2019). "Health Information Management: Implications of Artificial Intelligence on Healthcare Data and Information Management". Yearbook of Medical Informatics, v. 28, n. 01, pp. 056-064. https://doi.org/10.1055/s-0039-1677913
  • Sung, Joseph J Y; Stewart, Cameron L; Freedman, Ben (2020). "Artificial Intelligence in Health Care: Preparing for the Fifth Industrial Revolution". Medical Journal of Australia, v. 213, n. 6, pp. 253-255. e1. https://doi.org/10.5694/mja2.50755
  • Taplin, Ruth (2023). Artificial Intelligence, Intellectual Property, Cyber Risk and Robotics: A New Digital Age. Routledge. https://doi.org/10.4324/9780367857561
  • Van Laer, Jeroen; Van Aelst, Peter (2010). "Internet and Social Movement Action Repertoires: Opportunities and Limitations". Information, Communication & Society, v. 13, n. 8, pp. 1146-1171. https://doi.org/10.1080/13691181003628307
  • Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N; Kaiser, Łukasz; Polosukhin, Illia (2017). "Attention is All You Need." In: Advances in Neural Information Processing Systems 30 (NIPS 2017). https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
  • Wartella, Ellen; Rideout, Vicky; Montague, Heather; Beaudoin-Ryan, Leanne; Lauricella, Alexis (2016). "Teens, Health and Technology: A National Survey". Media and Communication, v. 4, n. 3, pp. 13-23. https://doi.org/10.17645/mac.v4i3.515
  • WHO (2009). Handbook on Indoor Radon: A Public Health Perspective. World Health Organization. https://www.who.int/publications/i/item/9789241547673
  • WHO (2021). "El radón y sus efectos en la salud." https://www.who.int/es/news-room/fact-sheets/detail/radon-and-health
  • Wimble, Matt(2016). "Understanding Health and Health-Related Behavior of Users of Internet Health Information". Telemedicine and e-Health,v. 22, n. 10, pp. 809-815. https://doi.org/10.1089/tmj.2015.0267
  • Zschorlich, Beate; Gechter, Dorothea; Janßen, Inger M; Swinehart, Thomas; Wiegard, Beate; Koch, Klaus(2015). "Health information on the Internet: Who is searching for what, when and how?". Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen,v. 109, n. 2, pp. 144-152. https://doi.org/10.1016/j.zefq.2015.03.003
Fuente de los datos: Dialnet