Why the State Should Regulate Artificial Intelligence in Health: Commercial Determinants of Capitalist Algorithmic Degradation And Artificial Intelligence Clinical Applications
DOI:
https://doi.org/10.11606/issn.2316-9044.rdisan.2025.231445Keywords:
Algorithmic Degradation, Health Disparities, Ethics, Digital HealthAbstract
This article aimed to describe the characteristics by which the development of ar tificial intelligence in clinical applications cannot be isolated from the general process followed by other income extraction mechanisms already practiced in digital technology platforms in other domains. The method utilized was a synthetic review of recent academic and journalistic literature, analyzing the mechanisms of the relationship between profit concentration and algorithmic degradation in various fields, including health. The results showed that these capitalist mechanisms inevitably lead to adjusting or tuning the algorithms and selecting the results to maximize the returns for capital. The logic behind technology corporations leads inexorably to a progressive process in which the best interests of patients must be put aside whenever there are good commercial reasons. This process is independent of the good intentions and altruism of health startups. In conclusion, without the active participation of the State, the ecosystem for generating digital health solutions will inevitably degrade in order to favor the extraction of income while harming the patients’ interests.
Downloads
References
AGGARWAL, Pushkar; PAPAY, Francis. Artificial intelligence image recognition of melanoma and basal cell carcinoma in racially diverse populations. Journal of Dermatological Treatment, v. 33, n. 4, p. 2257–2262, 2021. DOI: DOI: https://doi.org/10.1080/09546634.2021.1944970. Disponível em: https://www.tandfonline.com/doi/full/10.1080/09546634.2021.1944970.
AI SKIN cancer diagnoses risk being less accurate for dark skin. The Guardian, 2021. Disponível em: https://www.theguardian.com/society/2021/nov/09/ai-skin-cancer-diagnoses-risk-being-less-accurate-for-dark-skin-study.
ALI, Aran. The Business of Airbnb, by the Numbers. Visual Capitalist, 06 jan. 2021. Disponível em: https://www.visualcapitalist.com/the-business-of-airbnb-by-the-numbers/.
BAACK, Stefan. The human decisions that shape generative AI: Who is accountable for what?. Mozilla Foundation, 02 ago. 2023. Disponível em: https://foundation.mozilla.org/en/blog/the-human-decisions-that-shape-generative-ai-who-is-accountable-for-what/. Acesso em: 28 abr. 2025.
BELLEMARE, Michel Luc. Techno-capitalist-feudalism. Blacksatin Publishing Limited, 2020.
BENDER, Emily, et al. On the dangers of stochastic parrots: can language models be too big? In: CONFERENCE ON FAIRNESS, ACCOUNTABILITY, AND TRANSPARENCY (FAccT ’21), March 3–10, 2021, Virtual Event, Canada. ACM, New York, NY, USA, 2021 Disponível em: https://dl.acm.org/doi/pdf/10.1145/3442188.3445922.
BOCK, Anna; HSWEN, Yulin. How artificial intelligence will enhance imaging access and analysis. JAMA, v. 331, n. 23, p. 1979-1981, 2024.
BRIN, Sergey; PAGE, Lawrence. The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems, v. 30, p. 107-117, 1998. Disponível em: https://snap.stanford.edu/class/cs224w-readings/Brin98Anatomy.pdf.
BULIK, Beth. Is there a place for pharma in the emerging EHR Market? FiercePharma News, 11 maio 2015.
CAVLAN, O. et al. Real-world evidence: From activity to impact in healthcare decision making. Pharmaceutical and Medical Products. McKinsey & Company, 2017. Disponível em: https://www.paparrizos.org/press/PaparrizosMcKinsey18.pdf.
CHAMPAGNE, David; HUNG, Amy; LECLERC, Oliver. The road to digital success in pharma. Pharmaceutical and Medical Products. McKinsey & Company, ago. 2015.
CFM aponta inconsistências em aplicativo do Ministério da Saúde. Conselho Federal de Medicina, 21 jan. 2021. Disponível em: https://portal.cfm.org.br/noticias/cfm-aponta-inconsistencias-em-aplicativo-do-ministerio-da-saude-2.
CHEN, Irene et al. Ethical machine learning in healthcare. Annu. Rev. Biomed Data Sci., v. 4, p. 123-144, 2021. DOI: https://doi.org/10.1146/annurev-biodatasci-092820-114757. Disponível em: https://www.annualreviews.org/docserver/fulltext/biodatasci/4/1/annurev-biodatasci-092820-114757.pdf?expires=1743713644&id=id&accname=guest&checksum=16B02361C565DAEB497715604658D70E.
DOCTOROW, Cory. The “enshittification” of TikTok. 2023. Disponível em https://www.wired.com/story/tiktok-platforms-cory-doctorow/.
EMILIANA, Cecília. App de Pazuello sugere kit COVID a pacientes com sintomas genéricos. Estado de Minas Nacional, 20 jan. 2021. Disponível em: https://www.em.com.br/app/noticia/nacional/2021/01/20/interna_nacional,1230769/app-de-pazuello-sugere-kit-covid-a-pacientes-com-sintomas-genericos.shtml. Acesso em: 12 jan. 2021.
ENSHITTIFICATION. [s.d.]. Disponível em: https://en.wiktionary.org/wiki/enshittification.
FARZAN, Antonia Noory. A tech company gave doctors free software — rigged to encourage them to prescribe opioids, prosecutors say. The Washington Post, 28 jan. 2020. Disponível em: https://www.washingtonpost.com/nation/2020/01/28/opioid-kickback-software/.
FAUSTINO, Deivisson; LIPPOLD, Walter. Colonialismo digital: por uma crítica hacker-fanoniana. Editora Boitempo, 2023.
HENRY, Terry. Keep pharmaceutical promotion out of doctors’ electronic tools. JAMA, 16 jun. 2021. Disponível em: https://www.ama-assn.org/practice-management/digital/keep-pharmaceutical-promotion-out-doctors-electronic-tools.
KOÇAK, Burak et al. Bias in artificial intelligence for medical imaging: fundamentals, detection, avoidance, mitigation, challenges, ethics, and prospects. Diagn Interv Radiol., v. 31, n. 2, p. 75-88, 2025. DOI: https://doi.org/10.4274/dir.2024.242854. Disponível em: https://www.dirjournal.org/articles/bias-in-artificial-intelligence-for-medical-imaging-fundamentals-detection-avoidance-mitigation-challenges-ethics-and-prospects/doi/dir.2024.242854.
KUWABARA, Masashi et al. Effectiveness of tuning an artificial intelligence algorithm for cerebral aneurysm diagnosis: a study of 10,000 consecutive cases. Sci Rep, v. 13, n. 16202, 2023. Disponível em: https://doi.org/10.1038/s41598-023-43418-x.
MACY Beth. Dopesick: dealers, doctors and the drug company that addicted America. Nova York: Little, Brown & Company, 2018.
MAZZUCATO, Mariana; GERNONE, Fausto. Governments must shape AI’s future. Project Syndicate, 2024 Disponível em: https://www.project-syndicate.org/onpoint/governments-must-shape-ai-future-by-mariana-mazzucato-and-fausto-gernone-2024-04.
MELLO, Michelle; ROSE, Sherri. Denial—artificial intelligence tools and health insurance coverage decisions. JAMA Health Forum, v. 5, n. 3, e240622, 2024. DOI: https://doi.org/10.1001/jamahealthforum.2024.0622.
MILLER, Katherine. Should AI models be explainable? that depends. Stanford HAI, 16 mar. 2021. Disponível em: https://hai.stanford.edu/news/should-ai-models-be-explainable-depends.
MORALES-FORERO, Andres et al. An insight into racial bias in dermoscopy repositories: A HAM10000 data set analysis. J Eur Acad Dermatol Venereol, 14 jun. 2024. DOI: https://doi.org/10.1002/jvc2.477.
O’LEARY, Lizzie. How IBM’s Watson went from the Future of Health Care to sold off for parts. Slate, 31 jan. 2022. Disponível em: https://slate.com/technology/2022/01/ibm-watson-health-failure-artificial-intelligence.html.
REILLY, Tim; STRAUSS, Ilan; MAZZUCATO, Mariana. Algorithmic attention rents: A theory of digital platform market power. Data & Policy, v. 6, 2024. DOI: https://doi.org/10.1017/dap.2024.1.
RÍGOLI, Félix. Inteligência Artificial nos Sistemas de Saúde: Perigos e Promessas para alcançar a saúde para todos In: AITH, Fernando; DALLARI, Analuza Bolivar (Coords.). LGPD na saúde digital. 1. ed. Sao Paulo: Thomson Reuters Brasil, 2022. v. 1, p. 61-80.
ROSS, Casey. Epic’s overhaul of a flawed algorithm shows why AI oversight is a life-or-death issue. Stat News, 24 out. 2022. Disponível em: https://www.statnews.com/2022/10/24/epic-overhaul-of-a-flawed-algorithm/.
SAÚDE lança aplicativo para auxiliar médicos no diagnóstico da Covid-19. ISMEP, 14 jan. 2021. Disponível em: https://www.gov.br/saude/pt-br/assuntos/noticias/saude-lanca-aplicativo-para-auxiliar-medicos-no-diagnostico-da-covid-19.
SAXENA, L. The role of AI-powered decision-making technology in medicare coverage determinations. Special Report,, jan. 2022. Center for Medicare Advocacy. Disponível em: https://medicareadvocacy.org/wp-content/uploads/2022/01/AI-Tools-In-Medicare.pdf
SCHEFFER, Mario; SOUZA, Paulo. A entrada do capital estrangeiro no sistema de saúde no Brasil. Cad. Saúde Pública, v. 38, Sup. 2, e00239421, 2022. DOI: https://doi.org/10.1590/0102-311X00239421.
TAITSMAN, Julie; VANLANDINGHAM, Andrew; GRIMM, Christi. Commercial Influences on Electronic health records and adverse effects on clinical decision-making. JAMA Internal Medicine, v. 180, n. 7, jul. 2020.
TALIA, Domenico. Algorithms that can deny care, and a call for AI explainability. Computer, v. 57, p. 109-112, jul. 2024. Disponível em: https://www.computer.org/csdl/magazine/co/2024/07/10574502/1Y7B8DpFlw4. Acesso em: 28 abr. 2025.
VAROUFAKIS, Yanis. Welcome to the age of Technofeudalism. Wired, 2024. Disponível em: https://www.wired.com/story/yanis-varoufakis-technofeudalism-interview/.
WEN, David et al. Characteristics of publicly available skin cancer image datasets: a systematic review. Lancet Digit Health, v. 4, p. e64-74, 2022.
WONG, Andrew, et al. External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Intern Med., v. 181, n. 8, p. 1065–1070, 2021. DOI: https://doi.org/10.1001/jamainternmed.2021.2626.
YU, Kun-Hsing et al. Medical Artificial Intelligence and human values. N Engl J Med, v. 390, n. 20, p. 1895-1904, 2024. Disponível em: https://www.nejm.org/doi/full/10.1056/NEJMra2214183.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Journal of Health Law

This work is licensed under a Creative Commons Attribution 4.0 International License.
The Revista de Direito Sanitário/ Journal of Health Law adopts the conditions of the Creative Commons Attribution 4.0 Internacional. This license allows to share - "copy and redistribute the material in any medium or format for any purpose, even commercially" and adapt - "remix, transform, and build upon the material for any purpose, even commercially." Details at: https://creativecommons.org/licenses/by/4.0/deed.en
