AI Scribes: What Does Physician Review Verify?
DOI:
https://doi.org/10.11606/issn.2317-2770.v31i2e-262550Keywords:
Artificial Intelligence, Clinical Documentation, Patient SafetyAbstract
Artificial intelligence (AI) scribes, which record conversations between physicians and patients and generate clinical notes, are increasingly being adopted to reduce the burden of clinical documentation. However, AI-generated notes may contain omissions, information not discussed during the consultation, and unintended inclusions. This letter discusses the limitations of physician review and argues that a clinical note may be coherent without faithfully reflecting the original consultation. This concern is particularly relevant when omissions are difficult to detect, fluent text encourages automation bias, or the original audio is unavailable for verification. We also discuss ethical concerns, linguistic diversity, equity, and data protection, including implications for Brazilian Portuguese. Finally, we propose evaluation strategies that assess fidelity to the original consultation alongside note quality and explore verification approaches that are feasible in clinical practice.
Downloads
References
1. Olson KD, Meeker D, Troup M, Barker TD, Nguyen VH, Manders JB, et al. Use of ambient AI scribes to reduce administrative burden and professional burnout. JAMA Netw Open. 2025;8(10):e2534976. DOI: https://doi.org/10.1001/jamanetworkopen.2025.34976
2. Lukac PJ, Turner W, Vangala S, Chin AT, Khalili J, Shih YCT, et al. Ambient AI scribes in clinical practice: a randomized trial. NEJM AI. 2025;2(12). DOI: https://doi.org/10.1056/AIoa2501000
3. Taylor SL, Jost M, MacDonald S, Ren Y, Hilton S, Davenport S, et al. Quality of clinical notes created by ambient listening generative AI: pragmatic prospective pilot study. JMIR Med Inform. 2026;14:e86474. DOI: https://doi.org/10.2196/86474
4. Biro JM, Handley JL, Malcolm McCurry J, Visconti A, Weinfeld J, Gregory Trafton J, et al. Opportunities and risks of artificial intelligence in patient portal messaging in primary care. NPJ Digit Med. 2025;8(1):222. DOI: https://doi.org/10.1038/s41746-025-01586-2
5. Zhou L, Blackley SV, Kowalski L, Doan R, Acker WW, Landman AB, et al. Analysis of errors in dictated clinical documents assisted by speech recognition software and professional transcriptionists. JAMA Netw Open. 2018;1(3):e180530. DOI: https://doi.org/10.1001/jamanetworkopen.2018.0530
6. Abdelwanis M, Alarafati HK, Tammam MMS, Simsekler MCE. Exploring the risks of automation bias in healthcare artificial intelligence applications: a Bowtie analysis. J Saf Sci Resil. 2024;5(4):460–9. DOI: https://doi.org/10.1016/j.jnlssr.2024.06.001
7. Ursin F, Salloch S. The ethics of AI scribes as epistemic agents. JMIR Med Inform. 2026;14:e88235. DOI: https://doi.org/10.2196/88235
8. Koenecke A, Choi ASG, Mei KX, Schellmann H, Sloane M. Careless whisper: speech-to-text hallucination harms. In: The 2024 ACM Conference on Fairness, Accountability, and Transparency [Internet]. Rio de Janeiro: ACM; 2024 [cited 2026 Oct 20]. p. 1672–81. Available from: https://dl.acm.org/doi/10.1145/3630106.3658996
9. Lawrence K, Kuram VS, Levine DL, Sharif S, Polet C, Malhotra K, et al. Informed consent for ambient documentation using generative AI in ambulatory care. JAMA Netw Open. 2025;8(7):e2522400. DOI: https://doi.org/10.1001/jamanetworkopen.2025.22400
10. Ng JJW, Wang E, Zhou X, Zhou KX, Goh CXL, Sim GZN, et al. Evaluating the performance of artificial intelligence-based speech recognition for clinical documentation: a systematic review. BMC Med Inform Decis Mak. 2025;25(1):236. DOI: https://doi.org/10.1186/s12911-025-03061-0
11. O'Kane R, Stonehouse-Smith D, Shirazi M, Hutchinson R, Gregg M, Patel R, et al. A controlled single-speaker evaluation of ambient voice technologies' speech-to-text function using narrated surgical case-reports. J Med Syst. 2026;50(1):145. DOI: https://doi.org/10.1007/s10916-026-02471-5
12. Stetson PD, Bakken S, Wrenn JO, Siegler EL. Assessing electronic note quality using the Physician Documentation Quality Instrument (PDQI-9). Appl Clin Inform. 2012;3(2):164–74. DOI: https://doi.org/10.4338/ACI-2011-11-RA-0070
13. Weiner SJ, Wang S, Kelly B, Sharma G, Schwartz A. How accurate is the medical record? A comparison of the physician's note with a concealed audio recording in unannounced standardized patient encounters. J Am Med Inform Assoc. 2020;27(5):770–5. DOI: https://doi.org/10.1093/jamia/ocaa027
14. Croxford E, Gao Y, First E, Pellegrino N, Schnier M, Caskey J, et al. Evaluating clinical AI summaries with large language models as judges. NPJ Digit Med. 2025;8(1):640. DOI: https://doi.org/10.1038/s41746-025-02005-2
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Andre Massahiro Shimaoka, Luciano Rodrigo Lopes, Paulo Bandiera-Paiva
This work is licensed under a Creative Commons Attribution 4.0 International License.
Autores que publicam nesta revista concordam com os seguintes termos:
- Autores mantém os direitos autorais e concedem à revista o direito de primeira publicação, que permite o compartilhamento do trabalho com reconhecimento da autoria e publicação inicial nesta revista.
- Autores têm autorização para assumir contratos adicionais separadamente, para distribuição não-exclusiva da versão do trabalho publicada nesta revista (ex.: publicar em repositório institucional ou como capítulo de livro), com reconhecimento de autoria e publicação inicial nesta revista.
