AI to Transform How Physicians Handle Computerized Patient Information

AI will empower physicians and their staff to accurately, securely, and easily capture, analyze, and derive insights from vast amounts of distributed patient data.

Image generated by Google Gemini

Over the past few years I’ve had several informal conversations with my college roommate, Dr. David Kutsche, MD, which motivated me to collaborate with him to write this article. As healthcare has increasingly relied on technology for past three decades, physicians continue to face growing frustrations with complex software systems. These tools, designed to improve data management and patient care, often hinder rather than help. This article explores the challenges of managing computerized patient information, from unintuitive interfaces to missing data, and examines potential solutions using advanced information management and Artificial Intelligence (AI) technologies.

Introduction

For over sixty years computer systems have been an integral part of many businesses, such as banking, utilities and transportation. With the emergence of graphical user interfaces, personal computers and popular applications like spreadsheets in the 1980s, it became a lot easier for ordinary people to visually interact with data without having to be computer experts. Later with the advent of the Internet and Web browsers, computer applications became much easier and cheaper to access and use with more intuitive, standardized user interfaces. Physicians, nurses, pharmacists and other professionals in the healthcare industry also saw the emergence of specialized software applications for managing patient data. 

Interoperability between different computer systems has always been very important yet challenging to achieve. This is especially true in the healthcare industry where for the past two decades paper-based patient data have been replaced with electronic records systems. Those systems must accurately, securely and effectively exchange confidential patient data with other computer systems used by insurance companies, pharmacies, labs, hospitals, etc. However, many organizations have not achieved that goal, leaving physicians and other healthcare providers with incomplete and inaccurate patient data to make decisions. Furthermore, poorly designed application user interfaces with cumbersome means of inputting, retrieving and interpreting patient data frustrate physicians, reduce their effectiveness and impact their relationships with their patients and colleagues. In addition, this can lead to considerable negative impact on physicians’ work-life balance. Next we review various types of patient data that physicians need and their challenges dealing with that data using computer systems.

Primary types of patient data 

There is a wide spectrum of data about patients and their health that expands over time. Below are the primary types of patient data that physicians deal with as they interact with different computer systems throughout the patient’s diagnosis and treatment processes.

• Demographic data, e.g., age, ethnicity, gender, etc.

• Family history of illnesses, e.g., cancer, heart disease, depression, etc.

• History of social habits, e.g., smoking, alcohol, drug use, etc.

• Past medical issues that are resolved

• Current/active medical issues 

• Current vital signs, e.g., weight, height, temperature, blood pressure, heart rate, respiration, etc.

• Lab results, e.g., blood, urine, pathology, etc.

• Imaging results, e.g. X-ray, ultrasound, MRI, CT scan, etc.

• Current and historical medication lists

• Allergies

• Consultations from other physicians

• Hospital records from emergency room and inpatient treatments

• Ancillary treatments, e.g., physical therapy, home health and nurse visits, etc.

Most of these patient data are in unstructured format, e.g., in reports, notes, messages, etc. Organizing them in traditional data management systems, e.g., spreadsheets or relational databases, is time consuming, error prone and expensive. The recent AI advances in Large Language Models (LLM), Generative AI (GenAI) and semantic search provide an opportunity to efficiently retrieve accurate information from structured and unstructured data sources without having to consolidate all of them in a single database [1].

Computer systems used for managing patient data

Physicians often need to work with multiple computer systems to obtain the information needed for diagnosis and treatment of their patients. There is often a central software application that provides a single graphical user interface for inputting, retrieving and analyzing patient data which may reside in different systems. A major Electronic Health Record (EHR) system, which caters to a number of specialties, is in use across a broad range of practices, from community hospitals and independent practices to multi-specialty hospital groups. This system has a strong focus on patient engagement and facilitating remote care. However, its graphical user interfaces are not always efficient or complete for physicians to easily manage accessible patient information. Recently, this system has begun to utilize natural language processing (NLP) along with LLM and GenAI technologies to offer a conversational interface, similar to ChatGPT, that provides helpful information quickly while maintaining the context of an interactive conversation.

Computer systems used for managing patient data

Physicians often need to work with multiple computer systems for diagnosis and treatment of their patients. There is often a central software application that provides a single graphical user interface for inputting, retrieving and analyzing patient data which may reside in different systems. A major cloud-based Electronic Health Record (EHR) solution, which caters to a number of specialties, is in use across a broad range of practices, from community hospitals and independent practices to multi-specialty hospital groups and hospice care providers. This system has a strong focus on patient engagement and facilitating remote care. However, its graphical user interfaces are not always efficient or complete for physicians to easily manage accessible patient information. Recently, this system has begun to utilize natural language processing (NLP) along with LLM and GenAI technologies to provide human-like conversational interfaces which can quickly provide helpful information while maintaining the context of an interactive conversation. 

Challenges and considerations

Disparate patient data in multiple computer systems cause serious challenges for physicians and below are some specific examples:

• Unintuitive user interface: hard for physicians to figure out what to do, e.g., too many clicks to accomplish a task, confusing menus, poorly placed buttons and icons causing selection of the wrong item, confusing messages, etc.

• Unreliable applications that freeze or bring up different screens unpredictability.

• Software upgrades that cause side-effects and unexpected problems.

• Problems with searching with keywords that sometimes return wrong results even if the exact word is used in the search term. Ironically sometimes correct results are returned when partial keywords are used, but not when the entire word is entered.

By analyzing user behavior and system performance, AI can optimize user interfaces, making them more intuitive and efficient. Predictive analytics can improve system reliability by identifying potential issues and automatically fix them or suggest solutions. Natural Language Processing (NLP) can enhance search capabilities by maintaining an interactive dialog to quickly find the information a physician needs. Additionally, AI can streamline software upgrades by assessing potential impacts and automating testing processes.

Accuracy and security of patient data

Security and high quality of patient data are paramount and must be rigorously ensured. Because parts of patient data are manually entered into computer systems by non-medical personnel, they are susceptible to errors. Also, format, organization and level of detail of information provided by different healthcare facilities vary, thus making it more challenging for physicians to make sense of the patient data. Below are some areas that are especially problematic:

• Automatically generated transcriptions using voice recognition can have inaccurate data

• Open access to the patient data allows medical personnel from other sources to freely change the records, and consequently one person can introduce errors in records entered by another person.

• There is often too much information, possibly duplicate or contradictory, entered by multiple individuals which makes navigating and comprehending the data difficult.

• Important information, like medications, often gets lost in too much free-form notes entered by different physicians, nurses, etc.

To improve data accuracy, AI can validate data entry, standardize formats across different healthcare facilities, and enhance transcription quality. By employing NLP techniques, AI can extract crucial information from unstructured text, creating concise summaries and reducing information overload. To bolster data security, AI-driven access controls and audit trails can be implemented to monitor data modifications and prevent unauthorized access. Moreover, by detecting and resolving duplicate data, AI can help ensure data consistency.

Different types of devices for handling patient data

Physicians and nurses often must enter and retrieve patient data as they interact with the patient. Below are a list of common devices and techniques used for handling patient data in a medical facility:

• Desktop personal computer which has a larger screen, keyboard and mouse

• Tablets that offer easy, intuitive data entry without a keyboard but on smaller screens

• Patient data are generally typed in manually but some physicians use a human scribe or standard voice recognition application, both of which could introduce inaccuracies and add overhead cost to the office.

• Physicians must enter lab orders after examining a patient. This is a laborious and error-prone task.  One example is an exhaustive drop-down menu for an order so that, depending on how the order is named, the provider may select a test and find out later that it was not the test s/he expected. Another example is when a provider cannot find a test to order because it was entered in the database under a different name. S/he may want to order a leg ultrasound but cannot find it because the computer has it listed under Duplex and is not able to recognize the search for the ultrasound.  

By creating adaptive user interfaces, AI can optimize interactions across different devices. Speech-to-text technology powered by AI can reduce reliance on human scribes and improve data accuracy. Furthermore, AI can enhance order entry through intelligent suggestions and error prevention. To ensure data consistency, AI-driven data synchronization across devices is crucial. Ultimately, these AI applications streamline workflows, reduce errors, and improve overall efficiency in medical facilities.

Entering and retrieving patient data 

Prior, during and after visiting with a patient, a physician and his/her staff need to enter, retrieve and review various types of information about that patient. Below are some of the common methods and key challenges for entering and retrieving patient data.

• If physicians and their staff in different facilities are on the same computer system, much of the patient data can be automatically shared and updated electronically.  

• A fair bit of patient data, such as lab reports and records of past surgeries or problems, may be available in only hard copies that are faxed, dropped off or mailed. Such documents are then scanned into electronic files. These files first must be labeled in order to find them later. They are usually scanned by non-medical personnel, but they may not label the items accurately enough for the provider to easily find them later. That information is then essentially lost in the computer.   

• When admitting new patients, generally a medical assistant works with the patient to obtain the medical history, medication list, etc. and subsequently enters the data into the computer system. Also most medical computer systems have predefined forms to obtain specific information required by insurance companies, e.g., smoking or drinking habits. Entering all of this patient data is a fairly laborious and error-prone process and mistakes may not be discovered until much later, if ever. 

• When a physician performs a physical exam, s/he must enter the information directly into the computer system during the examination. During a 30-40 minute physical exam visit, 30-50% of the time may be spent on information gathering, updating the record and entering data in the computer application.  It takes longer for some providers than others depending on their typing skills or method of data entry.  It can take 10-15 minutes to enter orders, refill medications and provide written patient instructions, depending on the complexity of the visit. Many physicians are not skilled at typing or entering data into computer interfaces, which results in more time looking at the computer than is spent interacting with the patient.

• A physician must be able to carefully listen and focus on what the patient is saying. Therefore using voice recognition and transcribing software may not work well. 

• Many physicians prefer writing with a pen and have the computer system automatically convert the handwriting into structured text. Therefore, a stylus with a tablet computer may be much easier than a keyboard for a physician to use. However, this would require an intuitive and friendly user interface designed specifically for medical professionals.   

• It’s often a medical assistant who checks the patient’s vital signs and updates the records. Also later that medical assistant or a nurse may be summoned to draw blood, perform any necessary injections and update the patient’s computer records.

• Computer reports from hospitals, emergency rooms, etc. are often poorly formatted with mixing in demographic information, past history and other non-essential data with little relevance to the medical problem being addressed.  The main narrative of the medical encounter is frequently way near the end of a long list of pages to scroll through.  For a clinician, it would be much better to see the narrative of the visit right at the beginning of the report so it can be seen quickly in order to easily determine what needs to be done.

AI can significantly streamline patient data management. For instance, AI can automatically extract and structure data from scanned documents, improving accessibility and accuracy. Intelligent data entry tools can automate routine data capture, reducing errors and freeing up staff time. To enhance physician efficiency, AI-powered speech recognition can accurately convert spoken notes into text, minimizing interruptions during patient examinations. Furthermore, AI can suggest relevant medical terms and complete patient information based on context, accelerating data entry. By analyzing patient data, AI can identify patterns and inconsistencies, flagging potential errors or missing information. Additionally, AI-driven summarization of complex medical reports can provide physicians with a clear and concise overview of patient history.

Transferring patient data between computer systems

Because there is no central repository to hold all different types of patient data, medical professionals generally need to transfer data from one system to another. For medical facilities that use the same computer system for managing patient data, it is possible that patient data can be automatically transferred over the network.  But for patient data coming from external systems, such as lab or X-ray machines, data must either be manually entered into the central system or kept in separate computer or paper files. Furthermore, if lab results are from multiple labs, they are often reported in different formats, and consequently consolidating them into a single system for future reference and comparison poses additional challenges. As a result, patient data are often kept in multiple formats and in different systems which makes it even more difficult for physicians to access and analyze the most up-to-date data efficiently. 

Utilizing AI can help address these challenges by reducing errors and manual efforts by automatically extracting, standardizing, and transferring data between systems as well as aggregating data from various sources and improving data quality to generate helpful insights.

Areas of opportunity

Rapid advancements in information management and advanced AI technologies provide greater opportunities to create innovative solutions for medical teams to more effectively handle computerized patient information. But because computers don’t have the experience and insights of trained medical professionals, they are unable to always produce 100% accurate and reliable information. Below are some specific areas for developing innovative solutions for managing patient information across different computer systems.

Standardized medical information models

Safe and secure sharing of electronic health information (EHI) among different computer systems and with patients and healthcare providers requires standard representations of syntax and semantics of medical information. Over the past 15 years several interoperability initiatives have been sponsored by governments and the healthcare industry worldwide. The Health Level Seven (HL7®) Fast Healthcare Interoperability Resources (FHIR) [2] has a set of standards for bidirectional exchange (read/write) of granular medical data and documents. FHIR is required and implemented as part of the 21st Century Cures Act of the US Office of the National Coordinator for Health Information Technology (ONC) [3]. FHIR and similar interoperability standards must continue to evolve to also represent medical knowledge and expertise required by intelligent computer systems to automatically learn and reason about healthcare information to better guide physicians and patients. 

Real-time data collection, analysis and automatic recommendations

As more people use mobile devices to collect and analyze their health information, processing this information reliably, accurately and quickly in a distributed cloud computing environment is extremely important. For example, a person’s heart rate, blood oxygen level and body temperature can be continuously collected through a smart wristband which transmits this information to a smartphone connected to the Internet. The person's medical history as well as sophisticated diagnostic algorithms may be available in a central computing center that receives the person’s vital health records in real time. In case of a life threatening situation like a heart attack, the central system must accurately diagnose and transmit a recommendation back to the person’s mobile device in a matter of a few seconds. The system may also notify the person’s physician on file as well as a local medical emergency response team if the person is unable to do so. As 5G and other advanced networking technologies continuously speed up and improve quality of data transmissions, retrieval, analysis and interpretation of medical information will also become faster and more accurate especially by utilizing advanced AI technologies [4].

Advanced, self-learning user interfaces

As it was highlighted earlier, physicians and their staff spend a large percentage of their time entering, retrieving and reviewing medical information through different application user interfaces. With the rapidly growing usage of tablets and smartphones over the past 20 years, graphical user interface technologies have advanced significantly while improving user experience has become the number one priority for most application designers. A new generation of interactive applications now automatically learns from user interactions to predict intent to offer more personalized user experiences. Augmented analytics utilize natural language processing, conversational chatbot and advanced computer graphics technologies to provide more intuitive ways of gaining insight quickly and easily. A new generation of “smart” medical applications using these technologies will help physicians and their staff have more productive and pleasant experiences when dealing with computerized patient data. For example, finding and viewing a patient’s lab results will become faster and easier by automatically choosing the results that are relevant to the context of a medical diagnosis and potential treatment.  Also, mistakes and errors will be eliminated or significantly reduced through automated validation of data using various AI techniques.   

Speech recognition and smart transcribers

As discussed earlier, clinicians could spend several hours a day entering detailed data into EHR systems. This is by far the number source of frustration and burnout for physicians and their staff resulting in rushed and unpleasant visits with their patients. Although some physicians use dictation software, the extensive training required as well as an unnatural way of speaking to the computer (like calling out punctuation) often result in unpleasant experiences. Meanwhile, speech recognition and natural language processing technologies have advanced greatly and are now offered in many intelligent virtual assistants, such as Amazon Alexa, Google Assistant and Apple Siri. Over the past few years, several software products have used AI and LLM technologies for automatically transcribing medical conversations to text. Some of these products are even capable of comprehending terminologies for specialty care areas such as ​cardiology, neurology, obstetrics-gynecology, pediatrics, oncology, radiology and urology. As these products become more accurate, easier to use, less expensive and widely used in medical facilities, physicians and their staff will be able to spend more quality time with their patients instead of manually typing data into computers.

Collaborative AI agents

An AI agent acts like a human so it can learn, remember, plan, autonomously perform specific tasks, make decisions and interact with its environment and other AI agents. Unlike a pre-programmed robot, an AI agent has memory and utilizes LLMs, various software tools and other agents to decide how to perform its tasks and learn to improve its performance [5]. Frameworks and tools are rapidly emerging to develop and manage AI agents and their collaboration workflows. Collaborative AI agents can help physicians and their staff with a variety of tasks, including:

1. Organizing and understanding patient data from different systems

2. Collecting, cleansing, categorizing and formatting data to be transferred between different systems

3. Researching, collating, summarizing and writing reports

4. Attending staff meetings and patient visits to take notes and follow-up with action items

5. Tracking scheduled medical procedures at various steps and keeping track of progress

6. Responding to physician and patient requests and involving the appropriate teams

Access to AI agents as well as their activities must be carefully monitored, logged and controlled to ensure compliance with regulations and policies at all levels.

Risks and remaining challenges

Information about patients and their medical records are highly sensitive and accessing them is restricted by a myriad of regulations worldwide. Storing, sharing, and accessing computerized patient data present significant challenges and risks, particularly in relation to data security and privacy. National and international regulations, such as HIPAA [6] in the United States and GDPR [7] in the European Union, impose stringent requirements to protect patient information. The key risks include misuse, loss or unauthorized access to the data. Meanwhile adhering to complex regulations and standards can be costly and burdensome. To mitigate these risks, robust security measures, including encryption, access controls, and regular audits, must be implemented. Additionally, data sharing practices must comply with applicable regulations and patients must have a clear understanding of how their data is used.

Using AI for analyzing and interpreting patient information poses additional risks and challenges. A recent study by researchers at the University of Massachusetts Amherst [8] discovered the limitations of LLMs in the medical domain, specifically, both GPT-4 and Llama-3 frequently generated inaccurate or misleading information when summarizing medical notes. These types of "hallucinations" occurred mostly in areas of symptoms, diagnosis, and treatment, which reveal very limited availability of medical knowledge in these LLMs. These errors can have severe consequences as misinterpreted medical information can lead to incorrect diagnoses, inappropriate treatments, and potential harm to patients. These findings align with previous research demonstrating the limitations of LLMs in medical question answering, highlighting the complexity of translating human medical knowledge into a format digestible by AI systems. The study emphasizes the urgent need for improved medical domain expertise in LLMs to ensure their safe and effective use in healthcare settings.

To fully harness the potential of AI in healthcare, all industry and government organizations must work together to create clear guidelines for developing and using these tools safely, securely, ethically, and responsibly. This includes ensuring that AI is fair, transparent, and trustworthy for all patients [1]. Groups like the Coalition for Health AI [9] have already started this process by creating initial guidelines, which include a Blueprint for Trustworthy AI [10].

Summary

Improving quality of healthcare while reducing costs requires extensive digital transformation and automation through innovative technologies. Governments have mandated policies for usage of standard EHR systems to facilitate safer, faster and more accurate means of sharing medical information across different computer systems. However, software applications used by physicians and their staff for entering, retrieving and analyzing patients’ medical information are often hard to use, unreliable and error prone. In this paper we outlined challenges with these software applications and platforms that support them. We also explored opportunities for using advanced information management and AI technologies to improve these applications and their interoperability while adhering to various regulations for safeguarding confidential patient data.

References

1. How generative AI and large language models can close the gap between data and outcomes in healthcare https://www.weforum.org/agenda/2024/01/generative-ai-large-language-models-data-outcomes-healthcare 

2. The Health Level Seven (HL7®) Fast Healthcare Interoperability Resources (FHIR) standard https://www.hl7.org/fhir/index.html

3. The Office of the National Coordinator for Health Information Technology (ONC) https://www.healthit.gov

4. Tackling healthcare’s biggest burdens with generative AI https://www.mckinsey.com/industries/healthcare/our-insights/tackling-healthcares-biggest-burdens-with-generative-ai 

5. Collaborative AI Agents Will Change How We Work https://kiumarse.substack.com/p/collaborative-ai-agents-will-change 

6. HIPAA Compliance and Medical Records https://www.hipaajournal.com/hipaa-compliance-and-medical-records 

7. Protect Patient Privacy: The Definitive Guide to GDPR Compliance for Healthcare Companies https://www.kiteworks.com/gdpr-compliance/patient-privacy-protection-best-practices

8. How Often Do LLMs Hallucinate When Producing Medical Summaries? https://medcitynews.com/2024/08/ai-healthcare-llm 

9. Coalition for Health AI (CHAI) https://chai.org

10. Blueprint for Trustworthy AI Implementation Guidance and Assurance for Healthcare https://coalitionforhealthai.org/papers/blueprint-for-trustworthy-ai_V1.0.pdf