In the wake of a new University of Pennsylvania School of Nursing paper exploring the potential promise and pitfalls of integrating artificial intelligence deeper into hospital nursing environments, three of its authors agreed to answer LDI’s questions about the mounting discussions and tensions surrounding the issue.

They are LDI Senior Fellows and Penn Nursing leaders George Demiris, PhD, Professor of Biobehavioral Health Sciences at Penn Nursing and Informatics at the Perelman School of Medicine; Antonia Villarruel, PhD, RN, Dean and Professor of Family and Community Health at Penn Nursing; and Connie Ulrich, PhD, RN, Professor of Biobehavioral Health Sciences at Penn Nursing and Medical Ethics and Health Policy at the Perelman School of Medicine.

LDI: For readers who don’t follow this field closely, what is the single most important way artificial intelligence is likely to change nursing in the next few years?

We expect that the biggest change will likely be the shift of nurses’ time away from administrative and information-management tasks and back toward direct patient care. AI has the potential to help nurses synthesize large amounts of clinical information more quickly, automate repetitive documentation or logistical work, and identify early warning signs of patient deterioration. If implemented thoughtfully, this could allow nurses to spend more time communicating with patients and families, coordinating care, and exercising clinical judgment rather than navigating fragmented systems and burdensome paperwork.

LDI: Your paper suggests AI could be transformative for nursing. What does that actually look like at the bedside or in a hospital setting?

At the bedside, AI may function more as a clinical support layer than as a replacement for human care. For example, AI systems could help identify subtle changes in a patient’s condition before they become emergencies, assist nurses in tailoring patient education materials to an individual’s language preferences or literacy level, or help prioritize which patients require more immediate attention during busy shifts.

In hospitals, AI could also support staffing decisions, reduce unnecessary documentation burdens, and improve communication across teams. But the key point is that nurses remain autonomous, maintaining their relational and human-centered approach with patients and families. AI should augment nurses’ work, not replace their agency, empathy, ethical reasoning, and nuanced understanding that they bring to patient care.

LDI: What kinds of tasks could AI realistically take off nurses’ plates, and what parts of nursing should never be automated?

AI may realistically help with administrative or logistical tasks such as documentation support, supply management, scheduling assistance, summarizing patient information, or flagging concerning trends in clinical data. These are areas where automation can reduce burden without compromising the essence of nursing practice.

Nursing is not simply task completion. It involves critical thinking skills, ethical judgment, professional and moral accountability, evidence-based practice, and building trusting and therapeutic relationships with patients and families that AI cannot replicate. The application of AI-generated “advice” will still need critical judgment and evaluation by nurses to determine appropriate care. What should never be automated are the core human dimensions of nursing: compassion, advocacy, ethical decision-making, emotional support, and the ability to understand patients within the broader context of their lives and families.

LDI: From a patient’s perspective, how might care feel different if AI becomes more integrated into nursing practice?

Ideally, patients would experience more personalized and responsive care. AI could help tailor education materials, identify risks earlier, and support more proactive interventions. Patients may also benefit from nurses having more time for direct interaction rather than documentation.

At the same time, there is a risk that patients may feel distanced from care if AI is implemented poorly or in ways that reduce meaningful human interaction. The introduction of ambient listening, and knowing what they say is being recorded, might limit what is said and ultimately trust in the system. The challenge for health systems is ensuring that AI enhances the patient-nurse relationship rather than creating another technological barrier between patients and clinicians. Research is needed to better understand the ethical concerns of patients with the implementation of AI systems, including their privacy worries.

LDI: You raise concerns about bias and unreliable outputs in AI systems. What is a real-world example of how that could affect patient care?

One example involves predictive algorithms trained on incomplete or unrepresentative datasets. If an AI system is developed using data that underrepresents certain populations, such as older adults, rural communities, or racial and ethnic minorities, its recommendations may be less accurate for those groups.

For instance, a system designed to predict patient deterioration could fail to recognize warning signs equally well across populations, leading to delayed interventions or unequal care. Another concern is the lack of transparency or explainability, where AI operates as a “black box,” generating outputs that may include incorrect but highly convincing information.

There is a recent study that documented a high number of false citations — meaning fabricated published research references — in publications. When these false publications become integrated into systematic reviews, clinical guidelines, and decision-support systems, actions are not evidence based and can cause harm.

LDI: There’s a growing perception that AI could replace some health care jobs. How realistic is that when it comes to nursing?

It is unrealistic to think AI will completely replace nursing as a profession. Nursing involves far more than processing information or completing tasks. Nurses continuously interpret complex human situations, communicate with families, navigate ethical dilemmas, and provide emotional and psychological support.

What is more realistic is that AI will change nurses’ day-to-day work life. Some tasks may become automated or augmented, but that does not eliminate the need for professional nursing judgment. In fact, as health care becomes more technologically complex, the need for nurses who can critically evaluate and safely integrate AI into patient care may actually increase.

LDI: You argue that nurses need more training in AI. What does “AI literacy” actually mean for a working nurse?

AI literacy does not mean every nurse needs to become proficient in computer science methods and informatics tools. It means nurses should understand the basic capabilities and limitations of AI systems, including issues such as bias, accuracy, privacy, and reliability.

A nurse should be able to ask critical questions: Where did this recommendation come from? Was the system validated in populations similar to my patients? Could this output be biased or incorrect? AI literacy is really about preparing nurses to safely and responsibly work alongside these technologies rather than blindly trusting them.

LDI: Are nursing schools and hospitals currently prepared for this shift, or are they behind the curve?

AI is advancing much faster than educational and health care systems typically adapt. Most institutions are still catching up. Many nursing programs still lack structured AI education, and many hospitals are adopting AI tools before fully developing governance frameworks, training programs, or evaluation standards. The integration of clinical tools into curriculum and education is also a challenge.

There are important efforts underway, including recommendations from national nursing organizations, but we are still in an early transition period. The challenge is ensuring that implementation happens thoughtfully and proactively rather than reactively. At the University of Pennsylvania School of Nursing, we have formed several task force groups to examine the integration of AI into both education and research, and are working with our practice partners to generate guidelines for the integration of AI into nursing practice.

LDI: If an AI system contributes to a bad clinical decision, who is ultimately responsible — the nurse, the hospital, or the technology developer?

This is one of the critical unresolved ethical and legal questions in health care AI. Currently, responsibility is often diffuse across clinicians, institutions, and technology developers. However, AI systems themselves are not moral agents and cannot be held accountable in the same way humans or organizations can.

Ultimately, health care organizations and clinicians still retain responsibility for patient care decisions. That is precisely why rigorous oversight, validation, transparency, and governance are essential before deploying AI systems in clinical settings. As such, nurses must be part of the conversations that outline responsibilities for AI systems and address how inaccuracies or other missteps will be handled.

LDI: Do patients today even realize when AI is influencing their care, and should they be explicitly informed?

In many cases, patients probably do not realize the extent to which AI may already influence health care workflows or clinical decision-making. Yet AI can affect areas ranging from risk prediction to documentation to treatment recommendations. We need data to better understand patients’ preferences so they can make informed and voluntary decisions on the use of AI for treatment recommendations.

Patients should absolutely be informed when AI meaningfully contributes to decisions about their care. For example, they should be informed if ambient listening is used for documentation and have a clear understanding of what data are captured, stored, and used. Transparency is fundamental to trust. Ethical principles support the importance of informed consent and the right of patients to understand how their data are being used and how technology may shape clinical recommendations, especially as AI becomes more integrated into routine care.

LDI: There’s a common assumption that AI will save money and improve efficiency. Your paper suggests that’s not always the case. Why not?

AI implementation can introduce hidden costs and unintended consequences. Systems require training, maintenance, workflow redesign, oversight, cybersecurity protections, and continuous evaluation. In some cases, AI tools may even increase clinicians’ cognitive burden or documentation workload rather than reduce it.

There is also the issue of “algorithmic drift,” where performance changes over time as patient populations or workflows evolve. A system that initially appears effective may require substantial ongoing adjustment and monitoring. Efficiency gains are not automatic, and health care organizations need realistic expectations about both costs and benefits.

LDI: What is the biggest barrier hospitals face right now in safely adopting AI tools in nursing care?

One of the biggest barriers is the lack of robust governance and evaluation frameworks. Many organizations are eager to adopt AI quickly, but they may not yet have clear standards for validation, fairness assessment, implementation monitoring, or accountability.

Another major challenge is integrating AI into real-world clinical workflows. A technically impressive system can still fail if it does not fit how nurses actually deliver care or if it increases burden instead of reducing it.

LDI: You emphasize that nurses should help design AI systems, not just use them. What goes wrong when they’re left out of that process?

Nurses are critical to AI systems in health care because they represent a critical role in the implementation, use, interpretation, and evaluation of such systems. They bring the patient and family voice, and nurses’ everyday practice experiences, to systems as they are being developed. When nurses are excluded from design, AI systems often fail to reflect the realities of patient care. Developers may overlook workflow complexities, communication patterns, staffing needs, patient safety concerns, or the relational aspects of nursing practice.

This can lead to systems that are technically sophisticated but clinically impractical, burdensome, or even unsafe. Nurses bring expertise not only in patient care, but also in implementation, workflow integration, patient advocacy, and understanding how care decisions affect patients and families in real-world settings.

LDI: If everything goes right, what does a well-integrated AI future in nursing look like?

A successful future is one where AI quietly supports nursing care without overshadowing it. It is an issue of how nurses use AI versus how AI uses or dictates the work of nurses. Nurses would have better tools for identifying patient risks, accessing evidence, personalizing care, and reducing unnecessary administrative burden. Patients would experience safer, more equitable, and more responsive care.

Importantly, the future should remain deeply human-centered. AI should strengthen nurses’ ability to connect with patients rather than diminish it. The best AI systems will likely be the ones patients barely notice because they simply enable clinicians to provide better care.

LDI: What is the biggest mistake health systems could make right now as they rush to adopt AI?

The biggest mistake would be treating AI primarily as a cost-cutting technology rather than a process transformation that requires careful clinical and ethical oversight. If health systems prioritize speed, efficiency, or marketing over safety, transparency, equity, and clinician involvement, they risk introducing systems that may undermine trust, worsen disparities, burden clinicians, and potentially drive nurses out of the health care system.

Another major mistake would be excluding nurses and frontline clinicians from decision-making. Successful AI adoption depends as much on understanding patient care realities as it does on technical innovation.

---

---