News

How AI’s Growing Role in Nursing Raises Questions About Safety, Ethics, and Human Care

Penn Nursing Report Cautions That AI Systems May Add Rather Than Reduce Costs and Workflow Burdens

May 27, 2026

By:

Hoag Levins

Print:

A new paper from the University of Pennsylvania School of Nursing explores the potential and challenges involved in the AI automation of the clinical nursing space. (Generated image: Hoag Levins)

As artificial intelligence systems spread through hospitals and clinics, a growing debate is emerging over whether the technology will ultimately strengthen nursing care — or gradually replace parts of it.

That tension is at the center of a new University of Pennsylvania School of Nursing report, “Artificial Intelligence and Nursing Science: Opportunities, Challenges, Implications, and Guidelines,” published in the May-June 2026 edition of Nursing Outlook. The paper warns that while AI could reduce paperwork and improve patient monitoring, it also raises concerns about bias, accountability, patient privacy, and whether hospitals may view some nursing functions as replaceable.

Biggest Barriers

“One of the biggest barriers hospitals face in safely adopting AI tools in nursing care is the lack of robust governance and evaluation frameworks,” said Penn Nursing Dean Antonia Villarruel, PhD, RN, a co-author of the paper. “Many organizations are eager to adopt AI quickly, but they may not yet have clear standards for validation, fairness assessment, implementation monitoring, or accountability. Another major challenge is integrating AI into real-world clinical workflows. A technically impressive system can still fail if it does not fit how nurses actually deliver care or if it increases burden instead of reducing it.”

Companion Article

Penn Nursing Leaders Speak Out on AI’s Growing Role in Patient Care

Q&A Looks at Concerns Over Patient Trust, Workforce Pressures, Unreliable Outputs, and Hospitals Rushing AI Adoption

Penn Nursing is widely regarded as one of the world’s premier nursing schools, known for influential research in health policy, aging, chronic disease, health equity, maternal and child health, palliative care, and AI-driven patient care. Its Center for Health Outcomes and Policy Research (CHOPR) is internationally recognized for landmark studies showing how hospital staffing, workplace culture, and organizational structure affect patient outcomes, nurse retention, patient mortality, and the quality and safety of care.

Reshaping Nursing Science

This latest paper examines how artificial intelligence is reshaping nursing science, outlining both its transformative potential and the risks it poses to clinical practice, ethics, and research.

Drawing on a two-day interdisciplinary workshop at Penn, the authors argue that nursing science is uniquely positioned to guide AI integration because of its focus on patient-centered care, clinical workflow, and advocacy. AI can enhance nursing by automating administrative tasks, accelerating research, improving clinical decision-making, and enabling more personalized patient education and care. It also offers new ways to analyze large datasets to identify patient needs and population health risks.

Major Challenges

However, the paper emphasizes several major challenges. These include low AI literacy among nurses, risks of biased or incomplete data, unresolved issues around the reliability and “hallucinations” of AI systems, and the danger that AI could erode the core human and ethical dimensions of nursing practice. The authors stress that AI cannot replicate key nursing functions such as empathy, moral judgment, and patient advocacy.

The paper also highlights broader ethical and social concerns, including patient consent for data use, transparency in AI-driven decisions, and unclear accountability when AI influences clinical outcomes. It warns against framing AI systems as “agents” equivalent to human clinicians, arguing that this risks blurring responsibility and undermining professional standards.

To address these issues, the authors propose a set of guidelines for integrating AI into nursing science:

Expanding AI Education in Nursing: Artificial intelligence literacy should become a standard part of nursing education, much like pharmacology or clinical informatics. Nurses and nurse scientists need to understand not only how to use AI tools, but also how those systems are built, where they can fail, and how bias or inaccurate outputs can affect patient care. The authors say future nurses should be trained to critically evaluate AI systems rather than simply trust them.
Integrate nurses in AI Development: Nurses should help design and evaluate AI systems from the beginning instead of being treated merely as end users. Because nurses work closest to patients and clinical workflows, they are uniquely positioned to identify what problems actually need solving and whether an AI system works safely in real-world care settings. The paper calls for nurses to participate in interdisciplinary teams that include engineers, data scientists, hospital leaders, ethicists, regulators, and patients.
Rigorously Testing AI Before Widespread Use: Many AI systems are entering health care before they have been adequately tested in real clinical environments. The authors recommend using formal evaluation frameworks and implementation science methods to measure accuracy, reliability, bias, workflow impact, and patient outcomes. The goal is to ensure that AI systems improve care without introducing new safety risks or inequities.
Measuring the Real Financial and Workflow Costs: The authors caution against assuming that AI will automatically save hospitals money or reduce nurses’ workloads. They argue that nurse scientists should carefully study how AI affects staffing, documentation demands, cognitive burden, morale, and patient care continuity. In some cases, AI systems may create new burdens or hidden costs even if they appear efficient on paper.
Building Ethical Safeguards and Transparency: Ethical oversight must remain central as AI becomes more involved in patient care. The authors call for stronger safeguards around patient privacy, informed consent, data use, and bias prevention. They also argue that patients should better understand when AI is influencing clinical decisions and how their personal data may be used to train or operate those systems.

Finally, the paper calls for stronger partnerships between nursing scientists and industry, with nurses positioned not as peripheral contributors but as co-designers of AI systems. It concludes that for AI to improve health outcomes without compromising care quality or trust, nurse scientists must play a central role in its development, evaluation, and deployment.