Holly Fernandez Lynch, JD, MBE, 
is an LDI Senior Fellow, the John Russell Dickson, MD Presidential Assistant Professor of Medical Ethics, and Assistant Faculty Director of Online Education in the Department of Medical Ethics and Health Policy at the Perelman School of Medicine. Mohamed Abdirisak is a 2018 LDI SUMR Scholar. 

Institutional Review Boards (IRBs)—or “Research Ethics Committees” (RECs) outside the United States—are charged with protecting human research subjects. However, the quality of IRB performance is rarely subject to the same scrutiny that the boards themselves apply to research projects. In AJOB Empirical Bioethics, we conducted the first systematic analysis of 10 IRB quality measurement tools. We found widespread variability in their format, length, and content; a common emphasis on IRB structure and process; a general lack of attention to participant outcomes; and a failure to identify clear priorities for assessment.

Generally, all tools focused on IRB structure (e.g., organization, expertise, approach) and process (e.g., timeliness, risk assessment, and consent), which are only proxy measures for actual participant protection. While some tools provided clear and robust explanations of the IRB quality standards they applied, others were simply checklists and questionnaires. The most prevalent topics covered in the tools include review type (exempt, expedited, full board), IRB member expertise, training and educational resources, and protocol maintenance.

Notably, 8 out of 10 tools had no mechanism for assessing IRB impact on participant outcomes— a significant oversight given IRB goals related to participant protection. The two tools that did consider participant outcomes included assessments based on adverse events, failures of informed consent, and consideration of participant experiences.

Our results have implications for the research ethics and policy communities.

First, since there are costs to assessing IRB quality, it is important to prioritize measures that evaluate the most important criteria. Accordingly, IRB quality assessments should center (as much as possible) on their relationship to participant outcomes.

Second, IRBs should focus on assessing the robustness of board deliberation (i.e., who is deliberating and how). Serious concerns about participant protection are unlikely to be identified and prevented without strong deliberative processes between the right types of board members and staff.

Third, quality assessments should also consider the content of board deliberation when evaluating proposed research. They should give special attention to procedures most likely to protect participants, such as risk assessment and minimization, consent review, review of scientific quality and social value, and plans for responding to adverse events.  

Protecting the rights and welfare of human research participants without unduly inhibiting valuable research is critical. It is even more important in the era of COVID-19, when research must move quickly without undermining public trust. Further work is essential to developing meaningful outcome measures of IRB quality, to validate components of quality assessment tools based on their relationship to participant protection, and to incorporate measures that promote the identification of and deliberation about research ethics challenges. Some of these efforts are currently underway. Going forward, IRBs should operate as learning systems by gathering data about their performance in relation to their goals of participant protection and research facilitation, and making continuous adjustments in response.


The study, Evaluating the Quality of Research Ethics Review and Oversight: A Systematic Analysis of Quality Assessment Instruments, was published online in AJOB Empirical Bioethics on August 21, 2020. Authors include Holly Fernandez Lynch, Mohamed Abdirisak, Megan Bogia, and Justin Clapp.