Background Meaningfully evaluating the quality of institutional review boards (IRBs) and human research protection programs (HRPPs) is a long-recognized challenge. To be accredited by the Association for the Accreditation of Human Research Protection Programs (AAHRPP), organizations must demonstrate that they measure and improve HRPP “quality, effectiveness, and efficiency” (QEE). We sought to learn how AAHRPP-accredited organizations interpret and satisfy this standard, in order to assess strengths, weaknesses, and gaps in current approaches and to inform recommendations for improvement.Methods We conducted 3 (...) small-group interviews with a total of 19 participant representatives of accredited organizations at the 2019 AAHRPP annual meeting. Participants were eligible if they had familiarity with their organization’s approach to satisfying the relevant QEE standard.Results Participants reported lacking clear definitions for HRPP quality or effectiveness but described various approaches to assessing QEE, typically focused on turnaround time, compliance, and researcher satisfaction. Evaluation of IRB members was described as relatively superficial and information regarding research subject experience was not reported as central to QEE assessment, although participants described several efforts to improve consideration of patient, subject, and community perspectives in IRB review. Participants also described efforts to educate and build relationships with key stakeholders as important features of a high-quality HRPP. While generally satisfied with their approaches, participants expressed concern about resource and time constraints that pushed them to be reactive and automatic about QEE, rather than proactive and critical.Conclusions The relevant AAHRPP accreditation standard may obscure critical gaps in defining and measuring QEE elements. We recommend that AAHRPP: (1) offer a definition of QEE or require accredited organizations to provide their own, to help clarify the rationale and goals behind assessment and improvement efforts, and (2) require accredited organizations to establish QEE objectives and measures focused on participant outcomes and deliberative quality during protocol review. (shrink)

The primary purpose of Institutional Review Boards (IRBs) is to protect the rights and welfare of human research participants. Evaluation and measurement of how IRBs satisfy this purpose and other important goals are open questions that demand empirical research. Research on IRBs, and the Human Research Protection Programs (HRPPs) of which they are often a part, is necessary to inform evidence-based practices, policies, and approaches to quality improvement in human research protections. However, to date, HRPP and IRB engagement in empirical (...) research about their own activities and performance has been limited. To promote engagement of HRPPs and IRBs in self-reflective research on HRPP and IRB quality and effectiveness, barriers to their participation need to be addressed. These include: extensive workloads, limited information technology systems, and few universally accepted or consistently measured metrics for HRPP/irb quality and effectiveness. Additionally, institutional leaders may have concerns about confidentiality. Professional norms around the value of participating in this type of research are lacking. Lastly, obtaining external funding for research on IRBs and HRPPs is challenging. As a group of HRPP professionals and researchers actively involved in a research consortium focused on IRB quality and effectiveness, we identify potential levers for supporting and encouraging HRPP and IRB engagement in research on quality and effectiveness. We maintain that this research should be informed by the core principles of patient- and community-engaged research, in which members and key stakeholders of the community to be studied are included as key informants and members of the research team. This ensures that relevant questions are asked and that data are interpreted to produce meaningful recommendations. As such, we offer several ways to increase the participation of HRPP professionals in research as participants, as data sharers, and as investigators. (shrink)