Background The review of human participant research by Research Ethics Committees or Institutional Review Boards is a complex multi-faceted process that cannot be reduced to an algorithm. However, this does not give RECs/ IRBs permission to be inconsistent in their specific requirements to researchers or in their final opinions. In England the Health Research Authority coordinates 67 committees, and has adopted a consistency improvement plan including a process called “Shared Ethical Debate” where multiple committees review the same project. Committee reviews (...) are compared for consistency by analysing the resulting minutes. Methods We present a description of the ShED process. We report an analysis of minutes created by research ethics committees participating in two ShED exercises, and compare them to minutes produced in a published “mystery shopper” exercise. We propose a consistency score by defining top themes for each exercise, and calculating the ratio between top themes and total themes identified by each committee for each ShED exercise. Results Our analysis highlights qualitative differences between the ShED 19, ShED 20 and “mystery shopper” exercises. The quantitative measure of consistency showed only one committee across the three exercises with more than half its total themes as top themes. The average consistency scores for the three exercises were 0.23, 0.35 and 0.32. There is a statistically significant difference between the ShED 19 exercise, and the ShED 20 and mystery shopper exercises. Conclusions ShED exercises are effective in identifying inconsistency between ethics committees and we describe a scoring method that could be used to quantify this. However, whilst a level of inconsistency is probably inevitable in research ethics committee reviews, studies must move beyond the ShED methodology to understand why inconsistency occurs, and what an acceptable level of inconsistency might be. (shrink)

The review of human participant research by Research Ethics Committees or Institutional Review Boards is a complex multi-faceted process that cannot be reduced to an algorithm. However, this does not give RECs/ IRBs permission to be inconsistent in their specific requirements to researchers or in their final opinions. In England the Health Research Authority coordinates 67 committees, and has adopted a consistency improvement plan including a process called “Shared Ethical Debate” where multiple committees review the same project. Committee reviews are (...) compared for consistency by analysing the resulting minutes. We present a description of the ShED process. We report an analysis of minutes created by research ethics committees participating in two ShED exercises, and compare them to minutes produced in a published “mystery shopper” exercise. We propose a consistency score by defining top themes for each exercise, and calculating the ratio between top themes and total themes identified by each committee for each ShED exercise. Our analysis highlights qualitative differences between the ShED 19, ShED 20 and “mystery shopper” exercises. The quantitative measure of consistency showed only one committee across the three exercises with more than half its total themes as top themes. The average consistency scores for the three exercises were 0.23, 0.35 and 0.32. There is a statistically significant difference between the ShED 19 exercise, and the ShED 20 and mystery shopper exercises. ShED exercises are effective in identifying inconsistency between ethics committees and we describe a scoring method that could be used to quantify this. However, whilst a level of inconsistency is probably inevitable in research ethics committee reviews, studies must move beyond the ShED methodology to understand why inconsistency occurs, and what an acceptable level of inconsistency might be. (shrink)

We have followed with interest the commentaries arising from Moore and Donnellys1 argument that authorities in charge of research ethics committees should focus primarily on establishing code-consistent reviews.1 We broadly agree with Savulescu’s2 argument that ethics committees should become more expert, but in a different way and for a different reason. We have recently been working with the UK Health Research Authority analysing the outcomes of their ‘Shared Ethical Debate’ exercises.3 Each ShED exercise involves the circulation of a single research (...) project to a number of RECs. The resulting minutes from each REC are compared along with the final decisions made by each REC on the project under consideration. This process was originally an administrative exercise designed to promote greater consistency among the 60 or so ethics committees that were brought under the HRA ) umbrella within the UK National Health Service. Over 20 ShEDs have been run, and although the process has significant weaknesses and has not always been run consistently, the results of the exercises are fascinating. Broadly speaking, NHS RECs have been getting more consistent over time in terms of their decision-making, but the reasons for the final decisions as described in the committee minutes continue to vary widely. Qualitative research now needs to be done to understand why different committees can have such different discussions in relation to exactly the same research project and yet come to essentially the same conclusion. As part of our analysis of the ShED data, we looked at …. (shrink)

BackgroundPublication and outcome reporting bias is often caused by researchers selectively choosing which scientific results and outcomes to publish. This behaviour is ethically significant as it distorts the literature used for future scientific or clinical decision-making. This study investigates the practicalities of using ethics applications submitted to a UK National Health Service research ethics committee to monitor both types of reporting bias.MethodsAs part of an internal audit we accessed research ethics database records for studies submitting an end of study declaration (...) to the Hampshire A research ethics committee between 1st January 2010 and 31st December 2011. A literature search was used to establish the publication status of studies. Primary and secondary outcomes stated in application forms were compared with outcomes reported in publications.ResultsOut of 116 studies the literature search identified 57 publications for 37 studies giving a publication rate of 32 %. Original Research Ethics Committee applications could be obtained for 28 of the published studies. Outcome inconsistencies were found in 16 of the published studies.ConclusionsThis study showed that the problem of publication and outcome reporting bias is still significant in the UK. The method described here demonstrates that UK NHS research ethics committees are in a good position to detect such bias due to their unique access to original research protocols. Data gathered in this way could be used by the Health Research Authority to encourage higher levels of transparency in UK research. (shrink)

Background In their paper “Research approvals iceberg: how a ‘low-key’ study in England needed 89 professionals to approve it and how we can do better” Petrova and Barclay highlight concerns with the health research regulatory environment in the UK. Discussion As long-standing chairs of NHS research ethics committees, researchers, and also academics in research ethics, we are also often frustrated with the regulatory process in the UK. However, we think that Petrova and Barclay’s analysis is misleading because it conflates research (...) ethics with governance and funding processes, thus failing to adequately distinguish between the national coordinating function of the Health Research Authority, local research governance processes, and interactions with research sponsors and/or the Clinical Research Network. (shrink)

Valid consent requires the potential research participant understands the information provided. We examined current practice in 50 proposed Clinical Trials of Investigational Medicinal Products to determine how this understanding is checked. The majority of the proposals ( n = 44) indicated confirmation of understanding would take place during an interactive conversation between the researcher and potential participant, containing questions to assess and establish understanding. Yet up until now, research design and review have not focussed upon this, concentrating more on written (...) material. We propose ways this interactive conversation can be documented, and the process of checking understanding improved. (shrink)

Successful clinical trials are important for all of us, but they can be extremely complicated to design and run, so work must be done to consider what commonly goes wrong and how these issues can be addressed. Gelinas et al suggest an ethical argument for institutional prioritisation of clinical trials conducted among limited populations. This is to ensure successful recruitment and prevent competing trials rendering each other irrelevant through lack of statistical power. But they overlook the fact that effective prioritisation (...) already occurs, and their suggestion produces yet another hurdle for researchers to overcome. Their argument hinges around the claim that allocation of participants to trials represents an inevitable rationing scenario and, like all rationing scenarios, the best methods should be used for objectively determining how limited resources are distributed to achieve the end goal.1 Although they acknowledge that both individuals and clinicians may have preferences as to which trial a patient participates in, trial interventions are by definition experimental, and therefore choosing between trials is not really analogous to choosing between alternative clinical interventions. Here it seems that although clinical instinct is important when determining treatment options, a clinical trial is a specific attempt to create systematic justified knowledge and thus individual patient/clinician autonomy …. (shrink)

BackgroundWe conducted a survey to identify what types of health/medical research could be exempt from research ethics reviews in Australia.MethodsWe surveyed Australian health/medical researchers and Human Research Ethics Committee members. The survey asked whether respondents had previously changed or abandoned a project anticipating difficulties obtaining ethics approval, and presented eight research scenarios, asking whether these scenarios should or should not be exempt from ethics review, and to provide comments. Qualitative data were analysed thematically; quantitative data in R.ResultsWe received 514 responses. (...) Forty-three per cent of respondents to whom the question applied, reported changing projects in anticipation of obstacles from the ethics review process; 25% reported abandoning projects for this reason. Research scenarios asking professional staff to provide views in their area of expertise were most commonly exempted from ethics review ; scenarios involving surplus samples and N-of-1 studies were most commonly required to undergo ethics review. HREC members were 26% more likely than researchers to require ethics review. Need for independent oversight, and low risk, were most frequently cited in support of decisions to require or exempt from ethics review, respectively.ConclusionsConsiderable differences exist between researchers and HREC members, about when to exempt from review the research that ultimately serves the interests of patients and the public. It is widely accepted that evaluative research should be used to reduce clinical uncertainties—the same principle should apply to ethics reviews. (shrink)