Appendix to Report nr 10, 2015 (140 p.)
The Norwegian Directorate of Health develops clinical practice guidelines, and requested the Norwegian Knowledge Centre for the Health Services to summarise research findings on the effectiveness of guideline implementation interventions. This report is an overview of existing systematic reviews.
Key findings from the identified systematic reviews are that:
- Implementation interventions such as electronic decision-support, educational meetings, outreach visits, audit and feedback, and tailored interventions are probably effective, but:
- The size of the effect varies.
- The effect on clinical practice is most often moderate.
- The expected effect on health outcomes is modest.
- For other of the interventions, the size of the effect varied considerably across studies. It is difficult to explain this variation. Consequently, it is uncertain how much these interventions will improve adherence to clinical guidelines.
- For some measures, such as financial incentives and public release of performance data, evidence is lacking or scarce. We therefore cannot say how effective these types of interventions are.
The Norwegian Directorate for the Health Services is responsible for developing and supporting the implementation of clinical practice guidelines. Health professionals do not necessarily change their clinical practices when new guidelines are published, and there are many examples of gaps between clinical practice and recommendations in clinical practice guidelines. It is therefore unrealistic to assume that simply publishing guidelines will change practice.
In 2004, the National Health Services (NHS) in the UK published the report “Effectiveness and efficiency of guideline dissemination and implementation strategies”. This was a comprehensive systematic review of the effects of guideline implementation strategies. The report is still an important document, but is largely outdated. An updated review of available research evidence of the effects of interventions to improve adherence to clinical practice guidelines is therefore needed.
To review the current best available evidence of the effectiveness of various guideline implementation strategies.
We prepared an overview of systematic reviews. We used three sources for systematic reviews:
- An overview of systematic reviews we are in the process of preparing together with colleagues in the Cochrane Collaboration
- A supplementary search in the PDQ-Evidence database
- A limited search in PubMed
The authors of the Cochrane overview conducted several searches for systematic reviews over several years, including searches of PDQ-Evidence. The PDQ-Evidence database includes systematic reviews identified by searching 17 other databases, including PubMed and the Cochrane Library.
We summarised findings from systematic reviews that fulfilled the following criteria:
- Population (target group for the intervention): Health professionals
- Intervention: Any strategy for implementing clinical practice guidelines
- Comparison: No intervention, or another strategy for guideline implementation
- Outcomes: Professional behaviour among health professionals (adherence to clinical practice guidelines). We also included patient outcomes (health or health related behaviour), utilisation of health services, and resource use, if these were reported.
We included systematic reviews published in the last 10 years (after 2005), which did not have limitations that were important enough that we considered the results of the review to be unreliable. We excluded reviews that were limited to specific clinical topics.
We included 19 systematic reviews, 11 from the Cochrane overview and eight from the supplementary searches. The reviews address different guideline implementation strategies.
There is moderate certainty evidence that the following interventions probably increase adherence to clinical practice guidelines:
- Clinical decision-support systems (including reminders)
- Educational outreach visits (including «practice facilitation»)
- Audit and feedback
- Local opinion leaders
- Tailored interventions
- Educational meetings
We did not find evidence of moderate or high certainty that any strategy decreases adherence or is ineffective, although individual studies found little or no effect on adherence or even a decrease in adherence for nearly all of the strategies.
It is uncertain whether the following strategies improve adherence to clinical practice guidelines because the certainty of the evidence is very low:
- Internet-based learning
- Interprofessional education
- Printed educational materials
- Economic incentives
- Interprofessional collaboration
- Strategies to change organisational culture
- Public release of performance data
The quality of the evidence is generally too weak to draw conclusions about effects on patient outcomes, utilisation of health services, or resource use.
Our findings come from published systematic reviews. The main advantage with this approach is that it is efficient and reduces the risk of unnecessary duplication of systematic reviews. The main disadvantage is that we may have missed important findings of relevant studies that were not included in a systematic review, but the risk is probably small since most of the included reviews were published in recent years.
The impact of an implementation strategy may depend on specific characteristics of the strategy and the targeted health problem and professionals. For instance, the degree of participation in educational meetings varies and many targeted health professionals may not participate. The effectiveness of educational meetings may depend on the degree of participation. Similarly, the content of guidelines and health professionals’ attitudes to specific recommendations may influence the degree of adherence. This might explain some of the variation that we found in the effects of implementation strategies on adherence to guidelines.
Findings from qualitative studies support the logical assumption that the factors influencing guideline adherence will vary from one recommendation to another, and across contexts. It seems sensible, therefore, to identify determinants of clinical practice (barriers and facilitators of adherence) and to use implementation strategies that address those. Implementation strategies that are targeted at identified determinants of practice are referred to as tailored implementation strategies. We found good evidence that these strategies increase adherence, but the effect sizes were moderate, similar to other non-tailored strategies that increased adherence to clinical practice guidelines. Improved methods for identifying determinants of practice and for selecting implementation strategies that address those might lead to more effective guideline implementation.
Several guideline implementation strategies, including educational outreach visits, audit and feedback, educational meetings, and tailored interventions, have an effect on clinical practice. However, their impact varies and the size of the effects are most often small to moderate. The effects of several other implementation strategies are uncertain.