Is Audit Enough?
Following the appointment of Dr Bill Taylor as the clinical lead for Quality Improvement in the Clinical Innovation and Research Centre of the RCGP we posed the question 'Is Audit Enough'? to both Dr Taylor and Dr Amanda Howe, Vice Chair of RCGP Council. Read below to find out their thoughts on the future of this stalwart of quality improvement in general practice.
Dr Bill Taylor, CIRC Clinical Lead for Quality Improvement
Clinical audit was introduced to general practice in the 1980s as a method of quality assurance. Since then it has become embedded in the GMC Duties of a Doctor and can be included in appraisal as one type of quality improvement activity. However at a time general practice is under pressure it is essential that the appropriate quality improvement tools are used to assist the introduction and sustain the change. My comments concentrate on the place of audit in the 'local' environment which means how it is used within a general practice rather than a national audit.
For an audit to be effective, all relevant staff need to be involved, the criteria need to be evidenced based and likely to influence the outcome, the data collected needs to be of good quality, and a full cycle, including a re-audit needs to be undertaken. The culture, leadership and skills within the practice need to allow the results of the audit to be actioned and change introduced and sustained. Time needs to be found to conduct the audit and feedback of results needs to be effective as it the case with any quality improvement activity. A Cochrane review in 2012 concluded that 'audit and feedback generally leads to small but potentially important improvements in professional practice.' In this review the audit methodologies differed and only 21 of the 121 trials originated in the UK.
The problem arises, even with a full cycle, in that the measurement is made on only two points of time. This does not allow for common-cause variation which is due to natural or ordinary causes. A change can be judged, in the first place to be needed and having been introduced, to have succeeded or failed on conclusions drawn from only one or two data collections. This can be in part overcome if repeated short data collection periods with small samples are undertaken and small changed introduced as part of a repeated Plan Do Study Act (PDSA) process. This can be considered as multiple re-auditing and can be referred to as a rapid cycle audit. The traditional audit with two data collections has been described as a quality assurance process rather than one with repeated collections which is a quality improvement process.
Other methods can both capture natural variation but also allow judgements to be made and action to be taken to improve outcomes. In terms of data capture, if it is important to look at a trend then 'run' and 'control' charts can provide the basis for making a judgement. A 'run' chart is a series of data points in chronological order. A 'control' chart is a run chart with upper and lower limits which are statistically calculated. Some general practitioners have undergone training in these methods but the majority have not.
Pilots in quality improvement methodology are being conducted with GP trainees at present with a view to incorporating quality improvement into the proposed extended training. Examples of projects using different quality improvement methods in general practice and relevant training need to be made available to all GPs. The RCGP is developing a strategy which will likely put quality improvement methods as a major focus of membership support for such work.
In the meantime audit should be continued to be used but its limitations need be recognised and when carried out it needs to be undertaken to a high standard. It may be enough but enough may not be good enough to produce effective change.
Dr Amanda Howe, Vice Chair of RCGP Council
I recall learning about audit when I was already an established GP - it became part of the MRCCP, so I had to supervise trainee projects. It was the first time I had really launched any kind of database project in the practice - we were excited, and interested. Electronic records helped. And we were keen to know if there were things we could do better - particularly to identify patients who were not taking up our preventative services. In a relatively deprived community, we started to think population as well as patients - it was a very early step towards a more systematic addressing of unmet need.
As Bill Taylor's comments suggests, however, it is one thing to identify an issue, and another thing to change practice to avoid the same gaps emerging for the future. Don Berwick's work on quality improvement suggests a more cyclical 'Plan-Do-Study-Act' and CQI (Continuous Quality Improvement) processes because they are the most commonly used approaches that have been successful in family practice settings. PDSA enables testing of changes on a small scale to determine if they are beneficial. Each small improvement builds knowledge and confidence and enables continued reflection on the quality of the practice.
We also need to pay attention to the embedding of change into routine practice. Too many innovations slip away because we do not take the whole system with us into a change. Carl May's 'normalisation process' draws our attention to the need to make sure that changes have cognitive and practical fit with current skills and work practices, and that any suggested changes made are followed and re-adjusted if they are proving difficult (or being resisted!). Patient safety issues often arise from 'work around' when staff fail to utilise new devices designed to improve safety because they do not 'fit' working practices. Both these bodies of work make the point that Bill makes - there is more to systematic quality improvement than audit. But it's a start!
Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O'Brien MA, Johansen M, Grimshaw J, Oxman AD.
Audit and feedback: effects on professional practice and healthcare outcomes. The Cochrane Library (Online) 2012, Issue 6.
Dixon N, Pearce M. Guide to using Quality Improvement tools. HQIP (Online) October 2011. Available from here.
NHS Scotland, Scottish Government. Healthier Scotland. The Improvement Model and PDSAs.
Berwick DM. A Primer on Leading the Improvement of Systems. BMJ. 1996;312:619-222.
May C et al Development of a theory of implementation and integration: Normalization Process Theory. Implementation Science. 2009, 4:29