Article Text
PDF
Abstract
Objective To investigate the impact of EyeSi surgical simulators on posterior capsule rupture (PCR) rates of cataract surgery performed by first and second year trainee surgeons.
Design A Royal College of Ophthalmologists’ National Ophthalmology Database audit study of first and second year surgeons’ PCR rates over seven consecutive National Health Service (NHS) years. Participating centres were contacted to ascertain the date when their surgeons had access to an EyeSi machine and whether this was on-site or off-site. Operations were classified as before, after or no access to EyeSi.
Setting The study took place in 29 NHS Ophthalmology Units in a secondary care setting.
Results Two-hundred and sixty five first and second year trainee surgeons performed 17 831 cataract operations. 6919 (38.8%) operations were performed before access to an EyeSi, 8648 (48.5%) after access to an EyeSi and 2264 (12.7%) operations by surgeons with no access to an EyeSi. Overall, there was a 38% reduction in the first and second year surgeon’s unadjusted PCR rates from 4.2% in 2009 to 2.6% in 2015 for surgeons with access to an EyeSi, and a 3% reduction from 2.9% to 2.8% for surgeons without access to an EyeSi. The overall first and second year unadjusted PCR rates for before, after and no access to EyeSi were 3.5%, 2.6% and 3.8%, respectively. The decrease in the with-access to an EyeSi group PCR rate was similar for surgeons with access to an EyeSi ‘on site’ or ‘off site’.
Conclusions First and second year trainee surgeons’ unadjusted PCR rates have decreased since 2009 which has significant benefits for patients undergoing cataract surgery. This 38% reduction in complication rates aligns with the introduction of EyeSi simulator training.
- treatment surgery
- public health
- medical education
Statistics from Altmetric.com
Introduction
Cataract surgery using phacoemulsification is the most frequently performed surgical procedure in Western countries. Advances in operative technology over the last 20 years have improved cataract surgery outcomes, with 95% of patients who have no ocular co-pathology achieving a visual acuity of 6/12 or better.1
Cataract surgery requires good stereoacuity and spatial awareness, combined with excellent hand–eye coordination and the ability to use all four limbs simultaneously, as foot pedals are used to control the operating microscope and the vacuum and/or the ultrasonic energy of the phacoemulsification handpiece. Even small incorrect movements, or applying too much ultrasonic energy or vacuum, can cause injury before the supervising surgeon can intervene.
The most frequently recorded intraoperative complication of cataract surgery is posterior capsule rupture (PCR) with or without vitreous loss and this is widely regarded as the benchmark complication to judge surgical quality.2 3
Following PCR, there is a significantly increased risk of retinal detachment (×42), endophthalmitis (×8) and of visual loss (×5.6).1 Although there are many ocular and systemic comorbidities which are associated with an increased risk of PCR, one of the most significant risk factors is the experience of the surgeon, where first and second year trainees have an adjusted OR of 3.73 for PCR occurring compared with a consultant surgeon.2
Historically, cataract surgery training has followed an experiential model, of acquiring skills in the operating theatre, which is frequently unstructured. Patient safety concerns alongside the demands of increased efficiency have been the main factors in developing alternative methods of training.
Wet-lab training using animal models or synthetic eyes designed specifically for phacoemulsification surgery enable trainees to rehearse some of the steps of cataract surgery, but they have been criticised for lacking any form of objective assessment.
In ophthalmology, the most commonly used virtual reality (VR) simulator is the EyeSi surgical simulator (VR Magic) which was first introduced into UK training programmes in 2010. This consists of a mannequin head, instruments, foot pedals and a virtual–reality interface, which is seen through the operating microscope (figure 1). The cataract interface includes modules for all of the major steps of phacoemulsification surgery in addition to abstract modules, which train basic skills.
An EyeSi surgical simulator (VR Magic).
The EyeSi simulator enables trainee surgeons to practice every step of the phacoemulsification procedure, apart from creating corneal incisions, and the automated assessment categorises performance in five domains: target achievement, efficiency, instrument utilisation, tissue damage and microscope usage.
The effectiveness of any training programme can be evaluated using Kirkpatrick’s four level model.4 Kirkpatrick’s fourth level evaluates the impact a training programme has had on an organisation, such as reduced costs, increased productivity or improved results.
The aim of this report was to investigate the impact of the introduction of EyeSi training on the PCR rates for operations performed by first and second year trainees over seven consecutive National Health Service (NHS) years. The study utilised cataract surgery data from the Royal College of Ophthalmologists’ National Ophthalmology Database (RCOphth NOD) audit.
Methods
The RCOphth NOD was established under the auspices of the Royal College of Ophthalmologists in 2010 to collate pseudonymised data collected as a byproduct of routine clinical care for the purposes of national audit. In 2014, the RCOphth NOD audit was commissioned to perform the National Ophthalmology Database Audit (NOA) by the Healthcare Quality Improvement Partnership (www.nodaudit.org.uk). The NOA receives data collected on electronic medical record (EMR) systems and in-house databases from centres performing NHS funded cataract surgery in England and Cymru, including NHS funded operations performed in independent treatment centres.
The study time period concerns NOA eligible cataract operations performed between 1 April 2009 and 31 March 2016 which corresponds to seven consecutive NHS years, where the first year includes the operations before the introduction of standard EyeSi training in 2010. Using the RCOphth NOD, 29 centres were identified with the centres first cataract surgery in the study period performed between 2009 and 2012, and where each centre had at least 50 operations performed by first and second year trainee surgeons.
The centres were contacted to determine when they introduced EyeSi training and which other hospitals in their region had access to their EyeSi. Of the 29 centres, 23 had either purchased or had access to an EyeSi between 2010 and 2016 and six centres had no access to an EyeSi during this period. The supplied date of EyeSi introduction and the date of surgery was used to allocate all operations performed by a first or second year surgeon to either the before, after or no access to EyeSi groups. Centres with access to an EyeSi were further categorised as access on site or off site.
An eligible cataract operation was defined according to the criteria used for eligibility in the National Ophthalmology Database Audit. Only operations performed by a first or second year trainee (ST1, ST2 or a Senior House Officer (SHO)) were included.
All data were recorded on the Medisoft electronic record system (Medisoft Ophthalmology, Medisoft Limited, Leeds, UK, www.medisoft.co.uk) which mandates the recording of intraoperative complications. PCR was defined as posterior capsule rupture with or without vitreous prolapse or zonule rupture with vitreous prolapse. Statistical comparisons used χ2 tests and all analysis was performed using STATA V.14.
Patient and public involvement
Patients, carers and members of the public were not directly involved with this study.
Results
Over the study period, 265 first and second year surgeons performed 17 831 operations, 7711 (43.2%) by second year surgeons, 6455 (36.2%) by first year surgeons and 3665 (20.6%) by SHOs.
Overall, 6919 (38.8%) operations were performed by surgeons before they had access to an EyeSi, 8648 (48.5%) after they had access to an EyeSi and 2264 (12.7%) operations by surgeons with no access to an EyeSi. The proportion of operations performed by first and second year trainee surgeons with access to an EyeSi increased in each year of the study (figure 2).
The proportion of operations performed by first and second year surgeons from 29 contributing centres, presented as the proportion of operations within each NHS year according to the surgeons access to an EyeSi, either before or after the centre had access or in a centre with no access. The NHS year runs from 1 April to 31 March. NHS, National Health Service.
The operation case complexity according to the National Cataract Audit PCR risk factors were very similar between the before, after and no access to an EyeSi groups, with minor differences for the presence of some ocular co-pathologies which are low prevalence risk factors in themselves (table 1).
The percentage of operations performed by first and second year trainee surgeons according to the RCOphth NOD audit cataract surgery PCR risk factors
Over the study period, the unadjusted for case complexity PCR rates decreased for the with-access to an EyeSi group (figure 3). The PCR rates for the access to an EyeSi was ‘on site’ or ‘off site’ (2.8% vs 3.1%) with a difference of −0.4% (95% CI −0.9% to 0.2%; p=0.174) (figure 4). Overall, there was a 38% reduction in the first and second year surgeons unadjusted PCR rate from 4.2% in 2009 to 2.6% in 2015 for those trainees with-access to an EyeSi, a difference in proportions of 1.5% (95% CI 0.5% to 2.6%; p=0.003). For trainees without access to EyeSi, there was a 3% reduction from 2.9% to 2.8% over this time, a difference of 0.1% (95% CI −4.1% to 4.2%; p=0.975).
Unadjusted for case complexity PCR rates with 95% CIs for first and second year trainee surgeons in each NHS year, presented separately for operations performed by surgeons with access to an EyeSi and for surgeons without access to an EyeSi. The NHS year runs from 1 April to 31 March. NHS, National Health Service; PCR, posterior capsule rupture.
Unadjusted for case complexity PCR rates with 95% CIs for first and second year trainee surgeons who had access to an EyeSi in each NHS year, presented separately for surgeons who had access to an EyeSi on site and for surgeons whose access to an EyeSi was off site. The NHS year runs from 1 April to 31 March. NHS, National Health Service; PCR, posterior capsule rupture.
The overall first and second year surgeons unadjusted for case complexity PCR rates for before, after and no access to EyeSi were 3.5%, 2.6% and 3.8%, respectively. The RCOphth NOD PCR case complexity model for the reported cases confirmed that the case complexity was virtually identical for these three groups. Comparisons of the observed unadjusted PCR rate between these groups were as follows:
Before versus after access to EyeSi: 3.5% versus 2.6%; difference in proportions=0.9% (95% CI 0.4% to 1.5%; p=0.001).
Before versus no access to EyeSi: 3.5% versus 3.8%; difference in proportions=−0.3% (95% CI −1.2% to 0.6%; p=0.546).
After versus no access to EyeSi: 2.6% versus 3.8%; difference in proportions=−1.2% (95% CI: −1.2% to −0.3%; p=0.002).
Discussion
Construct validity studies have shown that the EyeSi can distinguish between novice and intermediate or experienced surgeons,5 6 and studies have demonstrated that technical skills acquired with EyeSi training are transferable to the operating theatre.7
Thomsen et al 8 found that there was a high correlation between the automated EyeSi performance scores and real-life cataract surgery performance, as measured by motion tracking metrics. The same group also demonstrated that a proficiency-based EyeSi training programme improved surgical performance by 32% in novice cataract surgeons (had only performed steps of surgery) and 38% in intermediate surgeons (1–75 cases) as measured by an Objective Structured Assessment of Cataract Surgical Skill rating.9
However, studies which have attempted to investigate whether simulation training reduced the incidence of surgical complications have been small, underpowered and without comment on case complexity.10
In the surgical literature, there is a wealth of evidence that skills developed during virtual reality training for laproscopic surgery 11 and endovascular surgery 12 are transferable to the operating theatre. There is also evidence that virtual reality training results in faster operating times, a reduced number of unnecessary movements and better video scores, in cadaveric animal surgery.
The randomised double-blind trial by Seymour et al 13 was the first study to provide evidence that simulation training can reduce the risk of complications during live surgery. It compared the performance of 16 surgical residents who had undergone laproscopic virtual reality training against 10 residents who had undergone standard training, demonstrating a five times higher risk of gall bladder injury and burns to non-target tissue and a six times higher chance of having an senior surgeon take over error in the standard training group. However, there have been no large multicentre studies, in any surgical specialty, which have assessed the impact of simulation training on patient outcomes.
In 2016, a systematic review and meta-analysis of 31 randomised trials that compared virtual reality training in laproscopic surgery to other models of training, or to no training, concluded that virtual reality simulation was significantly more effective than video trainers and at least as good as box trainers.14 The authors’ final comment was that ‘further studies to assess the impact on patient outcome and on hospital costs are necessary’.
The unadjusted for case complexity overall PCR rate in the UK remained relatively constant from 2001 to 2010 at close to 2%.1 15 The RCOphth NOD has since demonstrated a 30% reduction in the overall unadjusted for case complexity PCR rate from 2.0% in 2010 to 1.4% in 2017.16
However, for trainees in their first 2 years of training, the unadjusted PCR rate for surgery prior to 2006 was in the order of 5%, over double that of an experienced surgeon.15 17 These studies have also shown the long learning curve for phacoemulsification surgery and the higher risk of complications during the first 75–80 cases.7 15
This series of 17 831 cataract operations performed by 265 first and second year trainees, in 29 UK centres, is the largest reported series of cataract operations performed by junior trainees. A distinct strength of this study is that the RCOphth NOD extracts highly structured data conforming to a nationally agreed data set from contributing centres, and that the recording of intraoperative complications is mandated on the EMR system used by participating centres in this study. Although this analysis is retrospective, the clinical data is gathered prospectively as part of routine clinical practice, in hospitals that use an EMR as part of their cataract patient pathway.
Overall between 2009 and 2016, there was a 38% reduction in trainee surgeons’ unadjusted PCR rate for first and second year surgeons with access to EyeSi training, and a 3% reduction for surgeons without access to an EyeSi simulator. The changes in the unadjusted PCR rate are influenced by the limited number of centres with data for the qualifying period, the relatively small sample for the without access to EyeSi group, and we cannot rule out that a surgeon in a centre without access to an EyeSi could have had access in a previous placement. Even with these limitations, the observed 38% reduction has beneficial implications for patient care.
The observed reduction in first and second year surgeons’ unadjusted for case complexity PCR rate was similar regardless of if the EyeSi was on site or off site. It is also worth noting that the comparable reduction in PCR rates was achieved even though some trainees had to travel up to 35 miles to access EyeSi training. This is an important finding for training resource allocation, as a centre may not have the funds to purchase an EyeSi, but they could possibly arrange for their trainee surgeons to access the EyeSi in another centre within their deanery.
In our study, it was not possible to quantify which EyeSi modules individual trainees had completed, nor what scores they had achieved, because the majority of units with an EyeSi did not record this information. If the EyeSis in the UK were networked, as some are in the USA, it would be possible to prospectively collect this information and for future NOD studies to investigate whether there is a correlation between EyeSi proficiency and reduced PCR rates.
Previous RCOphth NOD studies have established the link between ocular and systemic comorbidities and PCR rates.18 In the before, after and no access to EyeSi simulation cohorts, there was very little difference in either the systemic or ocular comorbidities that are associated with higher PCR rates. Variations in case complexity can therefore not be used to explain the difference in PCR rates.
Another possible confounding factor that could account for the reduction in PCR rates would be improvements in surgical technology, such as the introduction of silicone capsule guard tips for irrigation/aspiration hand pieces, but these had not yet come into use in the UK during this study period.
We did consider the possibility that if a hospital did not have access to an EyeSi, this could be indicative of a poorer training programme, both financially and in terms of manpower. However, even if this were to be the case, the fact that the PCR rates for trainees before the advent of EyeSi training were lower in these units than those that went on to have access to an EyeSi (figure 3) suggests that the quality of mentor facilitated training was not affected by any possible disparity in funding.
Increasing expectations on the part of the public have stimulated discussion around the need for transparency in terms of quality of services and quality of surgical outcomes. This RCOphth NOD study contributes to this spirit of transparency of surgical outcomes and the results should be reassuring for patients, as they demonstrate that the risk of having a serious complication during cataract surgery performed by a trainee surgeon has decreased in recent years; with further developments in operative technology and teaching methods, it is feasible for this reducing trend to continue.
Currently, around 450 000 NHS funded cataract operations are performed each year in the UK and approximately 5% of these are performed by first and second year surgeons.15 Using these numbers and the observed reduction from 3.5% to 2.6% between the before and after access to EyeSi groups, this extrapolates to a conservative estimate of ≈200 fewer cases of PCR annually for junior trainee surgeons. A less conservative estimate would be ≈360 fewer cases of PCR annually, by using the observed reduction in the junior surgeon unadjusted PCR rate of 4.2% in 2009 and 2.6% in 2015. The benefits of avoiding PCR for a patient are the increased chance of a good postoperative visual acuity and decreased chances of infection or further eye surgery (eg, for retinal detachment).
The main barrier to improved patient outcomes through simulation-based training is cost. La Cour and colleagues addressed the issue of the cost of simulation training in the 2018 Jules Gonin lecture entitled ‘Simulators in the training of surgeons: is it worth the investment in money and time?’.19
The additional cost of a PCR in terms of extra surgical equipment and outpatient visits has been calculated as being between £1178.20 and £2124.67.20 If a patient goes on to develop a retinal detachment or endophthalmitis, additional costs of £5000 or more could be incurred. The other factor to take into consideration when calculating the true cost of PCR is the cost of medical negligence claims. Ali and Little21 analysed 324 malpractice claims from the NHS Litigation Authority related specifically to cataract surgery, from 1995 to 2008, and found that negligent surgery (including PCR and dropped nucleus) was the most frequent cause for a claim. The mean damages for a paid claim were £19 900. Taking these additional factors into consideration, we estimate that the average cost to the NHS of a PCR is in the region of £2000. If EyeSi training reduces PCR by 280 cases per annum, this equates to a saving of approximately £560 000 per annum.
The majority of the 20 EyeSi simulators in the UK have been purchased with charitable funds and not by NHS Trusts or by Deaneries. However, even if this were not the case, by our calculations, the cost of the 20 EyeSi simulators would be recouped within 4 years. Furthermore, the ongoing cost savings to the NHS would more than cover the £10 000 per annum cost of software updates for each EyeSi.
This analysis of Kirkpatrick’s fourth level of evaluation of a training programme, that is, its impact on the organisation, enables us to conclude that EyeSi training is highly effective as it not only improves ‘results’ but it also reduces costs for the ‘organisation’.
This is the first multicentre study, in any surgical specialty, to demonstrate an association between the introduction of simulation training and a reduction in complication rates during live surgery. We believe consideration should be given to making EyeSi training a more integral part of training programmes worldwide.
Acknowledgments
It is with deep regret that we note the death of our friend and colleague Robert Johnston, who sadly died in September 2016. Without his inspirational vision, determination and career long commitment to quality improvement in ophthalmology this work would not have been possible. We would like to acknowledge the support and guidance we have received from the National Audit Steering Committee which includes professional members, ophthalmologists and optometrists, and patient and public representatives with individual lay members as well as patient support groups being represented. We acknowledge the support of the hospitals that participated in the National Ophthalmology Audit and thank our medical and non-medical colleagues for the considerable time and effort devoted to data collection. The 29 participating centres included in this study are listed in alphabetic order below. Aintree University Hospital NHS Foundation Trust; Barking, Havering and Redbridge University Hospitals NHS Trust; Barts Health NHS Trust; Bradford Teaching Hospitals NHS Foundation Trust; Calderdale and Huddersfield NHS Foundation Trust; Cardiff & Vale University LHB; East Sussex Healthcare NHS Trust; Epsom and St Helier University Hospitals NHS Trust; Frimley Health NHS Foundation Trust; Gloucestershire Hospitals NHS Foundation Trust; Hampshire Hospitals NHS Foundation Trust; Harrogate and District NHS Foundation Trust; Heart of England NHS Foundation Trust; King's College Hospital NHS Foundation Trust; Leeds Teaching Hospitals NHS Trust; MEH at Croydon Health Services NHS Trust; Mid Cheshire Hospitals NHS Foundation Trust; Norfolk and Norwich University Hospitals NHS Foundation Trust; North West Anglia NHS Foundation Trust; Oxford University Hospitals NHS Foundation Trust; Portsmouth Hospitals NHS Trust; Royal Berkshire NHS Foundation Trust; Royal United Hospitals Bath NHS Foundation Trust; Sheffield Teaching Hospitals NHS Foundation Trust; Shrewsbury and Telford Hospital NHS Trust; The Mid Yorkshire Hospitals NHS Trust; University Hospitals Birmingham NHS Foundation Trust; University Hospital Southampton NHS Foundation Trust; Wirral University Teaching Hospital NHS Foundation Trust. We acknowledge VR Magic for granting permission to publish the image of the EyeSi surgical simulator.
References
Footnotes
Presented at This work was presented as a poster at the Royal College of Ophthalmologists Annual Congress in May 2018.
Contributors JDF designed the study, wrote the manuscript and is responsible for the overall content. PHD provided statistical analysis and contributed to the writing of the manuscript. RLJ designed the study and contributed to creation of the NOD. BB and OO contributed to compiling the NOD data. JMS contributed to compiling the NOD data and the writing of the paper.
Funding This study was supported through The Healthcare Quality Improvement Partnership (HQIP) commissioned Royal College of Ophthalmologists (RCOphth) National Ophthalmology Database Audit, which is part of the National Clinical Audit and Patient Outcomes Programme (NCAPOP) and the Clinical Outcomes Programme (COP). PHD, RLJ, BB, MO and JMS had financial support from HQIP, the submitted work being a ‘spin-off’ analysis from the National Cataract Audit. RLJ owned a stake in and worked as a Medical Director for Medisoft Ophthalmology Limited. The authors have no other financial relationships with any organisation that might have an interest in the submitted work and no other relationships or activities that could appear to have influenced the submitted work. The views expressed are those of the author(s) and not necessarily of the NHS, HQIP, RCOphth or the Department of Health.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Ethical approval is not required for National Ophthalmology Database studies.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data are available upon request.
Linked Articles
- At a glance