Improving productivity, costs and environmental impact in International Eye Health Services: using the ‘Eyefficiency’ cataract surgical services auditing tool to assess the value of cataract surgical services

Website and application design

The Eyefficiency tool consists of a website and mobile phone application (app) with a built-in toolkit on how to use the tool, thus avoiding the need for external support.23 The Eyefficiency tool successfully completed its first phase of piloting at four sites from the UK, India and South Africa in 2017.23 On confirming its feasibility, Eyefficiency garnered further support for continued development from the IAPB and Standard Chartered Bank’s Seeing is Believing grant. This helped the tool move on to its second phase of beta testing, which has completed two rounds, to date. This study reports the results of the second round of beta testing.

Beta sites were identified and each created a site-specific account on the website into which they entered facility-level data. The data entries were divided into five categories: basic information, which consisted of variables like facility location, number of operating theatres (OTs) used and number of cataract surgeries performed per year; patient pathway, which consisted of the number of appointments before and after surgery, number of patients on waiting list; staffing, where they entered information on the number and salary of staff involved in cataract surgical care; building where information on energy sources and the prices paid for them was collected; and materials which took into account the cost of surgical supplies and pharmaceuticals and end-of-life processes.

The sites then used the Eyefficiency app to record time-and-motion (TAM) data for their cataract surgeries. This phone app allowed the designated point person tasked with collecting TAM data to select the surgical procedure type and then press the time-stamp button at different milestones throughout the procedure. The procedural milestones are: (1) the patient is on the operating table, (2) the surgical drape is on the patient, (3) the first incision is made, (4) the incision is closed, (5) the drape is removed and (6) the patient is off the operating table. This allows the Eyefficiency tool to differentiate between the time spent physically operating on the patient and the time spent ‘turning over’ the OT in between patients. At the end of each surgical session, the surgical site can also record the total weight of surgical waste produced, using the Eyefficiency app. Prior to and after recording each case’s milestones, the user inputs data on additional factors that may impact this specific case, including the presence of clinicians-in-training and any operative complications that arose during the case.

Beta site selection and timeline

To test if the tool is universally accessible and acceptable and to capture regional variation, we sought to beta test at least one site in each of the seven IAPB/WHO regions. We contacted IAPB regional representatives to suggest surgical facilities that may have the capacity to test the tool, and out of the 28 shortlisted facilities, 24 met the requirements of the audit and agreed to participate. A point person at each beta site created a profile on the Eyefficiency website and downloaded the app, leading the beta testing at their facility. Of these initial 24 sites, 12 successfully completed the first audit round from October to December of 2018. This initial beta testing was used to improve the functionality and performance of Eyefficiency.

A second round of beta auditing ran from April through May 2019, with nine sites successfully completing the testing. Eight of the sites completed the first audit round and one site was new to Eyefficiency. Site attrition was primarily due to a lack of personnel available to conduct the in-surgery audit. In this paper, we report only the results of the second round of beta testing, from these nine sites spread throughout five IAPB regions.

The time spans were chosen to align with periods when all sites would be operating on cataracts (non-holiday times) and in a long enough window to ensure each site completed 1 week of, or at least 30 cases, whichever came first. The time between the two audit periods was used to improve Eyefficiency.

The beta test sites entered data about their facility into the Eyefficiency website and used the Eyefficiency app on a phone/tablet to gather information on their phaco, MSICS or ECCE cataract surgical sessions. After each audit round, the participating sites were administered an online qualitative survey asking them questions about their experiences using the tool. Two members of the Eyefficiency team also conducted semistructured interviews with each site, either in person or through video-conference calls, to discuss and validate the site’s results clarify any unexpected data, and enquire about strategies the site was developing to reduce their costs and carbon footprint or improve productivity.

Beta site data analysis

Case-to-case duration at each site was calculated by separating the type of procedure (phaco, MSICS, etc) and measuring the average number of minutes needed for each procedure step (ex: preoperative preparation in the OT, cut-to-close operating time, postoperative clean up where the patient is still in the OT and turnover time where no patients are present in the OT). SDs of each duration were calculated for each site for each type of procedure. The ‘cases per hour’ metric was calculated by taking the total recorded operating minutes and dividing by the total number of cases. In surgical lists that had both MSICS and phaco, MSICS cases per hour were calculated by separating out the MSICS cases and assigning half of the turnover time from before and after the MSICS case to that case. The remaining half of turnover time was assigned to the phaco case before and after the MSICS.

Supply and pharmaceutical costs per case were directly input by users. Cost of salaries per case was calculated using the user’s estimate of how long each staff member spent with a patient on the day of surgery and that staff member’s annual salary. Electricity costs were calculated by estimating the electricity intensity in kWh/m² of the OT from the facility’s electricity bill and multiplying by an OT intensity factor from literature. Based on the number of cases performed yearly in that OT, the average kWh per case was calculated and multiplied by the local price per kWh. This approach allocates OT downtime, that is, times when the OT is operational but not in use, to every case. Cost of laundry and sterilisation was either directly input by the user, or estimated based on assumed electricity use of the cleaning process and local price per kWh. Waste disposal costs were calculated based on prices input by the user and the amount of waste generated per case.

Carbon footprint or GHG emissions from activities at each site were calculated using a hybrid environmental LCA approach according to ISO 14040 standards. The LCA used both process LCA with physical data (including electricity use, commuting of staff and patients, reusable supply production and sterilisation, and waste treatment) and an environmentally extended input–output LCA with financial data (including the production of single use supplies) to estimate average emissions per case from each site. Further details on the methods can be found in Thiel et al’s 2020 article.23

After calculating the productivity (case duration), cost per case and GHG emissions per case, we compared across sites using descriptive statistics.

Site demographics

Of the initial 24 sites invited to participate in this second round of beta testing, 14 initiated the auditing process on either the Eyefficiency website or the app, and 9 sites fully completed the audit. The nine sites were located in five of the IAPB/WHO regions (with no sites from North America and Eastern Mediterranean Regions), and use seven different currencies (table 1, figure 1). Five of the sites have ‘dual bed’ systems, where each surgeon alternates operations on two side-by-side beds.

Request permission

Eyefficiency was used to monitor 475 cases across all beta sites and 30–187 cases per site, representing 0.4%–6.7% of each site’s annual caseload. Annual caseload of beta site locations ranges from 565 cataract surgeries per year to 47 102. All nine sites perform phaco, six perform MSICS, four perform ECCE and two perform ‘other’ approaches (such as cataract surgery with the femtosecond laser). The number of phacos each site measured using Eyefficiency ranged from 10 to 52 per site. Of the MSICS sites, one site logged only 1 case through Eyefficiency, two sites logged 5 cases, one site logged 22 cases and one site logged 133 cases (table 1). After data validation, no ECCE cases were recorded. Given the lack of data on ECCEs, their results are not discussed further in this paper. Other approaches, such as the femtosecond laser, are not yet supported in Eyefficiency and were not measured.

Every site reported surgeries conducted by trainee surgeons, with 123 out of the 475 recorded cases having a trainee involved. At each site, trainee cases ranged from 3 to 54 cases per site or 6%–48% of total cases recorded per site. The Eyefficiency tool also allows sites to record common surgical complications that may occur. These complications include phaco wound burn and posterior capsule rupture. Of all 475 recorded cases, there were 45 cases (9%) with complications. Three sites reported 0 complications, two reported 1 case with complications (0.5%–2% of recorded cases), one reported 3 cases (10%), one reported 7 (18%), one reported 12 (32%) and one reported 21 (66%).

Sites operate on cataracts between 2 and 6 days a week (average and median of 4). For patient pathways, the beta sites conduct between one and five preoperative visits (median of 2) with each patient (separate from the day of surgery) for average cases. Postoperative visits range between one and three visits per site, with a median of 2. For staffing needed prior to surgery, an average of two nurses per site saw patients preoperatively, with a maximum of four (India) and a minimum of zero (Mexico 2). Other staff members required for preoperative appointments, including booking or administrative personnel, ranged from one to five people per site, with a median of 1. On the day of surgery, the total number of staff that came into contact with the patient was highest for India, at 10 staff members, and lowest for South Africa with 5 staff members. The median number of staff members for across all sites was 6.

Productivity, throughput or use of operating time

Average case-to-case duration across sites ranged from 13 min (Chile) to 71 min (Mexico 2) for phaco, with a cross-site average of 36 min. For MSICS, average case-to-case duration ranged between 22 min (India) and 88 min (Mexico 2), with an average of 50 min (figure 2). Of note, the case-to-case duration looks only at a single bed and does not account for the potential efficiencies of a dual bed system. Of the sites operating on a single bed, surgical teams spent an average of 25% (South Africa phaco) to 56% (Hungary phaco) of the case-to-case duration actually operating (cut-to-close). Turnover time (time without the patient in the operating room (OR)) accounted for 5% (Chile phaco) to 40% (South Africa and UK phaco) of total case-to-case duration at each single-bed site. Cases per hour (for combined phaco and MSICS) ranged from 1.7 to 4.48 at single-bed sites and from 1.51 to 4.25 at dual-bed sites.

Request permission

Average cost per case

Converting all local currencies to 2017 Great British pound (GBP) (prior to the COVID-19 pandemic and any economic or supply issues therein), price per case ranged between £31.55 (India) and £399.34 (New Zealand), with the price of disposable supplies making up more than 70% of expenditures per case (both phaco and MSICS) at all sites except Eswatini (also known as Swaziland), where supply spending makes up 38% of all measured expenses (figure 3). The next highest category of expenditures at all sites was for staff salaries, ranging from 2% (India; and still the second highest spending category) to 45% of per-case expenditures. For Eswatini, staff salaries represented the highest spending, at 45% of per-case expenses. Energy expenses per case ranged from 15% of total expenses (Eswatini), to less than 3% at all other sites.

Request permission

Environmental footprint

Average weight of solid waste generation per phaco ranged between 0.19 kg per case (Eswatini) and 4.27 kg (UK), a 22-fold difference between the lowest generating and highest generating sites. For MSICS, solid waste at each site ranged between 0.18 kg per case (Eswatini) and 2.29 kg (New Zealand), shown in figure 4.

Request permission

Average per-case GHG emissions ranged from 41 kg carbon dioxide equivalent (CO2e) (India) to 130 kg CO2e (Hungary) for phaco. This is equivalent to driving a passenger vehicle between 164 and 522 km.24 For MSICS, GHG emissions ranged between 40 kg CO2e (India) and 119 kg CO2e (New Zealand), shown in figure 5. This is equivalent to driving a car between 163 and 480 km. At four sites (Mexico 1, Chile, India and South Africa), majority of emissions for phaco and MSICS originated from patient and staff travel (38%–73% of per-case emissions). The procurement of single-use or disposable supplies (including their manufacturing and upstream logistics) made up the largest proportion of GHGs at four other sites (the UK, Hungary, Mexico 2 and New Zealand), ranging from 48% to 67% of per-case emissions at each site. The exception is Eswatini, where energy production made up 52% of their GHG emissions (more than commuting and procurement of supplies), though this may be due to this site operating only 2 days a week and the assumptions made within the Eyefficiency tool about OT downtime. At all sites, the relative impact of reusable supplies and waste treatment was minimal, with less than 9% and less than 3%, respectively.

Request permission

Qualitative feedback on Eyefficiency

After testing the tool, the beta sites provided feedback to the Eyefficiency team via an informal online survey. Overall, every site was pleased with the Eyefficiency tool. Some commented that the tool was ‘easy to use and good for monitoring our performance and time management’, and that ‘it was very helpful in guiding us to become more efficient and thinking of ways to make cataract surgeries greener.’ Each site also provided suggestions to improve the user experience of the app and website, which was incorporated into further development of the app in order to increase user engagement. In particular, we aimed to reduce the possibility of data loss through regularly saving studies to the cloud and allowing clinicians to resume recording from another device in the event of their device failing. The TAM recording software was also changed in order to automatically sync studies with the website, rather than requiring an additional manual process. These changes resulted in reduced user frustration. Further validation of data inputs was also introduced, in order to make sure that answers to the Eyefficiency questionnaire were within valid ranges. This was in response to user errors that were noticed during manual validation of the forms by the research team, such as the use of multiple currencies.

Productivity, throughput or use of operating time

Operating time, cut-to-close, represented a relatively small proportion of case-to-case duration (only 25%–56% at single-bed sites), while some sites spent 20%–40% of case-to-case duration in turnover with no patient present in the room. More ‘productive’ surgical practices would logically seek to minimise the time spent in non-surgical activities such as turnover, as this maximises the use of ‘high-value’ components such as the surgeon, capital equipment and the use of the OR. While some sites are able to achieve faster turnover times, this is dependent on a variety of factors including regulatory requirements especially related to infection control activities, restocking supplies, staff training and scheduling.

Dual bed systems are thought to optimise turnover time by conducting preoperative and postoperative activities on the second bed simultaneously while the surgeon is still performing the surgery on the first bed. However, some dual-bed sites performed fewer surgeries an hour than single-bed sites. This may be the result of variation in data entry, as some sites may have monitored their top physicians while others may have monitored trainees or a mix of skills. It could also be indicative of other issues in the service. For example, if the surgery is delayed due to staffing shortages, missing supplies, equipment malfunction or tardy patients, a dual bed system will not improve throughput.

Average cost per case

Costs are one major factor limiting access to cataract surgery globally, and results show large variability in per-case costs at each site. This variation could be due to differences in supply selection and utilisation at each site. It could also be due to differences in procurement practices and local market capabilities that allow certain sites to optimise costs. In more developed countries, unused items and partially used items are thrown away with greater frequency.25 26 In addition, national and/or regional policies and approaches may encourage local manufacturing or economies of scale (ie, group purchasing organisations) to reduce spending.

Some variability may be the result of the methods used to analyse the data. In generating reports for individual sites, Eyefficiency maintains local currencies. However, for comparing across sites, Eyefficiency converts local currencies into GBP or £. Converting currencies certainly has limitations, especially in light of the global economic issues caused by the COVID-19 pandemic. Prices paid for supplies are also highly variable, affected by international trade agreements, global supply chains, availability of vendors, local manufacturing capability and availability, and contract negotiations. It is unclear how much of the variability between Eyefficiency sites is tied to resource utilisation (quantity of supplies used) or the economic situation of the test site.

Though the sample sizes are small, there were minimal differences in the price of supplies between phaco and MSICS approaches, which goes against published evidence of the cost efficiency of MSICS versus phaco.27–29 Eyefficiency asks users to input aggregated supply costs for an average case, rather than the individual costs of each case monitored, which could lead to the difference in previously published findings. That said, MSICS tended to be slightly less expensive at sites where both are conducted, mainly due to smaller reported supply costs.

Waste generation may be a better metric to compare the use of physical supplies, as it directly measures the physical consumption of supplies. For example, the New Zealand site has the highest expenditures on disposable supplies but its waste generation is relatively low, suggesting supply spending may be disproportionately high. Conversely, though the UK site ranks in the middle on its supply spending, its waste generation per case is nearly double that of the next highest site.

Environmental footprint

Results show limited variability in GHGs between study locations and between phaco and MSICS, which is likely due to the methods used to calculate GHGs. For example, supply prices are used to calculate GHGs from supply manufacture and these models are not as accurate as country-specific input–output models or process-based LCA methods.23 The energy calculations are made by allocating the hospital’s total electricity use to the floor area and case duration of the surgery, rather than some metering in greater detail. In addition, energy usage does not include non-electric heating sources, meaning GHGs in colder climates will likely be underestimated.

Eyefficiency’s GHG estimates for phaco are also different for the two beta sites previously studied, as covered in the Eyefficiency methods paper.23 Accuracy between GHG models has always been an issue in LCA methodology, which is why most standards suggest minimising this variability as much as possible.30 The accuracy of Eyefficiency’s GHG estimates is purposely limited, in order to make data collection reasonable for most clinical sites. Despite this limitation in the accuracy of GHG calculations, Eyefficiency enables a consistent methodology, applied to all sites. While Eyefficiency’s GHG calculations will not be useful for micro-level improvements at each site (for example, choosing one surgical supply item over another), understanding the relative GHG emissions between sites will help identify larger opportunities for improvements at each site and more broadly, across regions.

User impressions and improvements to the tool

Based on qualitative feedback received from the sites, Eyefficiency appears to be a useful auditing tool that brings awareness to the resources needed to provide cataract surgery and the processes employed. As a beta test, there are several future improvements the team would like to make to Eyefficiency, as more users use the tool. One important issue is to re-evaluate the importance of individual user inputs (questions) in generating the Eyefficiency report. The more user inputs requested, the more likely we are to lose potential sites. However, we do not want to further limit the accuracy or usefulness of the reports. The Eyefficiency team would also like to build statistical methods into the tool’s backend to automatically analyse results and compare across site, while keeping sites anonymous. This would expand individual site’s reports by comparing with other similar facilities, while also enabling larger reporting for regional best practices or policy development.

Patient and public involvement

It was decided not to involve patients or the public in the design, conduct, reporting or dissemination plans of our research because the research is focused on collecting quantitative data about existing clinical workflow, which largely did not require patient or public input. Only a small subset of the modelling data was the result of activity on the part of the patient (in particular, their travel distance and modality); however, collecting this data (rather than having the user input an estimate for their average patient) would disrupt established patient–clinician workflow and provide an excessively detailed segment of data collection relative to other data inputs. Future Eyefficiency research will likely require substantial public and patient input, particularly as increased datasets elucidate potential ‘best practices’. Patient and public input will be required to determine how they feel about initiatives that improve: (a) productivity, for example, twin-bed ORs that might impact privacy, patient opinions about inefficient or even wasteful units; (b) costs, anything that reduces cost while maintaining or improving quality is a widely recognised factor in improving access to prevention of blindness surgery; and (c) carbon footprint and environmental impacts. Patients are important drivers to redesign towards more sustainable healthcare.