Quality of refractive error care in Pakistan: an unannounced standardised patient study

Introduction

In 2020, it was estimated that over 16 million people in Pakistan experienced vision loss at distance, while more than 10 million faced vision loss at near.1 Undercorrected refractive error is the leading cause of moderate to severe vision impairment globally,2 and its prevalence is expected to rise due to the growing and ageing population, as well as limited investments in human resources for eye health.3

Universal health coverage (UHC) aims to ensure that all individuals can access health services they require without experiencing financial hardship, with quality-of-care being a crucial component within UHC.4 Effective refractive error care (eREC) is one of two global targets endorsed by Member States at the 74th World Health Assembly to measure progress towards achieving UHC.5 This target, which entails a 40 percentage point increase in eREC, is designed to monitor both the effective coverage and quality of refractive error care (Q.REC). Recent estimates have reported that substantial improvements in the quantity and quality of refractive services will be required to meet the global target.6 7

Q.REC in Pakistan has previously been identified as an area for improvement. A scoping review identified seven inter-related research themes, one of which included the quality of eye care services.8 Although most of the studies within this theme focused on cataract surgery rather than refractive error care, the authors recommended further evaluation of current eye care service quality, exploration of patients’ perspectives and investigation of methods for improving services.8

Optometry is a relatively new field in Pakistan, and advancements have been made in the availability of optical services over the last decade.8 9 However, it is speculated that a limited number of optical services employ qualified staff to appropriately prescribe and dispense spectacles. The lack of national or provincial regulations mandating qualifications in refraction and spectacle dispensing may also contribute to poor quality spectacles in Pakistan.10 Punjab province has previously been identified as an area with unmet need for spectacles, with an estimated age-sex standardised prevalence of 9.1%, corresponding to over 2 million adults.11 Earlier estimates in Pakistan reported 25.8% of spectacle wearers had an ‘incorrect prescription’, defined as an improvement in presenting visual acuity (VA) by one or more visual impairment categories according to the WHO’s ICD-10 vision impairment classifications, with correction.11 This definition is an imprecise measure, likely to underestimate service provision errors and lacks detail about where errors occur.

Simulated patients, also known as unannounced standardised patients (USPs) are considered the gold standard for assessing quality in clinical practice,12 USPs are ‘actors’ trained to covertly pose as patients in a standardised manner while observing clinical techniques and services provided. When implemented effectively, USPs minimise observation bias, as care providers are less likely to modify their behaviours when unaware that they are being observed or examined. USPs have been employed in low-income and middle-income settings to evaluate family planning, pharmaceutical dispensing and clinical prescribing patterns.13 Studies using USPs have also assessed refractive error outcomes.14–16

A Q.REC study shows that using USPs can provide evidence on the quality of refractive services being delivered and inform decision-making, as well as recommend practice and policy changes. The primary aim of this study was to evaluate the Q.REC in Punjab, Pakistan, by employing USPs to estimate the proportion of prescribed and dispensed spectacles that are optimally prescribed. Additionally, this study aims to assess the associations between spectacle quality and optical service and USP characteristics.

Materials and methods

The protocol for this cross-sectional multisite study has been published previously.17 Briefly, we recruited 12 USPs in total, four each from Jhang, Khanewal and Sahiwal districts in Punjab province, Pakistan. Eligible USPs were aged 18 and above, fluent Urdu speakers, had not had formal refraction training, no ocular or health conditions that could affect refractive error, no previous refractive surgery, no manifest or intermittent strabismus, no amblyopia and at least one refractive profile, defined as:

• Myopia : spherical equivalent<−0.50 DS in at least one eye.

• Hyperopia: spherical equivalent > +0.50 DS in at least one eye.

• Astigmatism: > 0.50 DC in at least one eye.

• Presbyopia: ≥ 1.00 DS above the best optical distance correction.

District hospital optometrists compiled the sample frames of the eligible optical services. A minimum sample size of 64 services would achieve a maximum 7% margin of error around the estimated proportion of 50% of spectacles meeting optimal quality. Eligible optical services provided refraction and dispensing services and USPs were unknown to staff. The chosen districts were expected to be representative of other agricultural districts in Punjab but may differ from metropolitan areas such as Lahore district. The initial recruitment approach was opt out,17; however, verbal explanations were identified as more culturally appropriate. Hence, an alternative opt-in approach was additionally used.

Each USP had their baseline prescription, pupillary distances and best-corrected distance and near visual acuities measured by three qualified local optometrists, reviewed for variability, and if within our tolerance limits (0.75 D), then averaged for each measurement. USPs received standardised training through an online course and a 5-day in-person training event. Their skills on providing standardised responses, observing refraction and dispensing techniques, data recording and ability to minimise being detected were assessed by the local optometrists before data collection commenced.17

USPs were trained to visit optical services in a standardised way to request a refraction, record optical service characteristics, order and collect spectacles, and request a written prescription. They also observed if qualifications/certifications were displayed and assessed communication. Research optometrists assessed VA and spectacle comfort. This study is reported according to the Checklist for Reporting Research Using Simulated Patient Methodology.18

Results

Optical service characteristics

A total of 98 optical services were identified across the 3 districts. Out of these, 69 optical services were selected and a total of 276 visits were attempted. In each district, USPs attempted to visit all selected optical services in that district. In Jhang, staff were unavailable to perform refractions at 30 visits (30.0%) from 16 optical services (64.0%), and unable to dispense spectacles at 5 visits (5.0%) from 2 optical services (8.0%). Spectacles were dispensed following all visits in Khanewal and Sahiwal (see figure 1). No USP reported being identified as a USP during their visits.

Request permission

Figure 1

Flow chart of unannounced standardised patient (USP) visits to optical services and dispensed spectacles.

The majority of staff providing refractive error care were male (98.8%). Eye care provider qualification or registration certificates were observed by USPs at 19 of the 69 services (5 in Jhang, 14 in Sahiwal) and 15.4% (37/241) of visits. In 28.6% of visits, no distance subjective refraction was conducted. The proportion of services that used each testing procedure differed between the districts (p≤0.046, online supplemental material 2), except for checking pinhole distance VA (Jhang: 7.7% vs Khanewal: 2.0% vs Sahiwal: 1.3%, p=0.10) and near VA prior to refraction (Jhang: 36.9% vs Khanewal: 41.0% vs Sahiwal: 47.4%, p=0.45). Although USPs perceived communication to be clear for the majority of visits (>85% for each during eye test, for outcomes and spectacles recommendations), poorer communication was more commonly perceived at optical service visits in Jhang (online supplemental material 2).

Discussion

Refractive error quality relies on the provider’s accuracy in refraction and identifying patient needs, and the service’s ability to produce spectacles accordingly. In this Q.REC Pakistan study, we found that 42.7% of prescribed and dispensed spectacles were optimal, with quality ranging across districts. The main issue with suboptimal spectacles was horizontal prism. Hyperopes were less likely to receive optimal spectacles, while focimetry on previous spectacles and clear communication increased the odds of receiving optimal spectacles.

Suboptimal spectacles can impact patient dissatisfaction, trust and community eye care seeking behaviours. We found that 50% of suboptimal spectacles had horizontal prism values outside the acceptable tolerances. Meanwhile, over 70% of written prescriptions were within tolerance limits, suggesting dispensing appears to be a key issue in the quality of care. Similar errors involving induced prismatic effect were reported in Central Anatolia, Turkey, where pupillary distance measurements were frequently omitted by services.21 In this study, USPs observed pupillary measurements performed in 14.5% of visits (online supplemental material 2). However, pupil distances were recorded on 27% of the written prescriptions, suggesting that autorefractors may have been used to obtain some measurements. Limited evidence exists on autorefractor accuracy for measuring pupillary distance, and it would be difficult for USPs to determine whether these devices were used. Suboptimal spectacles were associated with manual pupillary distance measurement, indicating potential need for additional training.

The 2007 Pakistan National Prevention of Blindness Plan for building human resources in eye health provided limited emphasis on training qualified optical dispensing or mechanic personnel. Additionally, the plan contained few details regarding dispensing tasks for ophthalmic qualifications.22 The current curriculum offers a multiyear dispensing course, which is an important step in enhancing the skill of optical dispensing professionals in Pakistan.

Focimetry can be used by eye care providers to assist with making a clinical judgement on whether new spectacles should be recommended or whether the full difference in change should be prescribed. However, our findings suggest that patients who have never worn spectacles or those who do not bring their previous spectacles to the eye examination may be at higher risk of receiving suboptimal spectacles. Additionally, individuals with hyperopia, which has an estimated prevalence of 29.8% in Punjab adults,11 may also be at increased risk. The observed variability in testing procedures highlights the importance of providing comprehensive training to ensure consistent outcomes for all patients and refractive error types.

In a systematic review, the prevalence of spectacle non-tolerance among patients has been reported to be 2.1%.23 However, the included studies assumed patients who did not return with complaints were satisfied with their spectacles, potentially leading to an underestimation of the true prevalence of non-tolerance. Like our findings, the main factors contributing to non-tolerance were errors in refraction, communication and dispensing processes.23 Although our study included USP observations of discomfort rather than specifically assessing patient non-tolerance, the USP observations of discomfort suggest an increased risk of non-tolerance for long-term wear.

People centredness is a crucial aspect of quality of care.24 Effective communication, which encompasses providing clear instructions for accurate prescriptions, understanding wearers’ needs, and enabling patients to comprehend relevant information, is a vital competency in eye care practice.25 As demonstrated in this study, effective communication throughout the eye examination process, including discussing outcomes and recommendations, is associated with achieving optimal spectacles.

The College of Ophthalmology and Allied Vision Sciences is one of the main education facilities that provide ophthalmic courses for the Punjab region, and learning modules on communication skills in general practice, inclusive health, and medical consent have been included in the optometry curriculum. However, it is unknown whether the staff at the optical services visited in this study have had access to the optometry training course or are aware of the importance of communication in their practice. Out of the 69 stores observed, only 19 displayed registration or qualification certificates, suggesting the majority of services might lack qualified or appropriately trained staff to provide optimal care. The true extent of this may be underestimated, as the hidden nature of USP observations limits their ability to assess staff qualifications, and there is no requirement to publicly present such qualifications. Therefore, a more comprehensive understanding of human resourcing in private optical services is required.

In 2007, it was reported that individuals in Pakistan often sought refractive services from local marketplace opticians, who often operated family businesses without formal training.22 Consequently, the National Committee for the Prevention of Blindness emphasised the need for strengthened ophthalmic personnel at the district and tertiary eye care levels. Since then, there has been limited information on the growth of eye care personnel over time. However, persistent challenges in personnel appear to be present in Jhang district. Services in Jhang provided a significantly lower proportion of optimal quality spectacles compared with the other districts, indicating a need for considerable support to enhance refractive error care in the region. Additionally, the considerable variation in testing procedures across districts underscores the need for a standardised curriculum for all courses. Such a curriculum is currently being developed by the National Committee.

Naturally, this study has limitations. First, there is no established benchmark defining the ideal, or minimally acceptable proportion of optimal spectacles dispensed from optical services. Nevertheless, the characteristics identified in this study associated with optimal spectacles provide evidence and opportunities to enhance clinical care. Second, the Q.REC for children cannot be inferred, as refractive error care in children can be more challenging with clinicians having varying prescribing philosophies for hyperopia,26 27 and myopia management is still evolving.28 29 Third, although 21 pairs of bifocal spectacles were dispensed, the quality of prescribing bifocal spectacles (and progressive addition lenses) cannot be confidently assessed, as our protocol is designed for evaluating single vision lenses, and does not currently take segment height measurements into account. Fourth, the USPs recruited did not have high refractive errors limiting our understanding on the full scope of refractive error care. However, the more challenging nature of accurately refracting patients with higher refractive errors would likely decrease the percentage of optimally prescribed spectacles. In future, USPs recruited with high refractive errors would likely require back vertex distance measurements to be included in quality assessment. Fifth, spectacle frame fit could further impact comfort and long-term wear, although the focus of this study was optimal lens quality and accuracy. Hence, future studies could further explore the influence of comfort and fit on spectacle quality.

This study highlights the need for enhancing the Q.REC in Punjab, Pakistan. It also provides outlines for specific opportunities to improve the Q.REC, which may contribute to an increase in eREC targets in Pakistan. Areas of clinical improvement and regulatory changes include upskilling the workforce, improved dispensing, improved refraction skills for hyperopia, less reliance on previous spectacles, greater emphasis on effective communication skills and additional support for Jhang region.