MATERIALS AND METHODS

Study design

A model based cost utility analysis was conducted in which QALYs were used as the measure of effects. QALYs are calculated by weighting the duration of health states by a preference based score of health related quality of life (HRQL) (utility), measured on a scale from 0 (dead) to 1 (perfect health).⁴

In the model, treatment of amblyopia started at age 3 (strategy “treatment”) was compared to the strategy “no treatment.” The main outcome measure was the incremental cost effectiveness ratio (ICER)—that is, the ratio of the differences in mean costs C¯ and mean effects Ē between the strategies “treatment” and “no treatment”:

For comparison, an incremental approach was chosen—that is, only the differences in costs and QALYs between the two strategies were considered. When calculating QALYs, only health states were taken into account with utility losses set off by unilateral visual impairment caused by amblyopia or by visual impairment caused by any other unilateral eye disease coming along later in life. In the latter case, this would cause bilateral visual impairment in a proportion of patients already affected by amblyopia; in all other people, this would result in unilateral visual impairment. Visual impairment caused by any bilateral eye disease was not considered in the model because this would cause bilateral visual impairment in both patients with amblyopia as well as those without, and would hence not set off differences in QALYs.

For the definition of health states used in the model, a threshold for the presence of visual impairment was set at a corrected visual acuity (VA) of <0.5 (20/40), in accordance with various therapeutic,^5,6 epidemiological,^7,8 as well as disability studies.⁹ Monocular visual impairment was defined as VA <0.5 in the worse eye and ⩾0.5 in the better eye, bilateral visual impairment as VA <0.5 in both eyes.

Decision analytic model

A decision tree was combined with a Markov model.¹⁰ Analysis was performed using the DATA software package (Version 4.0, TreeAge Software, Inc, Williamstown, MA, USA). The decision tree (fig 1) had one decision node which distinguished the strategies “treatment” and “no treatment.” In the strategy “treatment,” VA ⩾0.5 in the amblyopic eye was achieved by a probability equal to the treatment success rate (p_eff), resulting in the health state labelled “healthy.” If treatment was not successful (VA <0.5), this would result in the health state “unilateral visual impairment caused by amblyopia,” as would the strategy “no treatment.”

These two different health states were initial health states of two different Markov processes. In a Markov process, the course of a disease is divided into distinct states and transition probabilities are assigned for movement between these over a discrete time period called the Markov cycle.¹⁰ By attaching estimates of resource use and health effects to the states and transitions in the model, and then running the model over a large number of cycles, long term costs and effects can be estimated. One Markov process started with the initial health state “healthy.” Transition to the health state “unilateral visual impairment (no amblyopia)” took place when a unilateral non-amblyopic eye disease came along and caused unilateral visual impairment. The other Markov process started with the initial health state “unilateral visual impairment caused by amblyopia.” Transition to the health state “bilateral visual impairment (with amblyopia)” took place when the non-amblyopic eye was affected by an unilateral eye disease. In both Markov processes, the state “death” could be reached from all other health states.

The cycle length was set at 1 year. Simulation started in cycle 0 which corresponds to the 4th year of life and was completed in cycle 86 (90th year of life). To account for sex differences in mortality and incidence of visual impairment, all Markov processes were made sex specific.

Model parameters

For each model parameter, a (mean) value was defined, which was used for the base analysis, as well as a possible value range, according to which parameter values were varied in one way sensitivity analysis (table 1). Furthermore, for parameters which could, in principle, be sampled, distributions were specified for probabilistic sensitivity analysis (Monte-Carlo simulation¹¹) as follows:

For proportions, which are bound to a 0 − 1 interval, a β distribution was fitted.^11,12 Its parameters α und β were derived from estimates of the proportion’s mean μ and standard error σ using the method of moments estimation^11,13 (see appendix A).

Student’s t distributions were specified for the mean of continuous parameter values from small, approximately normally distributed, samples. If there was only information on the range, but not on the mean of a parameter, then a uniform distribution was used.

Success rate of treatment

In studies that have used VA ⩾0.5 as the threshold for treatment success, reported success rates range from of 50% to 100%.^{2,5,6,14–17} In a meta-analysis based on 23 studies with 689 patients included, Flynn et al⁵ found an overall success rate of 74.3%. In the model, the mean treatment success rate (p_eff) was set at 75% with the range 60%–90% used for univariate sensitivity analysis. This range was considered to represent a 95% confidence interval (95% CI), and the standard error s was estimated by the equation

where u and l are the upper and lower limits of the range, respectively.¹¹ For the Monte-Carlo simulation, a β distribution for (p_eff) was fitted based on estimates of the mean and the standard error as described above.

Utilities

The utility of the health state “bilateral visual impairment” was derived from a recent study¹⁸ in which an equation for converting VA of the better eye to a mean utility value (U) was derived via regression analysis:

For the incremental analysis, only the reduction of utility caused by visual impairment was relevant. Therefore, the utility of VA = 1.0 was set at U = 1.00—that is, it was increased by 0.11 compared to the utility predicted by equation (3). For a conservative estimate of the utility of the health state “bilateral visual impairment,” the utility of VA = 0.4 was calculated using equation (3) and then also increased by 0.11, resulting in U = 0.78. For health states without visual impairment, the utility was constantly set at U = 1.00. Thus, compared to no visual impairment, utility of bilateral visual impairment was reduced by 0.22. Based on utilities reported by the same working group,¹⁹ this reduction of utility (“disutility”) was varied by plus or minus ⅓—that is, from 0.15 to 0.29 for univariate sensitivity analysis, corresponding to utilities from 0.71 to 0.85. This interval was considered to reflect the 95% CI. The standard error was estimated using equation (2) and a β distribution was fitted for the Monte-Carlo simulation.

The utility associated with unilateral visual impairment caused by amblyopia has not specifically been investigated so far. However, in another recent study of the same working group, unilateral impairment caused by various diseases was found to cause a mean disutility of 0.08.²⁰ As individuals with only one sound eye, as a result of amblyopia since childhood, may develop compensatory visual mechanisms,¹ the reported disutility was considered a maximum. It was thus varied from 0.00 to 0.08 in univariate sensitivity analysis which corresponds to utilities ranging from 0.92 to 1.00. For the base analysis, a utility of 0.96 was used—that is, the middle of the interval. For the Monte-Carlo simulation, a uniform distribution on the interval [0.92;1.00] was used.

Transition probabilities of Markov model

Incidence of unilateral visual impairment other than amblyopia

Data on age and sex specific incidence of unilateral visual impairment were not available in the literature. Thus incidence had to be derived from prevalence studies. As such studies were not available for Germany, data from the Australian Blue Mountain Eye Study (BMES)^7,21 were used, assuming Australian population and healthcare characteristics comparable to those in Germany. To estimate the prevalence of unilateral visual impairment not caused by amblyopia, age and sex specific prevalence data of unilateral visual impairment⁷ were reduced by prevalence data of amblyopia.²¹ Age and sex specific transition probabilities (incidence) were derived using methods described by Miller et al²² (see appendix B) and labelled tp_mono_m for males and tp_mono_f for females.

Incidence of bilateral visual impairment in amblyopes

It was assumed that the prevalence of an eye being affected by unilateral, visually impairing eye disease other than amblyopia is the same for amblyopic and non-amblyopic eyes. Thus, in people with unilateral amblyopia, the prevalence of the non-amblyopic eye being affected, potentially causing bilateral visual impairment, was estimated to be half of the prevalence of unilateral visual impairment in people without amblyopia. Yet, there is evidence that VA in the amblyopic eye may improve when the other eye becomes visually impaired: In a recent study, some increase in VA in the amblyopic eye after 1 year was reported in 19%²³; in another study, 34 (23.6%) of 144 affected patients improved to VA ⩾0.5 in the amblyopic eye.²⁴ To obtain a conservative estimate of cost effectiveness, the estimated prevalence was reduced by 23.6% to obtain the prevalence of bilateral visual impairment. Based on these estimated prevalence data, age group and sex specific 1 year probabilities of transition from the health state “unilateral visual impairment caused by amblyopia” to the health state “bilateral visual impairment (with amblyopia)” were derived as described above and labelled tp_vis_m for males and tp_vis_f for females.

In univariate sensitivity analysis, the transition probabilities were jointly reduced and increased by the factor 0.5 and 1.5, respectively. For the Monte-Carlo simulation, a uniform distribution on the interval [0.5; 1.5] was used for this factor.

Mortality

Age and sex specific death rates were obtained from the most recent life table 1997/1999 of the German Federal Statistical Office.²⁵

Treatment costs

To estimate treatment costs, 12 experts of amblyopia treatment from different German treatment centres filled in a standardised questionnaire in which medical services and items (for example, glasses, patches, etc) possibly used for amblyopia treatment were listed.²⁶ The treatment of strabismus associated with amblyopia was not included. Experts were asked to estimate the mean number of services and items used per year during up to 9 years of treatment (follow up). Services and items were monetarily valued in euro (€) using “administrative” prices paid by the German statutory health insurance in the year 2002 (€1  =  0.99 US$ as of 1 July 2002). Thus, a third party payer perspective was applied, assuming no co-payments by the patients.

From these figures, mean estimated costs m* were calculated (and used for the base analysis) as well as the standard error s which reflects the uncertainty of the experts with regard to average costs (table 2).¹¹ In univariate sensitivity analysis, the treatment costs were varied according to their 95% CIs. For the Monte-Carlo simulation, treatment costs m were calculated using the equation²⁷

where t follows a standardised Student distribution with 11 degrees of freedom.

Discounting

In order to make costs and effects occurring at different times comparable, both were discounted at 3% as recommended by the panel on cost effectiveness in health an medicine.²⁸ In sensitivity analysis discount rates of 0% and 5% were also used. Discounting is a method of calculation which converts the value of futures costs and effects into their present value.²⁸

Sensitivity analysis

Univariate sensitivity analysis was performed by varying single parameter values according to the ranges described above and repeating the analytical solution of the model. To analyse the effect of uncertainty in all model parameters simultaneously, probabilistic sensitivity analysis (Monte-Carlo simulation) was performed. In Monte-Carlo simulations, values for all model parameters were randomly sampled from their respective specified distributions and the analytical solution of the model was repeated.¹¹ This process of resampling and recalculating the incremental costs and effects from the model was repeated 10 000 times to generate a distribution of the estimated ICER. Uncertainty intervals were estimated from the simulated data by taking the 2.5 and 97.5 percentile values to represent the end points for a 95% interval,¹¹ and cost effectiveness acceptability curves^29–31 were constructed.

RESULTS

Base result

The strategy “treatment” was associated with incremental costs of €2083 and incremental effects of 0.88 QALYs per amblyopic child. The ICER was €2369/QALY.

Univariate sensitivity analysis

Uncertainty with respect to the utility of unilateral visual impairment had the strongest potential impact on the ICER (fig 2). For a utility of 0.92, the ICER decreased to €1192/QALY; for a utility of 0.99, the ICER increased to €9148/QALY; if it was 1.00—that is, if utility was affected only by bilateral visual impairment, the ICER increased to €196 939/QALY. Figure 3 shows the association between the ICER and the utility of unilateral impairment for the utility range of 0.92 to 1.00.

Besides, uncertainty with respect to treatment costs had a marked, but much smaller, impact on the ICER. All other parameters had only little impact. The uncertainty regarding the incidence of visual impairment caused by other eye disease and the utility of bilateral visual impairment had almost no impact.

If costs and effects were not discounted, the ICER was €930/QALY. If the discount rate was 5%, the ICER was €3600/QALY.

Monte-Carlo simulation

Uncertainty intervals

Figure 4 shows the joint distribution of incremental costs and effects generated in 10 000 Monte-Carlo simulations on the cost effectiveness plane. The 95% uncertainty interval of the ICER was €710/QALY to €38 696/QALY which corresponds to the slope of the lines representing the interval end points in figure 4. Without discounting, the 95% uncertainty interval was €303/QALY to €11 172/QALY.

Cost effectiveness acceptability curves

When costs and effects were discounted at 3%, Monte-Carlo simulation yielded an ICER <€10 000/QALY in 88% and an ICER <€20 000/QALY in 95%. Without discounting, an ICER <10 000 was yielded in 97%, and an ICER <20 000 in 99%. This is displayed by the cost effectiveness acceptability curves in figure 5 which give the proportion of observed simulation results lying below varying threshold values for the ICER.

Because much of the variability of the results was due to the uncertainty with respect to the utility of unilateral visual impairment, Monte-Carlo simulations were also performed, keeping this utility constant at different levels and varying all other parameters according to their distributions. If this utility was 0.92, 0.94, 0.96, 0.98, or 0.99, in 95% of the simulations the ICER was below €1865, €2492, €3742, €7393, and €14 488, respectively.

DISCUSSION

The ICER of treatment for amblyopia starting at age 3 was found to be more favourable than that of many other health care interventions: In a recent survey of 228 published cost utility studies, which included cost utility ratios for 647 interventions, the median cost per QALY was found to be US$12 000.³²

If only bilateral, but not unilateral, visual impairment was associated with a loss in utility, then treatment would very likely not be cost effective. By contrast, if unilateral visual impairment was associated with a disutility of only 0.01, then the ICER fell below €10 000/QALY. ICERs obtained in Monte-Carlo simulation were mostly within limits likely to be acceptable to decision makers; if, for example, a decision maker wanted to spend €20 000 per QALY at maximum, he could be approximately 95% confident that QALYs generated by amblyopia treatment would not cost more than that. Many routinely used healthcare interventions have an ICER of €20 000/QALY or more.³³ Since the analysis included effects occurring during the remaining lifetime of up to 86 years, discounting had substantial impact on the ICER.

The results of this study were similar to those of a recent cost utility model of therapy for amblyopia by Membreno et al³⁴ which reported an ICER of US$1726/QALY when only costs of non-surgical amblyopia therapy were considered and a 3% discount rate was used, like in the study presented here. However, using a different approach, they assumed the mean disutility caused by amblyopia to be 0.03, whereas the present study used 0.04. On the other hand, mean costs of non-surgical amblyopia treatment estimated by Membreno based on clinical guidelines from a US third party payer perspective were lower ($1452). In the study presented here, costs were estimated based on expert opinion which was considered the best available evidence for resource use in current ophthalmological practice. If in the model presented here mean treatment costs of $1452 and a disutility of amblyopia of 0.03 were used, the ICER would stay almost the same at $2193/QALY.

While the study by Membreno et al was based on a deterministic decision analysis model, the study presented here used a probabilistic Markov state transition model. Whereas Membreno et al distinguished more different health states, in the Markov model presented here the number of health states was kept to a minimum, to enhance probabilistic sensitivity analysis.

For modelling, the course of disease had to be divided into distinct states and, hence, thresholds for the presence or absence of visual impairment had to be defined. Since in reality no such thresholds exist, this necessarily entails some simplification. However, this was taken into account by varying model parameters widely in sensitivity analyses. Besides, there seems to be considerable consensus that VA <0.5 (20/40), as used in this study, is a threshold for a relevant visual deficit, both bilateral and unilateral.^5–9

The incidence of unilateral visual impairment not caused by amblyopia was estimated based on the BMES in which only adults aged ⩾49 years were included. Hence, visual impairment before age 49 was not considered. However, its prevalence in the youngest age group (49–54 years) of the BMES was rather low. If the prevalence in the age group 49–54 years resulted from a constant incidence from birth onward, the ICER would be reduced only slightly to €2367/QALY.

For valuation of HRQL, utilities were used which were elicited for visual impairment caused by various ocular diseases. In the underlying studies,^18,20 neither the cause of visual impairment nor age, sex, education, ethnicity, co-morbidity, or duration had significant impact on the utility. However, the utility loss caused by amblyopia starting early in life could be comparatively smaller because of adaptation. Therefore, in sensitivity analysis the respective utility loss was varied widely, even to no loss at all.

Much of the uncertainty in results comes from uncertainty regarding the effect of amblyopia on quality of life. In order to reduce this uncertainty, the most important issue is to investigate the impact of amblyopia on utility.

APPENDIX A

The mean μ of a β distribution is

and the standard error σ is

A point estimate for the mean of the proportion was inserted in equation (5), and an estimate for its standard error was inserted in equation (6). Equations (5) and (6) were then solved to obtain α und β.

APPENDIX B

It was assumed that the age group specific prevalence reported in the BMES was present in the middle of the age classes. Thus, new age classes were defined, the middle of the original age classes being the limits of the new age classes. Sex specific transition probabilities q_t,j for the new age classes of the width t years and upper limit j years were derived using the equation

where P_j and P_j−t was the prevalence at the upper and lower limit of the new age class, respectively. One year transition probabilities q_j were derived using the equation