Hypothetical versus real willingness to pay in the health care sector: results from a field experiment

https://doi.org/10.1016/S0167-6296(01)00075-3Get rights and content

Abstract

We conducted a field experiment comparing hypothetical and real purchase decisions for a pharmacist provided asthma management program among 172 subjects with asthma. Subjects received either a dichotomous choice contingent valuation question or were given the opportunity to actually enroll in the program. Three different prices were used: US$ 15, 40, and 80. In the hypothetical group, 38% of subjects said that they would purchase the good at the stated price, but only 12% of subjects in the real group purchased the good (p=0.000). We cannot, however, reject the null hypothesis that “definitely sure” hypothetical yes responses, as identified in a follow-up question, correspond to real yes responses. We conclude that the dichotomous choice contingent valuation method overestimates willingness to pay, but that it may be possible to correct for this overestimation by sorting out “definitely sure” yes responses.

Introduction

In the contingent valuation method, individuals are asked about their hypothetical willingness to pay for a defined good (Mitchell and Carson, 1989). In the most commonly used elicitation approach, the dichotomous (yes/no) choice approach, individuals accept or reject only one price and opportunity to purchase the defined good. By varying the price in different subsamples of respondents, it is possible to derive the aggregate demand curve (Bishop and Heberlein, 1979). The hypothetical dichotomous choice question is often framed in terms of a vote on a referendum (Mitchell and Carson, 1989). The contingent valuation method was developed in environmental economics and has been widely used to estimate the willingness to pay for environmental changes (Hanemann, 1994). The method is also increasingly used in the health care field to estimate the willingness to pay for health care programs as a basis for cost-benefit analyses of health care programs (Diener et al., 1998).1

Despite its widespread use, the contingent valuation method is highly controversial among economists (Diamond and Hausman, 1994, Hanemann, 1994, Portney, 1994). The nucleus of the controversy is the extent to which hypothetical choices in the contingent valuation method correspond to real economic choices. The extent to which hypothetical choices mimic real choices is an old controversy in economics. Wallis and Friedman (1942) criticized the use of hypothetical choices in experiments over 50 years ago, and this topic has been debated ever since (Kagel and Roth, 1995, Thaler, 1987).

The correspondence between hypothetical and real willingness to pay has been studied in a number of laboratory experiments. Generally, these experiments have focused on the dichotomous choice approach and have compared the fraction of hypothetical and real yes responses (i.e. acceptances to buy the good at the given price) between experimental groups. In most of these experiments, the proportion of hypothetical yes responses has significantly exceeded the proportion of real yes responses (Cummings et al., 1995, Cummings et al., 1997, Loomis et al., 1997, Johannesson et al., 1998, Blumenschein et al., 1998, Cummings and Taylor, 1999).

The results of these experiments suggest that the dichotomous choice contingent valuation approach is associated with a general overestimation problem, which seriously undermines the usefulness of the method. This overestimation problem, often referred to as hypothetical bias, has motivated research into various calibration approaches for removing the bias. Cummings and Taylor (1999) recently tested a so-called “cheap talk” design for removing hypothetical bias. This approach includes an explicit discussion of the hypothetical bias problem in the contingent valuation question, to induce respondents to eliminate hypothetical bias from their answers. In a set of laboratory experiments involving four different public goods there was no significant difference between the proportion of real yes responses and the proportion of hypothetical yes responses with a cheap talk design (Cummings and Taylor, 1999).

An alternative approach to remove hypothetical bias that has yielded promising results is to use the degree of certainty in the hypothetical yes responses to sort out “false” yes responses. In two experiments Johannesson et al. (1998) and Blumenschein et al. (1998) tested the hypothesis that “definitely sure” yes responses, identified in a follow-up question, correspond to real yes responses. In the Johannesson et al. (1998) experiment, the “definitely sure” yes responses significantly underestimated the real yes responses whereas in the Blumenschein et al. (1998) experiment, the null hypothesis of no difference between “definitely sure” yes responses and real yes responses could not be rejected. In both of these experiments, data were also collected about the degree of certainty of the hypothetical yes responses on a scale between 0 (very unsure) and 10 (very sure). Utilizing the within-sample data from these experiments (i.e. data where the subjects responded to a hypothetical valuation question followed by a real valuation question), Johannesson et al. (1999) estimated a calibration function based on the degree of the individual’s self-assessed certainty (on the 0–10 scale) and a variable representing the price level. This calibration function removed the hypothetical bias from both experiments (Johannesson et al., 1999).

A weakness of the experimental work carried out so far on hypothetical bias and different calibration approaches is that the work has been carried out under rather artificial conditions in the laboratory using mainly college students as subjects.2 Field” experiments, conducted in the ‘real world’ are a crucial next step in this area of research. In this paper, we report the results of a field experiment of hypothetical versus real willingness to pay for a health care good. The good valued in this experiment is a pharmacist provided asthma management program. We examine whether dichotomous choice contingent valuation questions lead to hypothetical bias for this good. In addition, we test if “definitely sure” hypothetical yes responses correspond to real yes responses and if calibrated hypothetical yes responses correspond to real yes responses (using the calibration function estimated by Johannesson et al. (1999)). We find that the hypothetical yes responses significantly overestimate the real yes responses (38 versus 12%), but that there is no significant difference between the “definitely sure” hypothetical yes responses and the real yes responses (14 versus 12%). Our results suggest that it may be possible to eliminate hypothetical bias in dichotomous choice contingent valuation studies of health care by adding a simple follow-up question about the certainty of the hypothetical response. The design of the experiment is described below followed by the results. The paper ends with a discussion section.

Section snippets

Experimental design

The experiment involved two different experimental groups referred to as the “hypothetical group” and the “real group”. Subjects in the hypothetical group received a dichotomous choice contingent valuation question about purchasing a pharmacist provided asthma management program, whereas subjects in the real group were given the opportunity to actually purchase the program. Three different prices were used: US$ 15, 40, and 80. A pilot study was carried out to determine the prices. The study had

Background characteristics

Table 1 provides the background characteristics for the two samples. On average, subjects were somewhat younger in the real group and the proportion of women was higher. The average income and education was also slightly higher in the real group. However, none of the differences in background characteristics between the groups were statistically significant, with the exception of the gender difference that was significant at the 10% level (p=0.080).

Discussion

It is becoming increasingly common to carry out contingent valuation studies within the health care sector. The ability of the method to correctly estimate the willingness to pay for a health care program is, however, unclear. We carried out a field experiment directly comparing responses to a dichotomous choice contingent valuation question with real purchase decisions. As noted in the introduction to this paper, several laboratory experiments have shown that dichotomous choice contingent

Acknowledgements

We thank Jonathan Gruber and an anonymous referee for helpful comments. This research was funded by a Drug Information Association Research Grant.

References (34)

  • B.L Carter et al.

    Evaluation of hypertensive patients after care provided by community pharmacists in a rural setting

    Pharmacotherapy

    (1997)
  • R.G Cummings et al.

    Unbiased value estimates for environmental goods: a cheap talk design for the contingent valuation method

    American Economic Review

    (1999)
  • R.G Cummings et al.

    Homegrown values and hypothetical surveys: is the dichotomous choice approach incentive-compatible?

    American Economic Review

    (1995)
  • R.G Cummings et al.

    Are hypothetical referenda incentive compatible?

    Journal of Political Economy

    (1997)
  • R.B D’Agostino et al.

    The appropriateness of some common procedures for testing the equality of two independent binomial populations

    American Statistician

    (1988)
  • P.A Diamond et al.

    Contingent valuation: is some number better than no number?

    Journal of Economic Perspectives

    (1994)
  • A Diener et al.

    Contingent valuation studies in the health care literature: a review and classification

    Health Economics

    (1998)
  • Cited by (155)

    • The demand for a COVID-19 vaccine

      2022, Economics and Human Biology
    • Economic valuation of setting up a social health enterprise in urban poor-resource setting in Kenya

      2020, Social Science and Medicine
      Citation Excerpt :

      Despite its wide applicability, researchers are still skeptical about the derived welfare estimates from the CVM. The main challenge with the CVM is that respondents might not be always truthful when eliciting their WTP (Aadland and Caplan, 2003; Blumenschein et al., 2001; Champ et al., 2009; Murphy et al., 2005a, 2005b). Hence, there could be a discrepancy between the hypothesized WTP and the real WTP, which is often referred to as hypothetical bias.

    • The criterion validity of willingness to pay methods: A systematic review and meta-analysis of the evidence

      2019, Social Science and Medicine
      Citation Excerpt :

      However, since this date, both data on this gold standard and its use in the health sector have developed substantially, with some authors arguing that “the potential for survey instruments to provide valid estimates of WTP has been proven” (Donaldson and Shackley, 2002). However, there remains great concern about whether hypothetical values provide correct estimates of actual WTP and the evidence appears to be mixed (Munro, 2009; Loomis et al. 1996, 1997; Blumenschein et al., 2001). With more recent studies comparing stated and actual values performed since the last review, meta-analyses of the summary values will hopefully show consistent results regarding the magnitude of hypothetical bias.

    View all citing articles on Scopus
    View full text