Discussion
An estimated 30 million US adults and 10 million UK adults cannot read beyond a 3rd grade level34 35; almost all PROMs assessed in this study (using any formula) had a readability level beyond this. This is noteworthy. The study’s results suggest that most PROM questionnaires commonly used in ophthalmology require a reading comprehension level better than that recommended by the AMA and NIH for patient material. Moreover, when assessed solely using the FORCAST measure, which is the measure recommended for assessing questionnaire and survey text, all PROMs included in this study fell outside recommended readability levels.
These results support findings from literature in other healthcare disciplines. For example, using the FORCAST measure, PROMs used in oral disease23 and audiology24 were also consistently found to exceed the recommended 6th grade reading level. Another study (although using a different readability formula) reported the average reading level of 59 PROMs used in orthopaedics to equate to that of 16–18 years old. Beyond the readability scores, several PROMs were also found to be markedly lengthy. Eighteen per cent of PROMs were longer than 1000 words. For context, that is almost half the length of this manuscript. The brevity of PROMs has been highlighted in previous literature as a priority to both ophthalmology patients and clinicians; qualitative findings on important factors in questionnaire design from one study36 includes quotations like ‘must be short, practical and useful’, ‘you need something that’s basic and easy to fill in’ and ‘keeping it obviously as brief as possible’.
As the use of PROMs becomes more widespread, it is crucial that their content is accessible and understandable to the majority of their target population. Missing data in PROMs has been reported as a major problem,37 and it is certainly possible that readability may be a contributing factor to lack of motivation to complete PROMs.38 Prospective confirmation of this with empirical data would be helpful. Indeed, population data has shown that people with lower literacy levels are less likely to participate in volunteer activities and more likely to report poor health than those with high literacy levels.39 The NEI-VFQ 25 is currently the most commonly used PROM in ophthalmology and had a reading grade level of 10th grade using the FORCAST measure, and 9th grade using both Flesch-Kincaid and Gunning Fog. This is equivalent to the readability of Moby Dick by Herman Melville.
The results of this study have several implications for future practice and research. Importantly, PROMs that have already gone through extensive development and validation processes should not simply be discarded. However, when choosing from existing PROMs and when designing new PROMs, substantial attention should be paid to the complexity of language used, particularly with respect to word length. It is worth noting that current recommended practice for PROM design includes input from participants in the form of qualitative investigation, from which items are derived using participants’ own language. Therefore, the vocabulary used in PROMs should ideally align with that of a sample of the target population. Yet, the representation of individuals with lower literacy levels in these samples remains an inherent problem. When designing new PROMs, a number of steps can be taken to improve readability. Patients should be involved at each stage of the PROM development process and outreach exercises should be undertaken to ensure that these individuals represent a broad sample of the target population. It is advisable to avoid using technical language, use short sentences, write questions in a conversational style and use words and language consistently.24 40 41 Resources such as the Center for Disease Control and Prevention’s Plain Language Thesaurus for Health Communications,42 and the Living Word Vocabulary43 may be useful for assessing the readability level of certain words, and finding replacements where appropriate. This should be applied to items, response options and instructions. Where use of technical jargon is unavoidable, one may provide a simple glossary of terms used.
The methodology was a key strength of this study. The selection of PROMs was done systematically. A range of validated readability measures relevant to the study’s aims were used, each of which having been well-described in the literature, and used in readability studies of PROMs in different disciplines. In addition, this is the first study of its kind in ophthalmology, and highlights an important limitation and factor to considering when using PROMs. Results are limited by the fact that the analysis was restricted to PROMs that had been used in AMD, glaucoma and/or diabetic retinopathy. These conditions were chosen because they are three of the leading causes of blindness both in the UK and worldwide.27 However, future work ought to systematically review readability of PROMs used across ophthalmic conditions and should then concentrate on ensuring PROMs across ophthalmology are at appropriate readability level for their targeted respondents. Furthermore, results are discussed in the context of population norms and general public health guidance but there is no data available on the literacy levels of specific patient populations.
While careful consideration was given to choosing appropriate readability measures to the study’s aims, each of these formulae come with their own set of limitations. No readability measure is a perfect measure of comprehension.23 In addition, other factors, such as formatting, font and font size used, and method of administration, may all impact the final comprehensibility of a PROM. This is a particularly pertinent consideration for PROMs that may be used among a visually impaired population where reading a PROM in its traditional format may not be possible. Readability scores cannot be applied to situations where a PROM is read out loud to the participant; the ‘listenability’ of a piece of text does not equate to its readability, and listening skills have been recognised as distinct from reading skills.44–46 PROMs may be subject to other weaknesses beyond the scope of this study, such as those relating to their psychometric properties, or difficulties establishing unidimensionality. Finally, the AMA and NIH guidelines used as a benchmark reference are based on US literacy levels and may not be appropriate guidelines for literacy levels in other English-speaking countries. While there are no specific standards in the UK for written health materials, the government recommends that public facing written material should not exceed the reading level of a 9 years old (3rd grade).47 If this standard were applied to the PROMs identified in this study, 93%–100% of PROMs (depending on readability measure used) would fall outside the threshold.
To summarise, most PROM questionnaires and instruments used in three common eye conditions require a literacy level better than that recommended by the AMA and NIH for patient material. It is likely that a majority of PROMs use language at a level too advanced for most patients to read easily. Greater care is required in choosing and designing PROMs appropriate for the literacy level of a population.