Article Text
PDF
Abstract
Objectives Patients are often provided with medicine information sheets (MIS). However, up to 60% of patients have low health literacy. The recommended readability level for health-related information is ≤grade 8. We sought to assess the readability of MIS given to patients by rheumatologists in Australia, the UK and Canada and to examine Australian patient comprehension of these documents.
Design Cross-sectional study.
Setting Community-based regional rheumatology practice.
Participants Random sample of patients attending the rheumatology practice.
Outcome measures Readability of MIS was assessed using readability formulae (Flesch Reading Ease formula, Simple Measure of Gobbledygook scale, FORCAST (named after the authors FORd, CAylor, STicht) and the Gunning Fog scale). Literal comprehension was assessed by asking patients to read various Australian MIS and immediately answer five simple multiple choice questions about the MIS.
Results The mean (±SD) grade level for the MIS from Australia, the UK and Canada was 11.6±0.1, 11.8±0.1 and 9.7±0.1 respectively. The Flesch Reading Ease score for the Australian (50.8±0.6) and UK (48.5±1.5) MIS classified the documents as ‘fairly difficult’ to ‘difficult’. The Canadian MIS (66.1±1.0) were classified as ‘standard’. The five questions assessing comprehension were correctly answered by 9/21 patients for the adalimumab MIS, 7/11 for the methotrexate MIS, 6/28 for the non-steroidal anti-inflammatory MIS, 10/11 for the prednisone MIS and 13/24 for the abatacept MIS.
Conclusions The readability of MIS used by rheumatologists in Australia, the UK and Canada exceeds grade 8 level. This may explain why patient literal comprehension of these documents may be poor. Simpler, shorter MIS with pictures and infographics may improve patient comprehension. This may lead to improved medication adherence and better health outcomes.
- health literacy
- rheumatology
- medication adherence
- patient comprehension
- readability
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Readability of medicine information sheets (MIS) from three countries (Australia, UK and Canada) was assessed.
While readability formulae only measure the number/complexity of words/sentences, Australian patient literal comprehension of MIS was also assessed.
The study population was from a regional community and may not be representative of a more urban population.
Introduction
Health literacy is defined as the ‘capacity to obtain, process and understand written and oral health information and services needed to make appropriate health decisions’.1 Low health literacy has been associated with poorer health-related knowledge, increased hospitalisations, reduced immunisations, poorer health status and higher mortality.2 Patients with poor health literacy are less likely to successfully manage chronic disease3 and have greater difficulty in following instructions for prescription medications.4 Higher health literacy has been associated with increased medication adherence.5 6
Although the importance of health literacy and patient–physician communication on health outcomes is well recognised, many patients have difficulty in understanding what their physicians tell them.7 Immediately after leaving a consultation with their specialist, patients were able to recall less than half the information just provided to them.8 9 The provision of written health information in addition to verbal information significantly increases patient knowledge and satisfaction.10 Written information may also lead to increased adherence with treatment.9 However, designing effective written health information remains challenging due to differences in patient literacy levels.
The recommended level of reading difficulty for health-related written material in inconsistent. Some agencies have recommended up to eighth grade level11—the average reading level of an adult in the USA,12 13 whereas others have suggested levels as low as fifth grade to be more inclusive of those with limited literacy.14 No national guidelines exist in Australia, although the South Australian government has recommended up to eighth grade level.15 Despite these inconsistencies, many studies have found written health information provided to patients often exceeds these levels.16–19 While there is greater access to health-related information on the internet, this often also exceeds recommended readability levels.20 21
Literacy levels in Australia are poor, with up to 60% of the population having low literacy skills22 23—defined as the ‘minimum required for individuals to meet the complex demands of everyday life’.24 The International Adult Literacy Survey found 57% of Canadians fall into the lowest two literacy categories.25 In the UK, just under one in six adults has the literacy of an 11-year-old.26 A study of over 200 rural and urban Australian rheumatology patients found that 15% of patients had low health literacy and up to one-third of patients incorrectly followed dosing instructions for common rheumatology drugs.23 Ten per cent of patients with rheumatoid arthritis (RA) who attended an urban community-based Australian rheumatology practice had inadequate/marginal functional health literacy or a reading age at or below the US high school grade equivalent of seventh–eighth grade.27 Up to 24% of rheumatology patients at a US medical centre had a reading level of eighth grade or less.28 In 2002, one in six rheumatology patients at a Scottish hospital were illiterate and struggled to understand education materials and prescription labels.29 These findings are concerning, as rheumatologists often use medications such as methotrexate (MTX) or expensive biological therapies with severe side effects, even death,30 if taken incorrectly.
Given the importance of health literacy and its relationship to health outcomes and medication adherence, we sought to assess: (i) the readability of patient medication information sheets (MIS) given to patients by Australian rheumatologists and (ii) patient comprehension of these documents.
We also compared the readability of the Australian MIS to similar documents given to rheumatology patients in the UK and Canada.
Methods
Assessment of readability
Text from the MIS of commonly prescribed rheumatology medications available on the Australian Rheumatology Association (ARA) website31 was imported into a Microsoft Word document and readability assessed using Readability Studio (Oleander Software).18 32–35
Non-essential text including logos, headers, footers, hyperlinks and contact information was deleted prior to analysis as these may have adversely affected readability scores. Readability was assessed using a range of measures such as the Flesch Reading Ease formula, Simple Measure of Gobbledygook (SMOG) scale, FORCAST (named after the authors FORd, CAylor, STicht) and the Gunning Fog scale. The Flesch Reading Ease formula calculates an index score of a document based on sentence length and number of syllables. It is often used for school textbooks and technical manuals. The standard score is between 0 and 100, with a high score indicating the document is easier to read36 (however, it is possible to also gain minus scores and scores over 100). The SMOG formula calculates grade level and reader age based on complex word density and assigns a grade level (fourth grade to college level).33 37 It is particularly useful for secondary age readers and attempts to predict 100% comprehension, whereas most other formulae predict 50%–75% comprehension. Consequently, SMOG may produce grade level scores one to two grades higher than other formulae.33 37 The Gunning Fog formula calculates grade level and reader age based on number of sentences, their mean length and number of complex words (three or more syllables).38 The FORCAST readability formula was initially used for assessing technical documents by calculating the grade level of text based on number of monosyllabic words. It is the only test not designed for running narrative, for example multiple choice quizzes and applications. As sentence length is not considered, there may be some variability in grade level compared with other readability formulae.33
It was felt the above four formulae allowed comprehensive assessment of an MIS by focussing on various aspects: Flesch Reading Ease—sentence length and syllable number, SMOG—complex word density, Gunning Fog—sentence number/length and complex words and FORCAST—number of monosyllabic words and non-dependence on running narrative.
The readability of 10 corresponding MIS of a sample of commonly prescribed rheumatology medications published in the UK by Arthritis Research UK39 and from Canada published by Rheuminfo40 was also assessed as above. These 10 MIS were representative of the MIS available on both these websites.
Assessment of literal comprehension
Coffs Harbour is a growing regional city of 70 000 people located half-way between the Australian capital cities of Sydney and Brisbane. Its medical specialists provide services to another 50 000 people from the surrounding area. Rheumatology services are provided by two rheumatologists (PKKW and HB) under the auspices of the Mid-North Coast Arthritis Clinic (MNCAC). The MNCAC has over 16 000 patients on its computerised database.
A random sample of patients referred to the MNCAC was asked to read one ARA MIS31 containing information about one of the following medications which the patient was unfamiliar with: online supplementary material 1 MTX,41 online supplementary material 2 non-steroidal anti-inflammatory drugs (NSAIDs),42 online supplementary material 3 adalimumab (ADA),43 online supplementary material 4 abatacept (ABA)44 or online supplementary material 5 prednisone.45 All consecutive patients scheduled for a randomly selected consulting day were contacted via telephone by an investigator (MO or ET). Patients (n=261) were asked whether they were interested in study participation to determine what they understood after reading information from the doctor. Responses are outlined in figure 1. Those who expressed interest in study participation were mailed information about the study and a consent form to be returned in a stamped preaddressed envelope (n=142). Those who agreed to participate were assessed on the day of the planned consultation (n=95). These was no difference in gender or age between those included compared with those not contactable (data not shown). Comprehension was assessed by asking the patient to answer five multiple choice questions (see online supplementary material 6) about the content of the one ARA MIS they had just read. These questions were designed by two rheumatologists (PKKW, HB), a rheumatology nurse (DF) and an education academic with expertise in literacy (JJ). The questions were trialled on small focus groups of patients. A time limit of 15 min in a quiet well-lit room was provided. If needed, study participants could refer back to the MIS while answering the questions.
Supplemental material
Supplemental material
Supplemental material
Supplemental material
Supplemental material
Supplemental material
Inclusions and exclusions.
Patient and public involvement
Previous work by us found that up to 15% of patients had low health literacy and up to one-third of patients incorrectly followed dosing instructions for common rheumatology drugs.23 These findings prompted us to conduct this study which examined the readability of MIS routinely used in our clinical practice. Furthermore, some of our patients had previously commented that the ARA MIS were difficult to understand. A summary of study results will be disseminated to all study participants. Patients were not involved in the recruitment to and conduct of the study. However, many study participants indicated they hoped their study involvement would lead to the development of better written material for future patients.
Statistical analyses
Descriptive summary statistics (mean ±SD and median ±IQR range, as appropriate) were used to analyse parameters. Student’s t-test (unpaired) was used to compare means of normally distributed parameters. The Mann-Whitney U test was used to compare medians of groups. For all statistical tests, p<0.05 was considered significant. Data analysis was undertaken using GraphPad Prism V.6 (GraphPad Software).46 The correlation (r value) between comprehension score and various parameters (age, gender, postcode, highest level of education) was performed using STATA (Stata V.11.1, StataCorp).
Results
Assessment of readability
The mean (±SD) grade level for the ARA MIS calculated using Readability Studio was 11.6±0.1 with a mean reading age of 16.6±0.1 years (table 1). (These were obtained by calculating the mean of the FORCAST, Gunning Fog and SMOG mean grade level and reading age. Due to the heterogeneity of these instruments, the mean of each of these measures is available in the relevant table). The mean (±SD) Flesch Reading Ease score of 50.8±0.6 indicated the ARA MIS were either ‘fairly difficult’ or ‘difficult’33 (table 1). Overall, difficult sentences (>22 words) and complex words (≥3 syllables) made up 9.0% and 18.4% of the text, respectively (table 2).
Readability scores for Australian Rheumatology Association Medicine Information Sheets
Word and sentence statistics for Australian Rheumatology Association Medicine Information Sheets
As the validity of the above readability assessment measures has been questioned due to over-reliance on sentence and word length,47 48 we proceeded to assess patient literal comprehension of the ARA MIS.
Assessment of comprehension
A total of 261 patients were contacted, with 95 study participants (figure 1). Mean (±SD) age of study participants was 60±13.2 years, with 71/95 (75%) women and 24/95 (25%) men (table 3). Nineteen of the 95 (20%) patients had a university degree (table 3). Only 9/21 (43%) and 13/24 (54.2%) patients correctly answered all five questions for adalimumab and ABA, respectively (table 3). Only 7/11 (63.6%) of patients correctly answered all five simple questions assessing literal comprehension of the MTX MIS (table 3). Questions assessing comprehension of the prednisone MIS were correctly answered by most participants (10/11; 90.9%). Of concern, only 21.4% (6/28) of patients correctly answered all questions assessing comprehension of the NSAID MIS. Responses to the five NSAID questions are shown in figure 2.
Answers to NSAID questions.
Assessment of patient literal comprehension (n=95 patients)
Highest level of education achieved (r=0.33, p=0.001) and age (r=−0.3, p=0.0002) correlated moderately strongly with a higher comprehension score.
Comparison of readability scores for Australian, UK and Canadian MIS
Given our findings, we sought to determine using Readability Studio what the readability scores were for MIS used in other countries. The mean (±SD) grade level for 10 of the commonly used UK MIS was 11.8±0.1 with a reader age of 16.9±0.1 years (table 4). The mean Flesch Reading Ease score was 48.5±1.5 classified as ‘difficult’. Readability of the Canadian MIS was easier with a mean (±SD) grade level of 9.7±0.1 and mean (±SD) reader age of 14.8±0.1 years (table 5). The mean (±SD) Flesch Reading Ease score for the Canadian MIS was 66.1±1.0 classified as ‘standard’.33
Readability scores for Arthritis Research United Kingdom Medicine Information Sheets
Readability scores for Canadian Medicine Information Sheets
There was no significant difference in mean grade levels between the Australian and UK MIS (p=0.10). However, the mean grade level of the Canadian MIS (9.7±0.1) was less than that of the corresponding Australian MIS (11.7±0.1, p<0.0001).
The Australian MIS were the longest (mean ±SD, number of words=1474.1±44.6) (table 2) compared with the UK (mean ±SD, number of words=922.4±109.6) (table 6A) and Canadian MIS (mean ±SD, number of words=297.7 ± 19.2) (table 6B). The Australian MIS also had the highest percentage of complex words (three or more syllables, 18%), compared with the UK (16%) and Canadian (14%) MIS.
Word and sentence statistics for (A) UK and (B) Canadian Medicine Information Sheets
Discussion
We showed that the readability of commonly used rheumatology MIS given to patients in Australia, the UK and Canada exceeded eighth grade level—the recommended level for a low-literacy population.11 15 The Canadian MIS assessed were easier to read, although remained slightly above eighth grade level. We found that in a population of patients attending a regional private rheumatology practice where only 20% of participants possessed a university degree, patient comprehension of the Australian MIS was poor, with up to 79% of patients failing to correctly answer all five simple questions assessing literal comprehension of commonly prescribed rheumatology medications. As expected, a higher level of education achieved was associated with better comprehension (r=0.33, p=0.001). This, along with high readability scores, suggested that current ARA MIS may be too difficult for many patients to understand. While comprehension of the Canadian MIS was not performed, this would provide useful information about the effectiveness of these easier-to-read materials.
The Canadian MIS were simpler, more ‘readable’ and included pictures. Many studies have shown that incorporating pictograms into patient information material improves patient comprehension.49–54 One study of 60 patients showed that pictograms improved comprehension of patient information sheets from 40% to 93%.50 Another strategy to improve MIS readability is to shorten the document. However, a shorter, simpler MIS may remove important information and be inadequate for patients with high literacy. Yet, studies have shown both low and high literacy groups recalled information best when the text was easy.55 These findings suggest that written materials designed for patients with low health literacy may also be useful for a general audience.
It is important to consider the primary purpose of providing written health-related information to a patient. Although the provision of information as part of patient education to facilitate informed patient treatment decisions is important, worry over potential medicolegal exposure from a treatment-related adverse event continues to drive complexity of written materials.56
Potential limitations of this study include the type of population studied and the measures used to assess readability. All study participants were from Coffs Harbour, a large regional community on the east coast of Australia. Although one may expect literacy levels to be lower in a rural setting, previous work from our centre showed no difference in health literacy between our patients compared with an urban rheumatology private practice in a capital city.23
There has been criticism of readability formulae such as the Flesch Reading Ease formula, SMOG scale and the Gunning Fog scale.48 57–59 Readability formulae are usually based solely on word length or syllable number. They may therefore fail to adjust for patient familiarity with vocabulary associated with their illness, therefore overestimating the difficulty of written information when read by patients familiar with their disease.57 59 By necessity, health-related written material uses text characterised by polysyllabic technical jargon, which elevates readability formulae scores.60 For example, exchanging ‘adalimumab’ for ‘Humira’ in the Australian MIS increases the Flesch Reading ease score from 46 to 50 and reduces the Gunning Fog score from 12.7 to 12.5 (the SMOG remains unchanged at 12.8). Readability formulae fail to account for the stylistic properties of text as well as grammatical errors, which influences the readability of written text. Textual coherence, that is, the relationship and connection between sentences within a document and the relationship between the reader and practitioner are also unaccounted for. Finally, readability formulae do not usually consider visual and design factors which may influence MIS readability or patient comprehension.61 62 While the Flesch Reading Ease formula tends to over-estimate readability of health-related material due to its lower level of expected comprehension criteria,58 the SMOG formula is appropriate for assessing health-related written information as it has been validated against 100% comprehension.58 One approach to addressing these limitations is the use of a more holistic linguistic framework for assessing written patient information which incorporates structure, factual content and visual aspects of the material as well as the relationship between writer and reader.48 This method has been validated using RA medication leaflets in an Australian cohort of patients with RA.63 However, the education level of patients in that study exceeded that seen in our cohort, with 17/27 (63%) having completed tertiary studies compared with 19/95 (20%) in ours.
In view of the potential limitations of readability formulae, we were careful to assess patient literal comprehension of various ARA MIS. As suggested by the relatively low readability scores of the ARA MIS, patient literal comprehension of a selection of the ARA MIS was poor. Due to the simplicity of the five questions posed to the patients, we hoped a satisfactory score would be correct answers to all five questions. However, this only occurred in 21% of patients for NSAIDs and 40%–60% of patients for the MTX, ADA and ABA MIS.
Despite the confines and limitations of readability formulae, we believe they remain an important guide when developing written patient information or revising original drafts. This has been validated by several studies that used these formulae to simplify existing written patient information—resulting in enhanced patient comprehension.64 65 We hope the results of this study will encourage clinicians from rheumatology and all other specialities to consider the health literacy of their patients and readability of the written information they provide, particularly given the potential of technology to improve patient education.
Conclusion
Medication information sheets currently used by many rheumatologists in Australia, the UK and Canada exceed eighth grade level—the recommended level for a low-literacy population. This may explain why patient comprehension of the information contained in these materials is limited. Comprehension may be improved using simpler, shorter words and sentences with greater use of pictures and infographics. This may lead to greater patient medication adherence, understanding of their condition and reduced medication-related errors. It is hoped our findings will encourage all healthcare professionals to consider the appropriateness of written healthcare material provided to patients. The health literacy of patients should always be considered when communicating a management plan.
Acknowledgments
We thank all the patients who participated in this study.
References
Footnotes
Patient consent for publication None declared.
Contributors MO, ET: were responsible for data acquisition. MO: was responsible for drafting the manuscript and data analysis under the supervision of PKKW. PKKW, JJ, DF, HB: conceived and designed the study. All authors contributed to interpretation of data and revision of the manuscript and approve the final manuscript.
Funding This study was partially funded by an unrestricted research grant from Bristol Myers Squibb.
Competing interests None declared.
Ethics approval Approval as a low/negligible risk project was obtained from the New South Wales North Coast Human Research Ethics Committee (NCNSW HREC No LNR 150).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data available.