Discussion

This AGREE II analysis highlighted several areas where future CPGs for the management of adult cataract could be improved. All three guidelines fell short in Stakeholder Involvement (Domain 2) by not delineating specific roles of GDG members or the involvement of patient representatives. Diversity among GDG members is integral to formulating less extreme recommendations, preventing the promotion of treatments biased toward a particular specialty, and ensuring the comprehensive inclusion of relevant scientific evidence.15 The AGREE II states that all stakeholders in the care process, including patients, should be involved in recommendation development. It was not possible to determine if patient groups were consulted, with the exception of the RCO guideline, which mentioned public involvement in the guideline development process from the ‘College Lay Advisory Group’. None of the guidelines specified the outcomes gathered from public input. Included within Domain 2 is the requirement for full disclosure of GDG members’ roles and areas of expertise. Although the guidelines addressed GDG members’ areas of expertise, none mentioned their individual roles pertaining to guideline development.

Transparency of the guideline development process and strength of supporting evidence evaluated under Rigour of Development (Domain 3) presented areas for improvement across the board. First, the external review processes were unclear in all guidelines as they lacked descriptions of external review methods, external reviewers, patient input, information gathered from external review and how the outcomes informed the guideline development. In the AAO guideline, the role of the Consumers Union, the only non-medical external reviewing body, was not clearly defined as representing patients or patient advocates.16 If the Consumers Union was intended to represent cataract patients, this objective should be explicitly stated in the guideline. The other guidelines provided no details of the external review process, so it was impossible to verify if public input through the external review played a role in shaping recommendation formulation. Second, the rigour of supporting evidence was variable, including the lack of a clear evidence rating system (AAO) and failure to disclose detailed search terms (COS, RCO). Consistent evidence rating systems as used by the COS and RCO should be implemented in all cataract CPGs because they illuminate the strength of supporting evidence and clarify which statements are recommendations. Similarly, detailed evidence search terms and methods were only available for the AAO guideline. Literature search methods should be explicitly described to enable independent reproduction of developers’ conclusions and to facilitate transparency of guideline development methods.

On Clarity of Presentation (Domain 4), the guidelines employed different methods to ensure utility of recommendations, yet each displayed distinct needs for improvement. The RCO recommendations were flagged throughout the text using boxes, and the COS recommendations were easily identifiable with clear numbers and bold headings. However, neither document provided a concise summary of key recommendations. The AAO guideline provided a supplementary list of 11 major recommendations under ‘Highlighted Findings and Recommendations for Care’, but recommendations in the main text were harder to locate relative to those of the COS and RCO guidelines. Some of the recommendations were primarily summary statements (e.g. ‘cataract surgery is a procedure appropriately utilized in the United States’) rather than actionable recommendations.

Regarding Applicability (Domain 5), the guidelines varied in their consideration for resource implications, auditing criteria and the provision of supplementary documents for guideline implementation. While the AAO guideline included sections on Socioeconomic Considerations, Cost Effectiveness and Cost Considerations, and the RCO guideline included a discussion of problems for the provider, local eye unit and commissioner, overall there was insufficient disclosure of methods by which cost information was sought, and the impact of such considerations on guideline development. The guidelines provided sparse auditing criteria except for the mention that ‘outcome measures should be varied and should match the indications for surgery’ (COS). The intangible influences of differing healthcare systems’ methods of reimbursement are difficult to assess, and it may not be possible to control for the different influences health insurance might have on assessing applicability. Nonetheless, among the CPGs, there was an overall paucity of supplementary algorithms, surgical checklists and other tools to facilitate implementation of recommendations.

All CPGs should maintain Editorial Independence (Domain 6), yet all three guidelines failed to specify methods by which potential competing interests were sought and how the potential COI influenced the formulation of recommendations. The AAO provides a blanket statement online indicating compliance for all its CPGs with the Council of Medical Specialty Societies Code for Interactions with Companies,17 but no specific information on potential COI influence on the development of the cataract CPG. Lack of COI transparency is of particular concern when chair members and a majority of GDG members have financial interests, as was the case for the AAO. Whereas the AAO and COS guidelines mentioned independence of the funding body and provided a list of specific financial disclosures, the RCO guideline provided no mention of the funding body and no comprehensive financial disclosures for GDG members, stating only that the ‘members of the working party have made the Chair aware of their commercial relationships.’

We acknowledge the limitations to this study. First, rating systems involving human evaluators will always limit objectivity, including potential biases associated with nationality. However, the AGREE II system has been demonstrated to be valid and reliable, especially with multiple independent raters, as this study used.10 ,11 All raters completed the AGREE II tutorial which provides clear benchmarks for scoring along the 7-point scale.13 The independent evaluators were highly reliable (based on ICCs of 0.78, 0.74 and 0.80), which is consistent with previous AGREE II studies.9 Second, AGREE II examines the methodological rigour and reporting transparency rather than recommendation validity or evidence quality.9 While rigorous guideline development processes do not necessarily guarantee quality guideline content, suggestions for methodological improvements can be universally applicable and transparency can facilitate evaluation of clinical applicability. Finally, the AGREE II gives equal weight to all six domains, whereas domains may vary in depth and importance. We reconciled this potential issue by using the Overall Assessment and evaluator comments to assess each guideline holistically and identify specific areas for improvement.

In summary, the primary weaknesses of the adult cataract CPGs were in the areas of patient involvement, transparency of the guideline development process, applicability and editorial independence. We have formulated several recommendations for the targeted improvement of adult cataract CPGs, using models from the reviewed CPGs when possible (table 3). This assessment using the AGREE II instrument underscores the importance of regular evaluation of CPGs to optimise patient care.