Semantic segmentation of gonio-photographs via adaptive ROI localisation and uncertainty estimation

Introduction

Glaucoma is a group of neuropathies that irreversibly damage the optic nerve, possibly leading to blindness. Increased intraocular pressure, normally regulated in the anterior chamber angle (ACA) by the trabecular meshwork (TM), is widely recognised as a primary risk factor.1 The underlying cause of a limited TM functionality can be used to categorise the disease and choose the best medical treatment, making the ACA assessment fundamental.1

Current international guidelines recommend gonioscopy to assess the ACA.2 3 However, conventional (manual) gonioscopy presents known limitations, for example, challenging acquisition of digital images and steep learning curve. To overcome these, new devices have been developed.4 5 The acquisition of digital images of the ACA enables automated data analysis, for example, using deep learning (DL).

DL systems have pushed the state-of-the-art in image processing, with numerous applications in ophthalmology. Such systems can be divided into two main categories:

• Classification algorithms: assigning each input to one of a set of classes, for example, healthy/pathological sample discrimination in fundus images.6

• Segmentation algorithms: performing a pixel-wise classification of inputs, thus highlighting target structures like the vasculature in fundus images or retinal layers in optical coherence tomography (OCT) data.7–9

DL has been used for glaucoma-related tasks on OCT.10 In particular, the detection of angle closure using anterior-segment (AS) OCT cross-sections of the ACA has proved effective.11 12 However, AS-OCT does not allow direct inspection of the ACA surface, preventing the evaluation of important biomarkers, such as neo-vascularisations and TM pigmentation; the detection of anterior synechiae (SY) may also be problematic.

The literature on DL applications to gonio-photographs is much more limited. This imbalance and the complementarity of AS-OCT and gonio-photographs for ACA evaluation support the importance of research in automated gonio-photographs analysis.

The automatic detection of angle closure in gonio-photographs has been studied.13 However, algorithms so far only provide a global characterisation of input images, possibly missing important local features and preventing a comprehensive analysis of layers’ interfaces and their changes over time, for example, for follow-up.

We present a DL system for semantic segmentation of digital gonio-photographs to allow the assessment of local ACA morphology. Our algorithm provides an accurate segmentation of five anatomical layers: iris root (I), ciliary body band (CBB), scleral spur (SS), TM and cornea (C). We discuss the advantages of this model with respect to our previous work,14 which is, to our best knowledge, the only full publication on automated ACA layers segmentation. In particular we address the main limitation in ‘DL-based segmentation of gonioscopic images’,14 regarding results interpretability, in two ways: (1) implementing a region of interest (ROI) localisation module to refine segmentation results and (2) estimating output epistemic uncertainty through a Monte Carlo dropout approach.15 Moreover, we refine the model architecture and fine-tune training hyperparameters to improve the overall segmentation performance.

Our work fills a gap in current research for clinical applications of DL (automated analysis of gonio-photographs). Our algorithm can be used as backbone processing for the measurement of structures of the ACA, for example, size of anterior SY, and their changes over time. Other possible use comprise, but are not limited to, the localisation of the trabecular meshwork, as a preprocessing step for pigmentation grading (eg, prior to laser trabeculoplasty), and the improvement of auto-alignment and auto-tracking systems for data acquisition in non-contact clinical examinations, that are gaining importance due to the COVID-19 pandemic.

It is also worth noting that segmentation algorithms widely employed in other clinical applications are not suitable for off-the-shelf use with our digital gonio-photographs. This is due to ground-truth limitations discussed in the Materials and methods section. Our model design has been specifically conceived to process ground-truth data to maximise output accuracy and interpretability.

This paper is structured as follows: Materials and methods section describes the dataset and the model architecture; Results section compares the performance of the proposed model with those of our previous work14; Discussion section summarises our findings, highlighting improvements and current limitations.

Materials and methods

Network architecture and training

Figure 1 summarises our approach to provide an accurate segmentation map of ACA layers and deal with the limitations of our ground truth effectively. The data representation generated by the network encoder is processed in parallel by two independent network units: the semantic decoder, which returns the estimated class for each image pixel (the segmentation map), and the ROI decoder, which highlights an image ROI (the sharp and well-lit area). The two outputs are combined to provide a final segmentation map within the estimated image ROI.

Request permission

Figure 1

Network architecture overview with example of intermediate and final outputs for a given input image. C, cornea; CBB, ciliary body band; I, iris root; NA, un-annotated region; ROI, region of interest; SS, scleral spur; TM, trabecular meshwork.

The basic processing blocks are shown in figure 2A. The convolutional block (figure 2A1) (ConvB) is a sequence of dropout18 (0.2 drop probability), 2D convolutional layer (3×3 kernel size, with Kaiming-He uniform initialisation,19 zero-padding), instance normalisation20 and Leaky-ReLU activation21 (0.01 negative slope). Dense blocks (figure 2A2) (DenseB), inspired by,22 are sequences of four ConvBs with two intermediate concatenations of output pairs. The encoder block (figure 2A3) (EncB) is a DenseB followed by an input-output concatenation. The decoder block (figure 2A4) (DecB) is a DenseB followed by a ConvB that reduces the number of feature maps.

Request permission

Figure 2

(A) Basic processing blocks: convolutional (A1), dense (A2), encoder (A3) and decoder (A4) blocks; (B) detail of the proposed architecture: encoder (B1), semantic decoder (B2) and region of interest decoder (B3).

Figure 2B captures the three network components in detail. The encoder figure 2B1 is composed of two initial ConvBs with eight filters (drop out of the first ConvB is disabled), followed by three combinations of max pooling and EncB, each doubling the number of feature maps. A final DenseB generates the latent data representation. The semantic decoder figure 2B2 up-samples feature maps via max un-pooling23 and concatenates them with those forwarded by the encoder. The resulting feature maps are processed through DecBs, each reducing by a factor 4 the number of feature maps. The processing ends with a DenseB and a 1×1 2D convolution, followed by a six-class softmax activation (the NA class is considered for consistency but does not contribute to semantic decoder optimisation). We can interpret the sequence of encoder and semantic decoder as a segmentation U-Net24 trained via weighted categorical cross-entropy with equal weights for all the anatomical layers considered and weight 0 for the region of the image not annotated by the experts.

The ROI decoder figure 2B3 is a simple and convenient way to filter out the artefacts expected in the segmentation map periphery (since dark and blurred image areas are not informative enough to be correctly classified), overcoming the issue on results visualisation in ‘DL-based segmentation of gonioscopic images’.14 A detached copy of the data representation is processed through a DecB, a ConvB (dropout is deactivated) and a 2D convolution with a sigmoid activation. The intermediate feature maps are up-sampled by a factor 4 using bilinear interpolation.

To train the ROI decoder we first generated an ROI likelihood map from the original semantic annotations: all annotated pixels were first assigned value 1 (binarisation), gaps between layers (if any) were filled to obtain a dense region that is then smoothed to simulate a probabilistic distribution of annotator’s evaluation confidence (online supplemental figure 2). ROI likelihood maps were used as reference when computing the mean squared error loss for the ROI decoder, independently from the semantic segmentation optimisation.

After ROI binarization (threshold 0.5 in our results), the two intermediate results are multiplied.

Semantic segmentation of ACA layers and ROI localisation are two un-correlated tasks that rely on different interpretations of the same image features. The former looks for patterns and textures that characterise the different anatomical layers; the latter evaluates sharpness and illumination variations across the frame. We verified experimentally that other systems, like attention mechanisms,25 are not effective at addressing this problem since they do not deal with the two tasks independently.

Optimisation of model’s weights was performed through stochastic gradient descent with Nesterov momentum (0.9), learning rate equal to 0.01 and 8 images per batch. The model was designed and trained using Python (3.7.9) and PyTorch (1.7.0).

Results

Figure 3 compares an example of combined network output (edges of the segmentation map refined by the ROI, bottom right) with the ground truth delineation (top right). The segmentation is noticeably accurate and the ROI very similar to that highlighted by the annotator. The uncertainty (variance) map provides useful information about the model confidence in the results. In this case, the variance map only highlights layers interfaces, as expected even when the segmentation is accurate, since layers boundaries are often not very sharp features.

Request permission

We compared the performance of our previous14 and current models on the new dataset. First, we split it into a test set (31 images from 25 examinations of 17 patients) and a training-validation set (243 images from 189 examinations of 145 patients), with similar distributions of the features described in the Materials and methods section. We then randomly split the training-validation set into five folds of 29 patients each, to cross-validate the two models, each fold consisting of a variable number of images (mean: 48.6, SD: 5.3 for the previous14 and 5.7 for the new implementation). The algorithm in ‘DL-based segmentation of gonioscopic images’,14 used as baseline, was trained as per the original approach: assigning loss class weights equal to the ratios between the average size of each layer and the average size of a reference layer (I), and storing best model weights after each epoch returning a lower validation loss. The new system was trained to maximise segmentation accuracy within the annotated images area, storing model weights after each epoch returning an increased validation accuracy. Both loss and accuracy only account for pixels within the annotated image regions, since segmentation performance on the un-annotated area is, by definition, not measurable.

Comparing average per-layer metrics on the hold-out test set (table 2) between the two models, our new one returned overall higher Dice scores, in particular for CBB and SS (+2.4% and +5.6% on average), two layers of fundamental clinical importance. To interpret these values correctly, one must consider that, importantly, the average inter-observer Dice scores for CBB and SS reported in our pilot inter-annotator variability study17 were about 75% and 65% (the lowest), making our results consistent with the average agreement between experts. Our previous approach promoted sensitivity (less false negatives) for layers with lower inter-annotator agreement (CBB and SS) while the new one promotes precision (less false positives).

The pixel-wise accuracy within the annotated image region was much higher for the new model (91% vs 87%).

ROIs estimated by the model and those identified by the annotators cannot be compared quantitatively given the subjective estimation of their boundaries based on smooth illumination and focus transitions. To qualitatively validate the ROI decoder of our model we asked a clinician, blinded to our ground truth data, to verify that ROIs estimated by the model did not leave out any ACA feature of clinical interest. The result was that our model highlighted an appropriate ROI in every test image and in every cross-validation fold, suggesting a stable, reliable approach.

Uncertainty estimation highlighted local segmentation irregularities correctly.

Discussion

Gonioscopy is a clinical standard. New imaging devices allow to acquire digital gonio-photographs more consistently, enabling automatic analysis. The anatomy of ACA layers is clinically relevant as related to a high-prevalence disease (glaucoma), therefore automatic systems supporting diagnosis will be increasingly important. Systems for automated assessment of ACA conditions have been developed on AS-OCT data.11 12 However, AS-OCT scans do not allow the evaluation of important features of the ACA, such as the TM pigmentation, and make the measurement of others, for example, anterior SY, problematic. For these reasons, AS-OCT will likely not replace gonioscopy soon, remaining a complementary source of information.

Automatic systems for detecting angle closure in digital gonio-photographs have been proposed13 assigning only a single state to each image (a label that globally represent what the image shows) that may obscure relevant local conditions. In order to assess the morphology of ACA accurately and account for local variations of layers’ interfaces, a segmentation approach is certainly more suitable. State-of-the art semantic segmentation networks cannot deal with limitations posed currently by the annotation of our gonio-photographic images. Our previous DL-based segmentation model14 (the only one in the literature to our best knowledge) achieved a promising performance within the area annotated by experts, but failed to filter out the un-informative image periphery.

We presented here a solution to overcome previously reported limitations by adaptively identifying an ROI to refine segmentation maps and improve results readability. Moreover, our new calibrated model can support data analysis and interpretation further by highlighting uncertain segmentation areas. This is done estimating pixel-wise epistemic uncertainty as activations variance over multiple segmentation candidates. This will be further investigated in our future work.

The overall segmentation accuracy (~91%) is promising, although within a limited dataset. The layer-wise Dice scores on the test set correlate well with results from a pilot inter-annotator variability study.17

Finally, we discuss the current limitations of our research. We acknowledge the limited amount of data used. Although our dataset is representative for a variety of ACA features, larger datasets of annotated images are needed to train and evaluate comprehensively DL systems for gonio-photographs analysis in clinical practice.

The relative novelty and still limited use of digital devices for gonioscopy currently limits the availability of data for rarer conditions. In particular, more images representing complex layers morphologies, for example, SY, shall be collected and annotated in the future to further improve the generalisation capabilities of our models.

Our segmentation model fills a gap in current applications of DL for ophthalmology, enabling a more accurate data analysis than classification algorithms for angle closure and possibly providing a processing backbone for the measurements of clinical parameters, for evaluating changes of layers morphology over time and for developing auto-tracking systems for device alignment in non-contact clinical settings.