Discussion
Only right eyes were included in this study with NETRA36 to avoid any concerns about combining right and left eyes in single samples.37 Previous studies3–5 7–13 suggest that measurements with NETRA are valid6–10 13 26 27 and repeatable, even in the absence of cycloplegia8 26 27 but repeatability increases with cycloplegia.3 8 11 12 26 27 Direct comparisons of the results for repeatability from this paper to previous publications are difficult as other studies mainly analysed the validity of NETRA against subjective and/or objective refraction,3–7 9–13 including one paper by the authors here.8 However, with reference to McAlinden et al31 and Bland and Altman,38 95% within-subject SDs 
and repeatability values were calculated as well as SEs for quantities such as the mean differences of repeated samples as well as the LoA (table 2).8 26 27 31 The SEs are small (0.02 D to 0.03 D) for the mean differences and also for the LoA where SE is 0.03 D or 0.06 D. Thus, the mean differences and LoA for the repeated samples are precisely known. The mean differences are also small (table 2) even in the absence of cycloplegia; thus, for the participants their repeated measurements, on average, hardly differed but the differences themselves are more widely spread as seen in the Bland-Altman plots (figure 3). Repeatability,39 or measurement error (the estimated differences from the true and measured values),39 at a 95% level of confidence is:29–31 38 39 Repeatability 
(=2.77 
) and this provides the probable limits within which 95% of differences occur. So, for the right eyes (OD) and the stigmatic coefficients (FI=M) after removal of one outlier, the 
and the 95% repeatability is 
(Use of cycloplegia would likely reduce this value.) Thus, the differences of repeated measurements for participants is ≤1.6 D at a 95% level of confidence. So, even though the mean differences (see table 2) are very small, individual differences (see figure 3) with NETRA can be relatively large in the absence of cycloplegia and outliers can have marked effects. However, repeatability values are smaller (≈0.57 D) for the antistigmatic coefficients of power (FJ and FK or J0 and J45).
Previous studies3–10 13 40–44 including one8 by the authors addressed the issue of validity of NETRA in relation to methods such as subjective refraction,6 8–10 13 40–44 retinoscopy11 or objective refraction.12 43 In some studies,3 10–12 26 44 cycloplegia was included and, for example, Pamplona, found that with cycloplegia NETRA measured refractions within 0.5 D to that of subjective refractions and within 1 D to autorefraction.3 With cycloplegia, Pamplona3 found that absolute mean SE difference of NETRA and subjective refractions reduced from 1.35 D to 0. 4 D. (The authors here (NH and AR) also confirmed this where NETRA was compared with subjective refraction and absolute mean SE difference reduced from 1.17 D to 0.04 D,8 26 without and with cycloplegia.) Further studies by Pamplona et al4 5 found that, for example, NETRA and subjective refractions, even without cycloplegia, gave similar results on average (absolute mean differences for M, J0 and J45 −0.22, 0.07 and −0.01 D) but, as anticipated, there were individual eyes where the absolute differences were larger than these averages.
Bastawrous et al,9 in 34 eyes of adults (23–81 years, but mostly presbyopic) found mean SE difference of −0.32 D as against −0.31 D (Hasrod26) when comparing NETRA and subjective refractions, both without cycloplegia. Solaka et al10 similarly compared NETRA and subjective refraction without cycloplegia but analysed the clinical components of refractions separately10 without transformation to power vectors and the absolute mean differences for the spherical component ( 
) was 0.48 D to 0.64 D depending on which version of NETRA was used whereas Hasrod,26 without cycloplegia, found this to be greater at 0.98 D. Sample sizes were 48 eyes and 279 for Solaka et al10 and Hasrod,26 respectively. Gaiser et al,13 also compared NETRA and subjective refraction without cycloplegia in 27 adult participants and found absolute mean SE difference to be 0.31 D as against 0.94 D (for 279 right eyes) and 0.71 D (279 left eyes) from Hasrod.26 Hasrod’s sample included younger participants and the differences here might relate to factors such as fatigue with younger participants, possible outliers, and difficulties that younger individuals might experience with understanding and using NETRA. Tousignant et al40 found the following when comparing NETRA and subjective refractions without cycloplegia: mean difference for FI (=M) of 0.53 D and the 95% LoA width of 2.80 D. Hasrod26 measured the mean difference for FI (=M) of 0.97 D and 95% LoA width of 3.29 D (right eyes). With removal of an outlier for the right eyes, the LoA range decreased to 2.21 D (see table 2), and this emphasises the importance of outliers and their effects in analyses of refractive state (see figure 3). (Although not included here, the LoA range was smaller for the 279 left eyes at 1.42 D,26 even without cycloplegia.) Jeganathan et al42 also found small but significant differences when comparing NETRA to subjective refractions.42 Their study involved refractive states measured with (75 eyes) and without cycloplegia (152 eyes) and analysis was based on spherical equivalent (SE=FI=M). Mean differences for SE for NETRA and subjective refraction ranged from 0.25 D to 0.67 D, depending on whether cycloplegia was used and whether absolute SE or SE were applied to the analysis.
When comparing NETRA to methods other than subjective refraction, generally similar conclusions are reached, namely that cycloplegia is necessary with younger participants but repeatability is generally adequate to good. For example, Jeganathan et al42 also compared NETRA and retinoscopy while Li et al43 compared non-cycloplegic NETRA to table-mounted autorefraction (Topcon KR-800S) in a sample of 100 participants (19–92 years with mean age 54.6 years) and found that the mean SE difference was 0.11 D with 95% LoA width of 2.18 D. (Mean SE difference in NETRA refraction and subjective refraction was slightly larger at 0.34 D but with smaller 95% LoA width of 1.87 D.) They point out that NETRA produces results that are slightly less negative in power than for autorefraction or subjective refraction. However, cylinders across methods were similar and they recommend checking the spherical component of power before prescribing spectacle or contact lens compensations should NETRA be used in that process.
Ee and Samsudin44 compared NETRA with automated refraction and subjective refraction. They had 204 participants with mean age 36.6 years (SD, 15.7 years) and they found that NETRA had wider 95% LoA than for automated refraction (3.63 D vs 3.06 D, respectively). NETRA gave similar refractive results as for automated refraction but sometimes overminused results in comparison with subjective refraction, and they mentioned that there were practical issues that complicated the use of NETRA. (Authors such as Rosenfield and Ciuffreda45 or Rao et al46 compared other types of handheld instruments such as the SVOne Handheld Autorefractor (also uses a smartphone like NETRA) or Instaref R20 (also uses wavefront aberrometry as for NETRA) to autorefraction and subjective refraction but studies involving NETRA to methods such as autorefraction or retinoscopy are limited in number.)
The stereo-pair comets8 26 27 47 (figure 1A) and comet lengths (figure 1B) showed that for many eyes, there was little difference between repeated measurements as indicated by the mostly short comets (median length ≤0.5 D). There are, however, exceptions and longer comets are seen in figure 1A and an outlier with a comet length of 10.47 D was noted (see figure 1B). This outlier was retained in the analysis specifically to demonstrate that ocular accommodation can have profound effects with measurements with NETRA in the absence of cycloplegia. However, figure 1B illustrates that except for two eyes, the comet lengths were mostly <1.5 D. Repeatability and the differences in repeated measures for the right eyes can also be evaluated using figure 1C where a 95% confidence ellipsoid21 is shown for the mean difference. The ellipsoid is small (volume: 0.004 D3) with its centroid (a mean difference in clinical terms of only 0.09 –0.06 × 107) just above the origin that represents the null matrix (O D) or a zero difference in repeated measures. The ellipsoid overlaps the origin, so, on average, NETRA produces a mean difference that is very close to zero and the instrument is repeatable even in the absence of cycloplegia. However, repeatability for individual eyes is not necessarily as good and the outlier (see figures 1A, B and 3A) is an example here.
Similarly, the scatterplots with 95% distribution ellipsoids (figure 2), overlap closely with similar centroids (means) as provided in table 1, suggesting that the repeated samples for these right eyes, and by inference, their populations did not differ much. If this was untrue, the two ellipsoids would be seen as separate. Hypothesis tests of 95% level of confidence also indicated equality of sample variances and covariances, and likewise for their means.
Bland-Altman plots (figure 3) with mean differences (all very close to 0 D, see table 2) and their SDs (0.29–0.31 D, table 2) suggested that, in general, the distributions for the differences between repeated measurements for the antistigmatic coefficients of power (FJ and FK; or J0 and J45 if one prefers) had similar variability and good agreement. LoA widths at a 95% confidence level (see table 2) are smaller (1.12 to 1.23 D) for FJ and FK than for FI. So, stigmatic coefficients (FI=M) were less similar with weaker agreement (although mean differences remained close to 0 D, and the SD for the differences was greater at 0.55 D), and 95% LoA widths were larger (>3 D); outliers and the absence of cycloplegia probably had important influences. Outliers within the samples (see figures 1–3) could be due to instrument myopia.47 As for autorefractors, when a near target is viewed within NETRA without the use of a cycloplegic, the proximity of the perceived target stimulates proximal accommodation which can lead to overaccommodation and more negative or myopic readings. Other factors include internal and external illumination levels, astigmatic aberrations, optical decentration, the contrast and type of targets and direction of focus and extraneous variables such as age and user experience and possibly fatigue that may also influence instrument myopia.40–48
Possible limitations here are that the sample (n=279 participants) was relatively small and selected via non-random convenience sampling and various ophthalmic procedures (to exclude eyes with ocular disease or refractive states that would be outside the measurement range of NETRA) rather than via a randomised selection and thus results are not necessarily representative of the cultural, age, ethnic and gender diversity present in the general population. However, given the primary aim of this study that concerns NETRA repeatability, this sampling method was not regarded as a serious limitation. Previous studies3–7 9–13 involving NETRA used much smaller sample sizes (≤36 participants), and right and left eyes were analysed together in their sample/s and no mention was made of issues such as reflection of cylinder axes for, say, all left eyes or the impact this might have regarding analysis and effects on independence of interocular data. Although only results for the right eyes are included here, the investigation involved both eyes of 279 participants but analysed right and left eyes separately.8 26 27 Additionally, results were assessed using mainly multivariate statistical methods for dioptric power and refractive behaviour and this is one of very few studies8 26 27 where such methods were applied to NETRA to assess repeatability.
Further studies comparing non-cycloplegic and cycloplegic results would be useful in understanding the nature and extent of ocular accommodation on NETRA validity and reliability, especially concerning the spherical (stigmatic) component (FI) of power, and future papers by the authors will address these issues. NETRA measurements and possible correlations to factors such as ethnicity, gender, age and pupil size would also be helpful. Age and NETRA in very large samples would be useful, with and without cycloplegia, and subjective refractions also would be helpful. The use of NETRA (or similar mobile devices) for the self-monitoring of refractive behaviour of specific individuals with diabetes, keratoconus, juvenile myopia or during pregnancy might be other areas for exploration. The relationship between NETRA and prescribed lenses is another interesting topic to determine the extent to which such compensating lenses could be directly prescribed using only NETRA, again with and without cycloplegia for younger individuals. Interoperative use of NETRA, for example, with refractive surgery might be another area for investigation.
