Endocrine Journal
Online ISSN : 1348-4540
Print ISSN : 0918-8959
ISSN-L : 0918-8959
ORIGINAL
Assessment of factors that determine the mean absolute relative difference in flash glucose monitoring with reference to plasma glucose levels in Japanese subjects without diabetes
Satoshi YoshinoEijiro YamadaShuichi OkadaYasuyo NakajimaRyo ShibusawaRyota UeharaShunichi MatsumotoKazuhiko HoriguchiEmi IshidaTsugumichi SaitoMasanobu Yamada
Author information
JOURNAL FREE ACCESS FULL-TEXT HTML

2020 Volume 67 Issue 5 Pages 537-544

Details
Abstract

The Abbott FreeStyle Libre flash glucose monitoring system (FGM) is a recently introduced, but widespread continuous glucose monitoring system. While its mean absolute relative difference (MARD) value indicating its accuracy is acceptable with reference to the self-monitoring of blood glucose (SMBG) levels, few reports have examined the MARD in sensor glucose values of FGM (FGM-SG) with reference to plasma glucose (PG) levels and the factors determining it. We performed oral glucose tolerance tests (OGTTs) in 25 Japanese subjects without diabetes. Parkes error grid analyses showed that FGM-SG with either SMBG or PG levels as a reference met International Organization for Standardization criteria. The MARD in FGM-SG with reference to SMBG levels was 10.9 ± 4.1% during OGTTs. Surprisingly, the MARD in FGM-SG with reference to PG levels was 20.3 ± 10.3% during OGTTs, revealing a discrepancy in the accuracy of FGM-SG compared with that of PG levels; moreover, the MARD showed negative correlations with fasting blood sugar level, homeostasis model assessment insulin resistance index, and body mass index (BMI). Multiple regression analyses revealed that BMI contributed the most to the MARD when FGM-SG and PG level were compared, as lean individuals have a greater MARD regardless of glucose levels. Inaccurate FGM data could potentially increase the risk of inappropriate treatment; consideration of such factors is critical to ensure reliable FGM values.

KNOWLEDGE OF GLUCOSE LEVELS is important for patients with diabetes to maintain a tight glycemic control and to reduce the risk of diabetic complications [1]. While self-monitoring of blood glucose (SMBG) is a popular method for determining daily glucose levels in these patients, continuous glucose monitoring (CGM) has recently been introduced [2]. The Abbott FreeStyle Libre flash Glucose Monitoring System (FGM) was recently approved worldwide, including in Japan, and is a personal CGM system used by patients to determine real-time glucose levels. The system is based on a reading device that scans a sensor that is inserted into the body [3].

Recent studies have shown that FGM helps in improving not only glycemic control but also the quality of life (QOL) in patients with diabetes [4]. It might be used as a substitute for SMBG. Although SMBG-related criteria have been approved (ISO15197-2013) [5], no criteria have been developed for CGM. FGM-SG value determination based on interstitial fluid measurements, results in differences between SMBG and FGM-SG values [6]. The mean absolute relative difference (MARD) is widely recognized as a reasonable parameter to characterize the performance of CGM systems. MARD can be easily calculated to determine the analytical performance of CGM system [7-9]. More importantly, the MARD value, used to estimate CGM accuracy, is said to differ between FGM and SMBG values, depending on the study setting and patient characteristics [10]. A few reports have shown differences between FGM-SG and SMBG levels; however, no reports have examined the differences between FGM-SG and PG levels. Therefore, in this study, we examined the MARD of FGM in detecting PG levels by performing oral-glucose tolerance tests (OGTTs) in healthy Japanese volunteers without diabetes, and assessed the determining factors.

Materials and Methods

Ethics statement

This study was approved by the Gunma University Institutional Review Board (ID 1521) and registered with an official Clinical Trial Registry (UMIN, ID:UMIN000028191) and conforms to the provisions of the Declaration of Helsinki (as revised in Fortaleza, Brazil, October 2013). Subjects provided written informed consent before undergoing any study-related procedures.

Patients and FGM data analyses

Twenty-five healthy volunteers without diabetes were enrolled in this study. After the subjects were examined to exclude any with diabetes, the Abbott FreeStyle Libre Pro (Abbott Diabetes Care Inc, CA, USA) sensor was precisely inserted into the back of the upper arm. The SG values were downloaded and analyzed with the FreeStyle Libre© software. Downloaded datasets were further analyzed using EasyGV© (N. R. Hill, University of Oxford, Oxford, UK; available at www.easygv.co.uk) for the following parameters: mean glucose level, SD, mean amplitude of glycemic excursion (MAGE), and percent coefficient of variation (%CV).

Performing 75-g and 100-g OGTTs

To evaluate factors that determine the MARD in FGM-SG values, 75-g and 100-g OGTTs were performed 3–7 days and 7–14 days after installing FGM, respectively. Venous blood was collected at 0, 30, 60, and 120 minutes after glucose load to determine the fasting plasma glucose (FPG), fasting immunoreactive insulin (F-IRI), and PG levels; these were performed under contract by SRL, Inc (Tokyo, Japan). PG levels were measured by ultraviolet absorption spectrophotometry using collecting PF2 tubes supplied by SRL, Inc (Tokyo, Japan). Moreover, patients performed SMBG using OneTouch Ultra (LifeScan, Milpitas, CA, USA) at each time point. MARD was calculated with the following formula: MARD (%) = 100 × |(FGM-SG-reference glucose)|/blood glucose [7]. In this study, SMBG and PG levels were used as references. Parkes error grid analyses, the widely accepted tool for the clinical accuracy of blood glucose, were also performed. Each of the zones represents a degree of risk of an adverse outcome due to the error in measured blood glucose values [11].

Statistics

All results were expressed as mean and SD for continuous variables and as absolute numbers and relative percentages for categorical variables. Group comparisons of continuous variables were performed using analysis of variance and Wilcoxon rank-sum test for non-normally distributed data. The chi-square test was used for categorical variables. Spearman’s rank correlation was used to measure the correlation between two variables. Multiple comparison of the variable for 75 g OGTT was performed using the Dunnett’s test. Multivariate logistic regression analysis was performed for MARD of Libre and PG. Characteristics that were significant (p < 0.1) in the univariate analysis were included in the multivariate analysis. All tests for significance and the resulting p-values were 2-sided, with a level of significance set at 5%. Statistical analyses were performed using JMP 9.0.2 (SAS Institute, Cary, NC, USA).

Results

The baseline characteristics, including FGM data and analyses of 75-g OGTTs, are shown in Table 1. Subjects had a mean age of 34 years; HbA1c and glycated albumin levels were 5.3% and 13.1%, respectively, indicating normal glucose tolerance. The MARD between FGM-SG and SMBG values was 10.9 ± 4.1% during OGTT; this result is consistent with previously reported data [3, 6, 12]. Surprisingly, the MARD between FGM-SG and PG values was 20.3 ± 10.3% during OGTT, demonstrating a discrepancy in the FreeStyle Libre when FGM-SG and PG values were compared. Additionally, the MARD between SMBG values and PG values during OGTT was high (17.4 ± 7.3; Table 1). Since the MARD of FGM-SG values seemed discrepant, we performed Parkes error grid analyses of SMBG and PG values with FGM and found that all data fell within zone A and B (Fig. 1A and B) [11]. We also compared the values recorded during OGTTs and found that FGM-SG and SMBG values had similar profiles that were significantly different from those of PG values (Fig. 1C and D). Since interstitial fluid glucose levels lagged behind PG levels [13], we analyzed the data considering delay. We calculated the differences between FGM-SG and PG to the nearest sampling time (0), 15 minutes before 0 (Δ – 15 min), and 15 minutes after 0 (Δ + 15 min). As shown in Fig. 1E, the time delay of FGM-SG and PG was less than 15 minutes in our experimental setting. To assess the factors that determined the MARD of FGM, subjects were divided into two groups based on MARDs of 15% between FGM-SG and PG values. BMI, FPG, homeostasis model assessment insulin resistance index (HOMA-R), average SG level (CGM), 75-g OGTT-area under the curve (AUC) of PG, and MARD between SMBG and PG values were significantly different, indicating that these factors determined the MARD of FGM (Table 1). In addition, multiple regression analysis of MARD between FGM-SG and PG values revealed a significantly higher contribution of BMI (p < 0.01) and FPG (p < 0.05) (Table 1). A 75-g OGTT load not only changed the glucose levels but had a significant effect on the blood and the intracellular and extracellular spaces. This could have affected the intracellular glucose reading (both FGM-SG and PG values) and limited the validity of this study. Therefore, we performed the same analyses for hematocrit (Ht), blood urea nitrogen (BUN), and creatinine (Cr). We found that these were not significantly different, indirectly indicating that these factors were not influential (Table 1). The relationships between MARDs and FBS, and HOMA-R and BMI were evaluated and are shown in Fig. 2A, B, and C. Higher BMI could have affected PG, and therefore, BMI and PG might be correlated. The correlation could have affected the entire analyses in this study; thus, in these cases, multivalued correlations adjusted for 2-h PG of 75 g-OGTT were also performed. Nevertheless, the relationships between the MARD and FBS, HOMA-R, and BMI remained statistically significant (Fig. 2A, B, and C). Moreover, the relationship between MARD and average sensor glucose or 75 g OGTT AUC of PG were not significant (data not shown). Interestingly, multiple regression analyses showed that BMI contributed the most to the MARD. It has been reported that MARDs depend on glucose levels, besides BMI [14]; therefore, the correlation between MARDs and glucose levels determined by FGM-SG and SMBG were further evaluated. MARDs were decreased until approximately 80–100 mg/dL and then seemed to level off (Fig. 2D and E). Since MARDs were divided by the glucose levels; therefore, the glucose levels could have affected it. In other words, individuals with higher glucose levels might show a discrepancy between MARDs and the mean absolute differences (MADs). Therefore, MADs were also evaluated, and similar patterns were found (Fig. 2F and G). More importantly, while MARDs and MADs were affected by the glucose levels, BMI did not correlate with HbA1c, FPG, and average sensor glucose levels. Indicating that BMI was an independent factor of MARDs, at least in our experimental setting (Fig. 3A–C). Finally, as shown in Fig. 3D, when we divided the patients according to the MARD, the BMIs of subjects with MARD <15% were significantly higher than those with MARD >15%. Indicating that subjects with higher BMI showed more reliable FGM values than those with a lower BMI in our study setting.

Table 1 Characteristics of the study participants and analyses of the factors used for determining MARD
MARD: FGM-SG vs. PG All patients
n = 25
20.3 ± 10.3
MARD <15
n = 10
10.5 ± 3.2
MARD >15
n = 15
26.8 ± 7.7
Univariate
p
Multivariate
p
Multiple regression analyses for MARD Libre vs. PG standardized partial regression coefficient p
Number of male patients (%) 17 (68) 8 (80) 9 (60)
Impaired glucose tolerance (%) 7 (28) 4 (40) 3 (20)
Age (years) 34 ± 6.5 33.9 ± 6.6 34.1 ± 6.7 0.89
Body Height 166.7 ± 10.1 167.5 ± 10.0 166.1 ± 10.4 0.83
Body Weight 65.0 ± 15.2 71.7 ± 14.6 60.6 ± 14.3 0.1
BMI 23.2 ± 3.8 25.5 ± 4.1 21.6 ± 2.8 0.03 <0.05 –0.45064 <0.01
Ht 43.1 ± 3.9 42.9 ± 3.5 43.2 ± 4.3 0.84
BUN 12 ± 2 12 ± 2 12 ± 3 0.63
Cr 0.72 ± 0.3 0.72 ± 0.3 0.72 ± 0.7 0.97
HbA1c 5.3 ± 0.3 5.4 ± 0.3 5.3 ± 0.4 0.33
GA 13.1 ± 1.1 12.7 ± 1.3 13.4 ± 0.9 0.13
1.5 AG 21.8 ± 6.2 21.8 ± 7.5 21.7 ± 5.5 0.64
FPG 87.2 ± 6.7 91.2 ± 7.5 84.5 ± 4.7 0.01 <0.05 –0.4133 <0.05
F-IRI 5.2 ± 2.8 6.6 ± 3.1 4.3 ± 2.4 0.06
HOMA-R 1.1 ± 0.6 1.5 ± 0.7 0.9 ± 0.5 0.04 <0.05
HOMA-beta 78.1 ± 37.3 88.2 ± 47.9 71.3 ± 28.0 0.42
Average sensor glucose 104.6 ± 9.0 98.4 ± 8.7 108.8 ± 6.7 <0.01 <0.05
STDEV (mg/dL) 22.3 ± 4.1 21.5 ± 4.6 22.9 ± 3.8 0.41
%CV 21.4 ± 4.1 21.9 ± 5.0 21.1 ± 3.6 0.96
MAGE (mg/dL) 56.3 ± 12.5 55.1 ± 15.6 57.1 ± 10.4 0.6
MODD (mg/dL) 18.7 ± 3.7 17.9 ± 4.2 19.2 ± 3.3 0.3
N 1,220 ± 180 1,185 ± 234 1,244 ± 138 0.79
Insulinogenic index 0.8 ± 0.6 0.8 ± 0.7 0.8 ± 0.6 0.85
75-g OGTT AUC SMBG 17,728 ± 2,346 17,990 ± 3,127 17,541 ± 1,697 0.93
75-g OGTT AUC FGM-SG 17,497 ± 2,499 16,701 ± 2,822 18,009 ± 2,225 0.33
75-g OGTT AUC PG 15,463 ± 2,079 16,985 ± 1,861 14,448 ± 1,556 <0.01 <0.05 –0.21068 0.23
MARD: FGM-SG vs. SMBG 10.9 ± 4.1 11.5 ± 5.1 10.5 ± 3.3 0.93
MARD: SMBG vs. PG 17.4 ± 7.3 11.2 ± 6.3 21.5 ± 4.4 <0.01 <0.05

BMI, body mass index; Ht, hematocrit; BUN, blood urea nitrogen; Cr, Creatinine; HbA1c, hemoglobin A1c; GA, glycated albumin; 1.5 AG, 1,5-Anhydro-d-glucitol; FPG, fasting plasma glucose; F-IRI, fasting immunoreactive insulin; HOMA-R, homeostasis model assessment insulin resistance index; HOMA-beta, homeostasis model assessment beta-cell function; %CV, percent coefficient of variation; MAGE, mean amplitude of glycemic excursion; N, Number of measurements during CGM; MODD, mean of daily differences; OGTT, oral glucose tolerance test; AUC, area under the curve; FGM-SG; sensor glucose values of FGM; PG, plasma glucose; MARD, mean absolute relative difference; SMBG, self-monitoring of blood glucose.

Fig. 1

Parkes error grid analyses of FGM-SG for SMBG (A) and PG (B) values. Glucose measurement by FGM-SG, SMBG, and PG during 75-g OGTT(C) and 100-g OGTT (D). Differences between FGM-SG and PG to the nearest sampling time (0), 15 minutes before 0 (Δ – 15 min), and 15 minutes after 0 (Δ + 15 min) (E). n = 25, Abbreviations: FGM-SG, sensor glucose values of flash glucose monitoring; PG, plasma glucose; SMBG, self-monitoring of blood glucose; OGTT, oral glucose tolerance test.

Fig. 2

Correlations adjusted for 2-h PG of 75-g OGTT between the MARD in FGM-SG for PG values and FBS (A), HOMA-R (B) and BMI (C). Correlation of MARDs between FGM-SG and PG to PG levels: below 100 mg/dL and above 100 mg/dL (D) and between SMBG and PG to PG levels (E). Correlation of MADs between FGM-SG and PG to PG levels (F) and between SMBG and PG to PG levels (G). Smoothing was performed by the LOESS method. n = 25, Abbreviations: FGM, flash glucose monitoring system; PG, plasma glucose; SMBG, self-monitoring of blood glucose; OGTTs, oral glucose tolerance tests; MARD, mean absolute relative difference; FPG, fasting plasma glucose; HOMA-R, homeostasis model assessment insulin resistance index; BMI, body mass index; MAD mean absolute difference.

Fig. 3

Correlations between BMI and HbA1c (A), FPG (B) and average sensor glucose values (C). BMI in the range of MARD for PG <15 and >15 with FGM-SG. (D). n = 25, Abbreviations: FGM, flash glucose monitoring system; PG, plasma glucose; MARD, mean absolute relative difference; FPG, fasting plasma glucose; BMI, body mass index.

Discussion

Our data demonstrated that MARDs were accurate when FGM-SG values were compared with SMBG values but not with PG values (Table 1). Moreover, Parkes error grid analyses of FGM-SG values revealed that all datasets fell within zones A and B (Fig. 1A and B). The MARD in FGM-SG with reference to PG values correlates with FBS, HOMA-R, and BMI. Multiple regression analyses showed that BMI contributed the most to MARD determination. Most notably, FGM produced more reliable values during OGTT in subjects with a higher BMI, with no possible correlation with PG in healthy patients.

Parkes error grid analysis is a method used to determine the accuracies of CGM sensor glucose (CGM-SG) values, typically using SMBG values as a reference. Our data supported not only previous results that FGM-SG values are reliable with reference to SMBG values [12] but also the fact that they are also reliable with reference to PG values. However, our data revealed that FGM-SG values were significantly different from PG values, since the MARDs in FGM-SG with reference to PG values were very high during the 75-g OGTT (Table 1 and Fig. 1C). These data indicated that under certain circumstances, FGM-SG might be unreliable. Moreover, we should re-consider the MARD between SMBG and PG values, since our data demonstrated that it was as high as that between FGM-SG and PG values. We routinely use SMBG values as a reference for determining MARD and tend to believe it to be accurate CGM. In fact, a recent paper showed that SMBG is not very accurate and few machines meet the ISO15197-2013 criteria [5].

While low FPG has been reported to affect MARD between FGM-SG and SMBG [3, 6, 8, 12], reasons underlying the effect of BMI on MARD in FGM-SG values were not identified. While it has been reported that MARDs are not affected by BMI in patients with a relatively high BMI [13], a recent report showed that MARDs can be affected by BMI in certain circumstances [15]. Additionally, while a racial component (the relatively low weight of Asian individuals) may have affected our analyses, sensor stability may be higher in patients with high BMIs. It has been reported that the accuracy of FGM depends on the location of the sensor at insertion and thus, sensor stability, which may increase in locations rich in subcutaneous tissue, is very important [16].

Several limitations should be considered while interpreting our findings. This study had a small sample size and included healthy volunteers with characteristics different from those of patients with diabetes. It has been shown that the MARD between CGM-SG and SMBG values were based on study conditions, and differs between the at-home and the clinical research setting values [10]. In this study, the subjects did not have diabetes and had less glucose variability. More importantly, OGTTs were performed under clinical research laboratory conditions. Therefore, the MARD during OGTT might not be consistent with that seen during the course of the day in patients with diabetes. Moreover, the Food and Drug Administration (FDA) recommendations for the evaluation of blood glucose meter accuracy emphasize the need to evaluate over the widest possible value ranges. The present analysis failed to meet this criterion a priori because we recruited healthy volunteers. Use of OGTTs in patients with diabetes is not permitted in Japan, and as the goal of this study was to assess the factors that determine MARD of FGM, relatively uniform conditions for glucose measurement are preferred. As a result, our study was performed under standard and consistent conditions with respect to patients, location, temperature, and humidity such that the SMBG and PG values were less influenced [10]. While MARD, in the relatively “ambient” conditions of our study might be overcome by strong factors, such as patients with diabetes, the factors determining MARD analyzed in our study should be considered, and at least might reflect on MARD in a normal glucose range or in patients without glucose excursions. Although MARD is one of the indicators of the accuracy of CGM systems [17], it is not always reliable [10, 18]. More importantly, many of the previous articles explaining the definition and meaning of MARD suggested that MARD is an indicator of accuracy. However, MARD can be affected not only by the accuracy but also by the precision, because MARD may be larger with high accuracy but low precision measurements [17, 19]. Finally, Comparing the performance of CGM systems revealed that MARD estimation is more complex than expected [10, 18, 20, 21]. Therefore, FGM and other CGM systems, such as Dexcom, should be compared under the same conditions as in our study, to examine their accuracy.

In summary, our investigation showed that, under some circumstances, FGM-SG values differ from PG values but not from SMBG values; though acceptable by Parkes error grid analysis. Moreover, the MARD in FGM values is affected by BMI. FGM use has spread widely but few criteria have been developed for maintaining its accuracy. In fact, since FGM as well as CGM have rapidly developed and are marketed in only about 50 countries, the data will never be comprehensive for patients with diabetes and will require regular updating [22]. Inaccurate FGM data could potentially increase the risk of inappropriate treatment. Therefore, such analyses could be one way to help guide the care of patients, using FGM. Future studies with larger study populations and patients with diabetes exposed to different conditions (such as the eating environments as well as exercise) are necessary to validate these results.

Abbreviations

%CV, percent coefficient of variation; AUC, area under the curve; BMI, body mass index; BUN, blood urea nitrogen; CGM, continuous glucose monitoring; CGM-SG, continuous glucose monitoring-sensor glucose; Cr, creatinine; FGM, flash glucose monitoring system; FGM-SG, flash glucose monitoring-sensor glucose; F-IRI, fasting immunoreactive insulin; FPG, fasting plasma glucose; HOMA-R, homeostasis model assessment insulin resistance index; Ht, hematocrit; MAD, mean absolute difference; MAGE, mean amplitude of glycemic excursion; MARD, mean absolute relative difference; OGTTs, oral glucose tolerance tests; PG, plasma glucose; QOL, quality of life; SMBG, self-monitoring of blood glucose

Acknowledgements

Not Applicable.

Disclosure

Funding information

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of interests

None of the authors have any potential conflicts of interest associated with this research.

References
 
© The Japan Endocrine Society
feedback
Top