Technology has played a significant role in diabetes care for more than 50 years, from the advent of the original insulin pump to more recent artificial pancreas technologies. In healthcare more broadly, the use of technology through digital health has gained traction over the last 20 years, with interest expanding to mobile wireless technology, or mHealth.1 mHealth is defined as the use of mobile and wireless technologies to support health, and includes apps, text messaging systems, and web portals.2
With over 5 billion unique mobile subscribers worldwide, mHealth holds significant potential to expand access to healthcare to those for whom it may traditionally have been limited, due to either lack of access to care or low socioeconomic status. Diabetes, which is largely a patient-managed condition with growing prevalence in underserved communities, is an ideal candidate for mHealth integration.3,4 The role of technology in insulin delivery and glucose monitoring technologies is well documented, but the same is not necessarily true for mHealth.5 With an ever-expanding number of apps available, it is challenging for patients to identify which apps to use and for providers to be able to give evidence-based recommendations. A study of German- and English-speaking patients with diabetes, who were recruited from online diabetes communities, found that MySugr, Dexcom and MyFitnessPal were the most frequently used apps in 2019. In patients using these and other apps, there was a higher level of diabetes self-care, which included following a diet, physical activity, and blood-glucose monitoring.6 Despite this reported benefit, research into mHealth has lagged, with limited high-quality recommendations available to guide both consumers (patients) and care providers.
Safety, efficacy, and effectiveness of interventions have historically been the primary outcomes of interest in diabetes research. These clinical outcomes have remained the cornerstone of research, particularly with respect to glycemic control and the oft-chosen outcome of improvement in glycated hemoglobin (HbA1c). However, we propose that historically valuable clinical outcomes may not be the best measures of mHealth interventions, and alternate endpoints should be considered that evaluate technology from the patient viewpoint as a complement to clinical outcomes.
Clinical outcomes
Fingerstick blood glucose and HbA1c have long been the standard for clinical diabetes outcomes measures. This is largely as a result of data demonstrating an association with reduction in microvascular diabetes complications with reductions in these measures.7,8 While self-monitoring of blood glucose is well-established, the degree to which the information obtained is valuable is limited by the frequency and the timing of the specific blood sugar checks and may not accurately reflect overall trends, nor important nadir or peak glucose values. This is of particular concern as many studies focus on glucose data to assess efficacy, while this may not be appropriate for mHealth interventions such as meal tracking apps, where the app–user interface and usability is arguably more important and glucose data are less relevant as a primary outcome.
Even the gold standard HbA1c may not necessarily provide sufficient information to optimally assess intervention efficacy in mHealth studies. Wide variations in daily behaviors, interactivity with mHealth platforms, and blood sugar values can, on average, appear well-controlled based on an accepted HbA1c definition of <7%. Additionally, HbA1c may be inaccurate in individuals with a variety of medical conditions, who are often excluded from studies for this reason.9 The long duration of time required to see an effect on HbA1c can also be challenging when assessing mHealth interventions, where feedback is nearly instantaneous and iterative product development is the norm. Three months may be enough time for a new version of an app to become available, making the results obsolete.
Technological advances in diabetes, specifically the advent of real-time and intermittently-scanned continuous glucose monitoring (CGM) have allowed for improved characterization of glycemic variability through multiple measures and a new set of outcomes, including time in range and time spent in hypoglycemia. These technologies provide near continuous glucose values, allowing for a richer data set for analysis9,10 and most CGM devices now communicate directly with mobile devices and apps. Time in range can be adjusted to different targets, so it can be better individualized for groups in which the glycemic targets may not be as strict, such as elderly individuals or those with multiple medical comorbidities. Additionally, the time spent in hypoglycemia is an important safety consideration and identifying previously unidentified hypoglycemia can allow for specific interventions to reduce hypoglycemia. Glycemic variability is an increasingly important factor that is clearly associated with hypoglycemia and quality of life that has become available through more widespread incorporation of CGM into clinical care. Various methods of defining glycemic variability are available including, standard deviation, coefficient of variation, and interquartile glucose ranges. Although the benefits of these diabetes technologies are well-described, their integration with mHealth is still in the early stages and more studies and innovation are needed to determine how best to assess for potential benefits of the mHealth component itself.5
Patient-reported outcomes
Historically, patient-reported outcomes (PROs) have been notably excluded from clinical trials in diabetes care.11 PROs are defined as reports of health status, quality of life, or functional status that are directly reported by a patient.12 These outcomes are centrally important to those living with diabetes and provide important complementary data to guide clinical decision making and regulatory approvals. Given the increasing focus on patient-centered care and research, PROs are increasingly being included in clinical trials and have recently been integrated into evaluation and labeling of medical devices by the US Food and Drug Administration.11,13,14 However, there has yet to be agreement regarding which PROs to measure and how best to measure them. Additionally, mHealth is not considered to be a traditional medical device, so regulation surrounding mHealth does not necessarily follow the same standards as traditional diabetes technology.
At a national level, funding sources and societal groups, such as the Patient-Centered Outcomes Research Institute, National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK), and the American Diabetes Association (ADA) have called for increased attention to PROs through funding opportunities directed at improving PRO measures and determining how best to incorporate them into clinical care. Furthermore, the NIDDK and ADA convened a workshop of experts in 2017 to develop consensus and move towards increased use of standardized, validated, generalizable PRO measures both in research and clinical practice.15
With little data currently available surrounding diabetes mHealth, the incorporation of PROs into this research remains a limited, but essential, area for development. PROs are of particular importance for mHealth, as these mobile tools have evolved out of a movement to transition towards highly accessible, patient-centered care delivery. However, high rates of technology and mHealth application abandonment exist, and this must be overcome to make mHealth interventions useful.16 A recent retrospective study of real-world observations in patients using the MySugr app by Debong et al. included both clinical outcomes and patient satisfaction as end points, an important shift towards recognizing the need for both in study design.17
When evaluating mHealth, user experience in an important PRO, in addition to traditional measures such as patient burden and resultant quality of life measures. User experience is a broad outcome that encompasses the usability, accessibility, usefulness, desirability, navigability, and credibility of a technology.18 Usability has been defined as the “extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.”19 Despite an expanding market with over 325,000 mHealth applications, usability is, unfortunately, frequently overlooked and scant data exists regarding usability in regards to diabetes mHealth.20 Fu et al. reported that an increase in app usability was associated with higher competence and autonomy in diabetes management, along with improved connectivity with healthcare providers, suggesting that focusing on these aspects of care may improve success with mHealth interventions.21 Usability is critical to the success and sustainability of any direct patient-facing health technology and ultimately, if a technology is found to have poor usability for the patient, the provider, or the larger health system, other outcomes are irrelevant.
Conclusions
As the mHealth market continues to rapidly expand, it is important to step back and consider the tools and metrics we employ to critically evaluate these technologies. mHealth is openly accessible to patients and not subject to strict regulatory oversight, and additionally is rapidly evolving. This becomes increasingly complicated as mHealth continues to directly integrate with traditional diabetes technologies. It is crucial that our clinical outcomes keep pace with technological advancements, but should not necessarily take precedent over PROs, and instead be used in conjunction with each other to fully assess an intervention. As mHealth is increasingly intended to facilitate access to care, in addition to traditional PROs such as burden, distress, and quality of life, the importance of user experience in technology and mHealth should not be overlooked as it plays a crucial role in their sustainability and effectiveness.
