Wearing Your Health on Your Sleeve
How big data from mobile apps and sensors may revolutionize healthcare
My phone buzzes. It’s Mood Matters, a mood-tracking app developed by the startup Ginger.io. “We notice you haven’t logged any recent physical activity,” it alerts me, linking to an article about the connection between depression and exercise. I glance at the band on my wrist, a Fitbit fitness tracker that’s unrelated to the app, and see that I’ve only walked a measly 800 steps today. I scroll over to see my heart rate—at least I’m relaxed, I think. I scan my to-do list and then stand up for a quick walk around the block. Each step I take, eventually, is relayed to the cloud and stored as a bit of information in a data center with all the other steps people are taking around the world, forming a massive data set describing when and how we move.
Right now, that data set—the vast amount of information already collected by mobile health devices—is mostly looked at through the lens of very basic statistics to answer questions of curiosity. How many steps do American Fitbit owners walk, on average? Which occupations are most active? Whose heart rates spiked during the Super Bowl?
There’s a vision for the future, though, that is far more complex.
In this future, devices on our wrists, in our phones, or tucked in our pockets are more than step-count monitors. They track all aspects of people’s health and act as part-counselor, part-physician, part-coach, alerting us to health concerns or spurring us to make lifestyle changes. Patients could be alerted if they have signs of impending heart failure, worsening Parkinson’s disease, or a low blood sugar crash—among many other things. Moreover, as devices funnel increasingly large amounts of information to the cloud, they give scientists a rich and ever-changing platform to use for research—letting them make new connections between facets of people’s behavior and health that have never been linked before.
Many of the arguments in favor of the increasing use of mobile devices to monitor patients center around preventive medicine—the idea that many chronic diseases such as diabetes and heart disease can be prevented by changing people’s diets or exercise patterns, saving healthcare systems vast amounts of money.
“It’s easy to give a pill, it’s straightforward to do a procedure, but to change patients’ behavior is the holy grail in medicine,” says Alan Yeung, MD, La Ka Shing Professor of Medicine at Stanford School of Medicine. “Phones, together with wearable devices, can for the first time provide some objective evidence of behavior.” And once researchers understand what influences behavior they can set to work changing it.
“The ultimate goal is to improve health outcomes for people,” says Ray Browning, PhD, associate professor of health and exercise science at Colorado State University. “We have a lot of chronic disease in this country that’s preventable with changes to behavior.”
Today, mobile fitness devices are exploding in popularity, but we’re only a small way toward that vision coming true. “We’re at the very beginning of mobile devices starting to impact how medicine is being practiced,” says Eric Topol, MD, a Scripps Research Institute cardiologist who has written extensively about the technological future of medicine. He thinks the slope toward full adoption likely has a hockey stick shape. “We’re starting to get closer to that rapid rise.”
Interviews with a handful of researchers and companies who are pushing the field forward suggest that, while many of them are making lofty promises for the future, challenges remain: showing the clinical utility of devices and apps through not just anecdotes but well-designed clinical trials; getting both doctors and patients alike to buy into the use of the devices; and developing new computational methods to parse the steady stream of data from the mobile device fire hose.
In the decades-off vision of wearable health trackers, devices on our wrists or in our pockets can collect all sorts of data about our breathing and our eating and markers in our blood. But for now, the majority of wearable medical devices are fitness trackers that rely most heavily on one piece of data: our movement.
“The lower-hanging fruit in the field right now is physical activity data because it’s so ubiquitous,” says Ida Sim, MD, a co-director of the Biomedical Informatics Institute at the University of California, San Francisco, as well as an investigator with Mobile Sensor Data-to-Knowledge (MD2K), an NIH-funded Big Data to Knowledge (BD2K) Center of Excellence. “The thing being measured is a physical quantity and it’s pretty easy to represent and to calibrate between devices.”
Devices like the Fitbit, or numerous phone apps, rely on accelerometers to tell users how much they’ve walked each day. Tiny crystal structures embedded in these devices sense movement—as your arm swings or your body moves up and down in a tell-tale walking pattern—by detecting changes to the direction they’re pulled by gravity. Then, they transmit a voltage relaying this information. It’s a relatively simple and cheap technology these days—the accelerometer in the latest iPhone has an estimated cost of 65 cents and the gyroscope (a similar technology that detects the tilt of the phone) costs less than two dollars.
But—at least until now—that step count has rarely been linked to real health advice beyond the idea that more activity can help you lose weight and lower your overall odds of a plethora of obesity-related chronic disease. Now, though, that’s starting to change. Researchers are now seeking meaningful associations in the data deluge from early adopters, with the ultimate goal of showing that their app or device of choice can make people healthier or less likely to develop conditions like heart disease.
Yeung, with colleagues at Stanford, has developed MyHeart Counts, one of the apps that Apple is touting as the future of its smartphone and smartwatch-based technologies. When you download MyHeart Counts—as roughly 26,000 people did in the 24 hours after Apple’s latest press conference on its new open-source ResearchKit—you’re asked a series of questions on your lifestyle and family history of heart disease. Then, using the iPhone’s movement sensors, the app records your every motion for seven days.
“Using the gyroscope and GPS on the phone, you can easily tell whether someone is just sitting around or whether they’re being active,” says Yeung. If someone has time, they can also take a six-minute walk test—their phone records how far they are able to walk in that time period. It’s a classic test that’s been used by cardiologists for years, but usually relies on a stopwatch and marks along the edge of a long hospital corridor. With the phone app, users can perform the test anywhere, and at the end, MyHeart Counts provides each user with a calculated “Heart Age.” “A person signing up might be fifty years old, but we might calculate that their heart health is more like that of a sixty year old,” Yeung says. Again, this computation is not new—it is based on existing data published by the American Heart Association—but the app makes it more accessible and understandable to the average person. Yeung hopes it will motivate people to set a goal of lowering their heart age.
Like many other devices and apps, though, the challenge with MyHeart Counts is proving any sort of clinical utility. Generally, to show that patients benefit from an intervention (whether it’s a drug, a counseling session, or a device), researchers have to show that people who use it have less disease—or longer lives—than those who don’t.
For now, Yeung and his colleague are tracking whether those who continuously use the app—and get reminders to exercise—see a decrease in their computed heart age. Eventually, they’d like to test out the app in patients who are at higher risk of heart attacks, for example those who have diabetes or have had heart bypass operations. Could an app tracking their behavior signal which of these patients need doctors’ visits or new medication strategies?
“If we evolve the app to be more clinically relevant, we’d like to give it to every heart patient at Stanford,” Yeung says.
The question remains whether reminders on an app or device are sufficient to motivate behavioral change, says Browning, who collaborates with Stanford’s Mobilize Center on a project that will apply new analytical techniques to movement data from devices and apps.
“I’ve never heard of a person who says, ‘The reason I don’t have a more active lifestyle is because I don’t have a wearable activity monitor to tell me how active I am,’” Browning points out.
He says that it will take massive public health campaigns—ones he likens to anti-smoking campaigns—to get activity on people’s radar. Until then, the majority of the population that already doesn’t spend much time exercising isn’t about to pick up a fitness tracker and start working out, he says.
Inside Your Head
If tracking movement with the aim of preventing obesity, diabetes, and heart failure is the low-hanging fruit in mobile medicine, then tracking movement with the aim of detecting downward spirals in depression is the next branch up. People who are depressed are less likely than usual to call and text friends, more likely to stay home, and less likely to exercise. And these are all things that can be detected by a smartphone’s call logs, text message records, and gyroscope using an app such as Mood Matters from Ginger.io.
“Of course everybody deals with depression slightly differently,” says Joe Grimberg, head of marketing at Ginger.io. “But clinically we know that social isolation and physical lethargy are markers of depression.”
By giving clinicians access to day-to-day trends in patients’ activity levels, the app is designed to detect when caregivers should intervene to help patients who have depression, as well as guide psychiatrists’ conversations with those patients, Grimberg says.
They’ve partnered with psychiatrists at the University of California, Duke, and other medical research centers to test out what happens when clinicians can see the daily behavior patterns of their patients. Each institution has launched slightly different trials using the app, targeting different patient populations. Those pilot studies are ongoing. At the University of California, San Francisco, teams of psychiatrists and nurses are interacting with hundreds of depressed people who were recruited online, treating them and following their moods with no face-to-face visits. Instead, doctors and nurses receive alerts if certain behaviors are flagged—if a patient reports that they’re hearing voices, or feeling suicidal for instance.
For now, the flagged behaviors that Ginger.io uses to alert a medical team that it’s time to call a patient are based on previous knowledge of psychiatric disorders. Eventually, though, Ginger.io could use their data to find new “flags” that inform clinicians of how their patients are doing. So far, most of the data collected by Ginger.io—showing that the app can, in fact, gauge depression severity—has been published in the form of patent filings or presented at meetings. The Ginger.io team and their collaborators, however, are aiming to publish more recent studies on the effectiveness of interventions in peer-reviewed journals. In pilot studies, Grimberg says, they’ve gotten overwhelmingly positive feedback from both patients and providers.
“The great thing about the smart phone as a medical device is that it’s already a part of most people’s daily life,” Grimberg says. “Ninety percent of smart phone users have their phone within three feet of them all day.”
And an app like Mood Matters is more accurate at capturing a patient’s mood than a questionnaire in a psychologist’s office, the company has found—not only because of its activity meter, but because of its built-in questions and journaling function, which give more frequent insight into a patient’s feelings compared with reports gleaned at occasional in-person appointments.
Data from All Sources
Alex Markowetz, PhD, a computer scientist at the Universität Bonn in Germany, says that it’s not just smartphones that hold the power to reveal someone’s behavioral trends—it’s all the computers we interact with every day.
“Theoretically, we can track any human-machine interactions,” Markowetz says. “A smartphone works, but so does your email client, or World of Warcraft, or Skype, or your car or your fridge.”
Markowetz’s group has already developed an app called Menthal Addicted that tracks people’s smartphone usage, hoping to pinpoint factors that make people more prone to becoming addicted to technology. Now, they’re teaming up with clinicians to make Menthal Depressed, which, like Ginger.io, uses a person’s phone to detect depression; Menthal Skilled, which requires users to complete a simple cognitive puzzle to unlock their phone and could pinpoint the earliest signs of dementia; and Menthal Dopa, which uses a phone’s accelerometer and gyroscope to detect hand tremors and track the severity of Parkinson’s disease over time.
All the Menthal projects, Markowetz says, are geared around the idea of getting more constant data about a patient’s life to inform a clinician.
Right now, the typical clinican has two ways to get information about a patient: a questionnaire or an office visit. But these isolated data points present a tiny fragmented view of mental health, Markowetz says. “They are just poor ways to steer medication or therapy.”
To develop each Menthal app, Markowetz closely analyzes how skilled doctors assess the symptoms of patients they see. He then tries to recreate that using sensors and the right computational analysis.
“A doctor might say, ‘I noticed that a patient was slouching and looking downward and that made me think they were depressed’,” Markowetz says. “So then I say, ‘okay what sensors can we use to determine someone’s posture and the direction of their gaze?’”
But finding such sensors isn’t necessarily easy. For example, there’s no device to measure slouching right now, although Markowetz suggests that a head-mounted device like Google Glass might work. In any event, it can take many iterations of programming and analysis to fine-tune the data signature that points toward a behavior—differentiating slouching from nodding, or eating from drinking, for instance.
And making technology applicable to healthcare requires more than the right sensors and computational methods, Markowetz adds. Sensors need to be unobtrusive and require little buy-in from the patients. If you ask someone to fill out a questionnaire on their smart phone every day 365 days a year, they’re going to get tired of it quickly. Likewise, if you ask someone to wear a soap-bar-sized sensor on their forehead while they sleep, it likely won’t last long. It will take small devices that require no input from users to truly make mobile medicine applicable to the whole population, Markowetz says.
The technology to detect a person’s steps may now be ubiquitous, but to make mobile medicine more broadly useful—beyond the prevention of obesity and depression—will take not only the most unobtrusive sensors, but sensors that can detect physiology and measures of health beyond movement.
About five years ago, Emre Ertin, PhD, an electrical engineer at Ohio State University who is also part of MD2K, developed AutoSense, a sensor suite that contains a one by two inch sensor that is worn on a person’s chest. It is designed to track levels of stress by measuring the electrical activity of the heart as well as a person’s breathing rate, temperature, and movement. As it records the data, AutoSense streams it to a cell phone.
In 2012, Ertin’s research team turned to developing a stress sensor that works wirelessly without touching the skin. The device they came up with could fit in a pocket and used radio waves to sense a person’s heart and lung motion. But there was a problem: The sensor was too sensitive to changes in water content. “Wireless waves don’t move well through water,” Ertin says. For measuring stress, this was an annoyance: The team had to figure out how to make the sensor give consistent readings even if the body’s liquid levels changed. “But then we got an idea,” Ertin says. “Maybe we could use the sensor to monitor lung fluid levels in congestive heart failure patients.”
Almost a quarter of patients hospitalized for congestive heart failure are rehospitalized within a month, and more than half within six months; it’s a number that doctors are always trying to lower. But it’s tough to predict which patients will have recurring problems. A sensor monitoring fluid in the lungs (a telltale sign of heart failure) could help pinpoint these patients early.
As part of MD2K, Ertin’s group is collaborating with clinicians to use the wireless monitors to track patients with chronic obstructive pulmonary disease (COPD). Like heart failure, COPD is characterized by fluid in the lungs.
Some at MD2K, using sensors similar to those Ertin has developed, are working on detecting when cigarette users smoke. “There’s a very particular gesture and breathing pattern that goes with smoking,” Ertin says. MD2K-affiliated researchers at the University of Massachusetts developed RisQ, which uses a wristband to detect smoking behavior with 95.7 percent accuracy. Such data might provide a patient (and his or her doctor) with hints as to when they’re most likely to pick up a cigarette, he says, helping tailor interventions that work.
Many challenges remain. At the 2014 International Conference on Information Processing in Sensor Networks, for example, Santosh Kumar, PhD, of the University of Memphis—a long-time collaborator of Ertin and the director of the MD2K center—presented data on attempting to use his sensors to detect when drug addicts took a dose of cocaine. But training a device to recognize cocaine use wasn’t quite as easy as smoking. A person’s heart rate, blood pressure, and breathing patterns change when they use cocaine. But they also change for any number of other reasons—exercise, fear, stress, and other drugs for instance. The data Kumar’s team collected—922 total days (over 22,000 hours) of data from drug users—was incredibly noisy. And they had to figure out how to clean it up enough to find meaningful trends without losing the signal of the cocaine use. It took multiple iterations of modeling, statistical analysis, and data processing to get the data to this point—and in order to detect all cocaine events, the researchers still had a false positive rate of 1 per day. “In conclusion, detection of cocaine use from physiological measurements collected in the field setting is challenging,” Kumar and Ertin, together with the student (lead) authors, wrote in the paper describing the work.
Sifting Through the Data Dump
For now, most mobile medical devices are designed to measure a piece of data that doctors already know they want—and has already been correlated to clinical outcome. Doctors already knew that lung fluid helps predict congestive heart failure, for instance; now they just have a new way to track that. And psychologists already knew that less active patients are more likely to be depressed.
But one promise of mobile medicine—at least for basic researchers—is that the vast amounts of data being collected can help reveal connections that clinicians don’t yet know about. It’s mobile medicine as a discovery tool.
“There’s definitely this thought that because we have all this data, eventually we’ll learn something from it,” says Sim.
It’s what online giants like Facebook and Google have already been able to do in other realms by tracking the online activity of people—discover what makes one person more likely to click an ad for a restaurant and someone else more likely to click an ad for a movie. Doctors want to do the same thing for healthcare: Ask what it is about a person’s data footprint that can predict whether they’ll get sick or whether they’ll respond to a treatment.
“We can take all of these data points and find trends in them,” says Joe Kvedar of Harvard Medical School and the Center for Connected Health. “It’s what other industries have already done with complex, ever-changing data feeds.”
For now, though, that data pool just needs to grow. Is there such a thing as “too much data?” Most of the researchers interviewed for this story answered that question with a resounding no.
“There’s certainly no such thing as too much data right now,” says Markowetz. “That’s the paradigm shift that’s happening. Right now, you collect data because you can and because data storage is cheap. And then at some point later, you can say ‘I have all this data, I wonder if it can help me answer a new question?’”
As long as researchers can keep up with managing the data, Topol says, there will always be a place for it. “There is this concern about TMI, too much information,” says Topol. “And that is something that can be preempted by really great algorithms and analytics that filter out the signal from the noise and get the critical elements out of the data.”
But sufficient data for researchers, Sim points out, might be too much for consumers or their doctors. “For individual patients and clinicians, there most definitely is a problem of too much data,” she says. “We’re almost there already.” One key to moving forward, she suggests, is targeting the right data pools to the right people—clinicians might not need to see everything that researchers see, for instance.
A Common Language
As the data pool grows, another challenge emerges: the need for data from different devices to be standardized and compared.
When Sim orders a potassium level on a patient, for example, she doesn’t care whether GE or Siemens produced the machine that measures it. “The data has to be device-agnostic,” she says. “I can’t be dealing with what machine the number came from.” Likewise, if she wants to get information on a patient’s daily activity for the past month, Sim doesn’t want the information from a Fitbit to be presented to her differently than the information from an iPhone.
The priority for tech companies, Sim points out, is to find their niche, market their product, and keep their data proprietary. But for researchers and clinicians, data that’s in different forms depending on where it’s from isn’t useful.
“All these people who are trying to get data from multiple sources, and across heterogeneous platforms are starting to see the value of standardization,” says Sim, who is a co-founder and a principal scientist of the non-profit Open mHealth, which aims to design an open, common language for health data.
Open mHealth has already outlined an initial set of suggestions for standardizing such mobile sensor data, and Sim says they’ve garnered some interest from tech companies who want to learn more.
“People see the need for an open standard,” she says. “We’re only three years in and we’re already getting traction. We’ll be rolling out more this year."
The Tech World Meets Healthcare
Many of the mobile medical devices discussed here promise to give individuals more power over their own health. But mobile medicine also promises to ease the burden on our healthcare system by giving doctors tools to be more efficient, helping clinicians make the move toward truly personalized medicine, and providing platforms for clinical trials. But for this to happen, doctors and nurses will have to buy in to the power of wearable sensors.
“Doctors are not particularly enamored by many of these tools because it’s a challenge to their control which has been in existence since the beginning of the profession,” says Topol.
Changing their minds will require technology that’s easy to use. “If you go into a doctor’s office with a huge stream of EKG data that’s been collected 24 hours a day, there’s not much they can do with it,” says Ertin. Software is needed to summarize these huge streams of data into a form that clinicians will find useful.
Yeung, as he moves forward with MyHeart Counts, imagines an algorithm that summarizes how each patient’s heart health is doing—a green light means they’re doing well, a yellow light might warrant a call from a nurse, and a red light would indicate the need for a virtual doctor’s visit to discuss changing medications or approaches.
“This kind of system needs to be created to take the workload away from the physician,” Yeung explains. “I could easily be following ten thousand or more patients anywhere in the world and only seeing the ones that need to be seen, rather than following fewer patients and scheduling frequent visits with all of them.”
For the Menthal apps, Markowetz always aims to get a single number that captures a patient’s status. “What if I had a single number that’s your Parkinson’s severity number per day?” he asks. “Now, I can chart this over the past six months, and that’s something that really tells a doctor how you’ve been.”
One day, healthcare may move this way. Rather than wait for your annual exam to have a doctor test the status of your health, you get daily updates, reminders, and notifications about your own body and behavior. When it’s needed, you’ll be alerted to contact your doctor who will have a plethora of information at his or her fingertips to diagnose and treat you.
“This is going to be a whole rebooting of how medicine will be practiced to benefit the consumer,” Topol says.
An App a Day Keeps the Doctor Away
This spring, consumer electronics giant Apple went from being an intermediary in the mobile health market—their iPhones could track steps or host third-party fitness apps—to being a major stakeholder. Apple’s jump into the big time could mean good things for those who want to see mobile health go mainstream.
“Personally, I’m excited to see the Apples and Googles and Samsungs of the world take on preventive healthcare,” says Ray Browning of Colorado State University. “You’re talking about a lot of horsepower all of a sudden being thrown at these problems. And these companies are historically highly successful at changing behaviors.”
In March, Apple unveiled ResearchKit, which aims to transform the way data for clinical trials is collected. ResearchKit launched with five apps—including MyHeart Counts which tracks cardiovascular disease (see main story). Other apps log patients’ symptoms of diabetes, Parkinson’s disease, asthma, and breast cancer. But the power behind ResearchKit is that the data isn’t just for patients’ own curiosity—it’s linked to research programs that want to use the data to answer questions about diseases. The Wall Street Journal reported that in the first month of availability, more than 60,000 patients signed up for the apps—essentially volunteering themselves as clinical subjects.
Then, in April, Apple announced a second major collaboration: a partnership between Apple, IBM’s new Watson Health Cloud, Johnson & Johnson, and Medtronic. Together, the companies are launching an effort to tailor data storage and data analytics to clinicians. The data they try to harness will be drawn not only from ResearchKit, but from Apple’s earlier HealthKit, the basic suite of fitness apps that comes pre-installed on iPhones.
Already, a survey by Reuters news agency found that more than half of the top US hospitals have rolled out pilot programs using HealthKit. With patients’ consent, doctors can view data collected by their smartphones and add it to electronic medical records. Google and Samsung have also launched collaborations with hospitals and medical record providers to pair their software more closely with clinicians. Like all mobile health efforts, however, it remains to be seen whether buy-in from these major tech companies can not only streamline the way data is collected, but change patient behaviors.