App scans user’s face, monitors voice for signs of anxiety, depression

One of the app’s main features uses AI to scan pill bottles. It then gathers the information to create a health profile about the user.

The app can send users information about their medication to remind them when to take their next dose.

“It sends you insights based on the medicines you take, for example, if you’re taking Ozempic It knows that you might be overweight or you might have diabetes,” explained the app’s founder Dr. Renee Dua. “It will remind you to screen your urine to make sure there’s no protein or screen your eyes or your feet to make sure you’re not suffering from complications of diabetes. It does a lot to tell you about yourself with minimal effort.”

Dua has worked in healthcare in California for over 12 years. She said she created the app based on her own experience taking care of her aging parents.

“Both of my parents suffer from a few chronic diseases. They take a few medications each. I don’t live near them, but I want to keep an eye on them. Together allows us to share information and nudge each other. I can see when my dad took his medicines or if he has an appointment coming up. It really keeps me in the know, it gives me a lot of peace of mind,” Dua said.

Dua explained the app also uses AI to gather a user’s vitals.

“The vitals feature uses spectrums of color within your skin to check your blood pressure. You take the selfie and within 60 seconds you have a reading of your blood pressure, your heart rate, your heart rate variability and your oxygenation levels,” she said.

The app recently released a new feature aimed at detecting mental illness.

“When you talk into the Together app, it will literally listen to the intonation – the sound of your voice, the way you’re saying things – and make an assessment on your mental vitals. Are you experiencing or do you sound as though you’re anxious? Do you sound as though you’re depressed?” Dua shared.

Michael Pencina is the chief data scientist for Duke Health. He said the use of AI to make healthcare more manageable is a “potentially great advance.

“It’s very hard, regardless of your age, to keep track of everything that you need to be doing, that your doctor prescribes, or you need to improve health,” he said.

Duke Health currently uses similar technology to help diagnose autism in children.

Pencina said the use of AI in this way is an example of moving from “operational application of AI to clinical diagnostic and prognostic.” It’s a move he said needs to be handled with caution.

“I strongly believe there is potential for clinical AI, but the bar is higher on a number of levels. The performance of these algorithms has to be thoroughly tested and found to be good and accurate because we don’t want misdiagnosis in either direction,” Pencina said. “Also with imaging data, there are known examples of bias and health inequity related components. So while I see potential here, that part of the functionality really needs thorough testing.”

Along with ample testing, Pencina said he also believes there should be national standards and guidelines adopted to help with safety when it comes to the use of AI.

“That’s what worries me the most in our current ecosystem. There is a lot of potential and a lot of enthusiasm. I kind of call it a gold rush, we see the gold of AI,” Pencina cautioned. “There is not as much attention paid to the evaluation and assurance that it really serves the human person. I think that’s the focus that we as society need. That’s where I’m kind of leading our Duke AI health efforts that that that we’re making.”