While on a flight from New York to Paris, Justin Tan’s father suffered a stroke and later woke up in a Parisian hospital, half paralyzed. He was an active, well-known doctor in Montreal, but his doctors told him he may never be able to return to work. After he racked up more than $250,000 in rehabilitation costs following the stroke, Tan and several of his engineering friends decided they could build a better system.

“We wanted to improve motivation so patients can be excited and engaged in the programs,” said Daniel Schacter, COO and co-founder of healthcare startup Jintronix.

To address engagement, accessibility, and cost-effectiveness of stroke rehabilitation programs, Jintronix created an gamified exercise platform that provides immediate feedback so patients and clinicians can participate in the sessions remotely or at a hospital. The software utilizes Microsoft Kinect combined with Windows computers.

Coincidentally, 7,000 miles away, a collaborative team of academic and Microsoft researchers in Asia were creating a similar program for stroke victims. Stroke Recovery with Kinect is a partnership with Microsoft Research Asia and Seoul National University, with funding from the Korean Government Collaboration Program.

“It’s a great example of how a grand societal challenge can find the greatest minds working on a similar solution, and in this case to revolutionize stroke recovery,” said Miran Lee, senior manager of Microsoft Research Connections.

Using the Microsoft Kinect software development kit (SDK) that hooks up to a PC, twenty sensors capture movement from the body’s 48 skeletal points on a 3D camera. With Kinect’s capture technology, Jintronix and Microsoft’s stroke recovery system created cost-effective programs for clinicians and patients. Therapists can use them at the office, enabling them to see more patients at a time, and patients have cheaper, efficient therapy sessions if they use it at home.

Microsoft’s Stroke Recovery with Kinect has three main programs: one evaluates manual dexterity and coordination with a timed game in which patients pick up blocks and place them in a box; another challenges them to achieve a target body position; the third is an outer-space game that assesses reflexes. All three provide them with immediate scores and reinforcement.

WATCH: How Stroke Recovery with Kinect works (or watch below)

 

The Jintronix games were developed after the team researched which gaming exercises and sports best fit with traditional therapy poses. For instance, skiing allows them to assess weight-shifting, and “Fish Frenzy,” one of their most popular games, moves patients through various exercises while their fish is on the hunt for food.

Currently, the Jintronix software is being beta tested at 60 clinics and hospitals around the world. The team is waiting on FDA approval in the U.S., with the expectation that the system should be available by April at the latest. So far, the team has built seven games that work smoothly. They plan to release two per month, which will automatically upload to the software.

Overall, both software programs have reported positive feedback from patients and clinicians. However, the system remains a work in progress.

“While we are very excited about the breakthroughs achieved to date, this remains a research project and we are still evaluating our next steps,” Lee said.

The Microsoft researchers plan to incorporate social media so patients can communicate with one another while working out, in order to build camaraderie. They also hope to use technology such as Windows Azure for big data analysis to integrate machine learning into the system.

One of the biggest ongoing concerns will be HIPAA privacy compliance. Schacter said it is always part of the conversation with this system, which has been dubbed “telerehabilitation.” But the software is double-encrypted, and the Jintronix team can’t even see patients’ names or other personal information themselves.

“Of course, nothing is infallible, but we are doing it as least as strong as we can within our ability,” he added.

According to Schacter, the Jintronix team is also developing their software to work with the next-gen Kinect for Xbox One (the SDK will be available later this year). This new version will allow them to detect facial expressions and elevated heart rate. In the case of pain or fatigue, the system automatically adjusts the intensity of the game for the patient.

The Microsoft researchers remain excited about the possibilities of using Kinect to improve the healthcare field. “These patients have suffered paralyzing effects of strokes and we can see the potential of a gesture technology, like Kinect, providing an easy and cost-effective way to help get these people back on their feet. The opportunities for this prototype are exciting and we look forward to seeing where this research takes us.”