Background and Discovery

I had the unfortunate luck of completely tearing my ACL and going through physical therapy for 3+months after surgery.  During this time, I quickly became familiar with the short comings of physical therapy as well as some of the issues PT's commonly face and its effect on patients recovery.

This lead to me think about how to improve the UX for both the patients as well as the Physical Therapists utilizing new and emerging technology, in particular OpenCV.


Initial Research

I chatted with my PT as well as the other PT's at the office asking them about their jobs, frustrations, things that make their jobs harder as well as what they thought would help them do their jobs better. Further, I asked them about their patients and their patterns, behaviors, and frustrations with PT in general.

In addition, I was patient at the time, so I got to experience all the short comings and modern PT and the ineffectiveness of some of the typical procedures and methods. 


Gap Analysis@2x.png

Gap Analysis

Doing research on if other tools existed, I did not find any (2009).  OpenCV for video was in its early stages and other than in-office or paper instructions, the tools available for PT was non-existent. Further, the benefits of each PT method (in office vs at home) were quiet different. 




Through my discussions with PTs, I created two basic personas: the Therapist and the Patient. My focus for this project would be on the patient and creating a tool that would help them perform their exercises at home properly and provide feedback and metrics.



OpenCV Engineering

My experience with OpenCV allowed me to think of how to engineer a solution to capture body movements and calculate the movement against a pre-determined limit. Because the technology was limited in how to process video, there was the constraint to do this seamlessly and in real-time. Below are some of my engineering proposals.

Object Recognition | My first approach was to try to set up the program to recognize distinct body features (face, hand, foot) to create a rough wireframe of the body and its major joints. At the time, the technology wasn’t robust enough to accurately recognize these body features accurately and reliably.  In addition, for some of the anticipated exercises, these body parts may become obscure.

Multiple Marker | Another approach would be to put markers on the three body parts and connect a line to created the desired angle and ROM information. This method worked well in testing, however, there was some issues with tracking multiple objects and losing the joints angle (for instance, when the wrist is near the shoulder, the two markers are read as a single one)

Single Marker | In my quest to simply the tracking process, i decided to remove the number of tracking points from 3 to 1.  I would draw a red circle on screen which depending on the exercise, the user would then line up a specified body part.  This would act as a reference point to compare the single marker and then an angle could be calculated.



OpenCV Engineering@2x.png

Prototyping and Testing


Multiple prototypes were created and tested with set goals.  

Prototype 1 | 

  • Video Tape a person exercising
  • Track the angle in which a specified body part is moving
  • —Determining target angle: 3 levels
  • Give feedback if the user exceeds the target angle

Prototype 2 |

  • Refine robustness of capture for real world conditions
  • Refine Feedback system

Prototype 3 |

  • Add multiple exercises to system

User Testing


Once I had a workable and robust prototype, I then tested users on it to get feedback and identify issues. Ideally, I would have wanted to use actual PT patients, but from an ethics standpoint, this was not done. Instead, I went to the local gym and let people play with it while doing the exercises I had set up. 

At this step,I had user use the prototype and think-aloud to gathering insights about how users felt about it and what features they would want in a more polished product.  This would be used to further refine the prototype and expand its capabilities. 

Further, I wanted to test to make sure the technology and user interaction would work together. I was concerned by having a person position themselves relative to a marker on the screen would cause problems.  However, I found this was received well and considered helpful as it gave them a set starting point for each exercise!

While testing was generally positive, I had to be cautious about novelty effects and feature creep.