Abstract
This paper presents a multimedia e-health framework to conduct therapy sessions by collecting live therapeutic data of patients in a non-invasive way. Using our proposed framework, a therapist can model complex gestures by mapping them to a set of primitive actions and generate high-level therapies. Two inexpensive 3D motion tracking sensors, a Kinect and a Leap, are used to collect motion data of a given subject. Data can then be displayed on screen in a live manner or recorded for offline replaying and analysis. The system incorporates an intelligent authoring tool for therapy design, and produces live plots that show the quality of improvement metrics for a given patient.